UNIXy Goodness » OpenSolaris

Virtualization terms

Chris Quenelle — Wed, 16 Jun 2010 01:23:26 +0000

Okay, before I forget, I’m writing it all down.

We have to test against all this stuff, and it’s becoming more and more convenient to use virtualization as a way to share lab resources, so I figured I’d go make sense of all the terminology that’s flying around. I understood 80% of it, but I could never understand all of it at once. A lot of this was extracted from Wikipedia.

Here are the things that affect my life: Xen, VirtualBox, VMWare, LDOMs, Zones, Containers.

Hypervisor : Software that emulates a hardware platform, so that Operating Systems can run on top of it, as if they had hardware to run on.

OS Virtualization: When you have one OS (one kernel) running multiple user-spaces. Applications think they are on separate machines.

There are two kinds of Hypervisors, some run directly on hardware (Type 1), and some run as applications (Type 2).

With those terms defined, here is a description of the technologies, features, products that I listed at the top:

Hypervisors:
- Running on hardware – Type 1 Hypervisor
  - Xen: Hypervisor that runs on hardware, supports x86 (aka Sun xVM)
  - LDOMs: Hypervisor that runs on hardware, supports SPARC
- Running as an application – Type 2 Hypervisor
  - VirtualBox: Hypervisor that runs as an application, supports x86
  - VMWare: Hypervisor that runs as an application, supports x86

OS Virtualization
- Solaris Containers/Zones

The terms “zone” and “container” seem to interchangeable. I have not found a source that is both clear and authoritative that can tell me the difference.

Zones are capable of running different versions of Solaris inside one Global OS instance.

There are lots of things I glossed over here, but my goal was keep it short and sweet.

Trivia:

You can run a specific old version of Linux inside a Solaris zone.
The VMWare company probably supports products on other chips than x86
There are lots of differences between the features of Xen and LDOMs that I didn’t discuss

Which version of Sun Studio do I have installed?

Chris Quenelle — Fri, 14 May 2010 17:47:09 +0000

Recipes for supported packaging formats

Sun Studio is available on three different packaging systems. Here are some examples that show you how to get information about the Sun Studio packages on each kind of system.

IPS packaging system – on OpenSolaris
SYSV packages – on Solaris 10
RPMs – on SuSE and RedHat Linux

If you want to know what version of a Studio component you’re using, the steps are shown below. The compiler or tool you’re interested in might be on your search path (you can find the location with “which cc”) or you might already know the full path. Once you have the full path, here are the things you might want to find out:

Find out the name of the package containing that binary.
Dump out information about that package.
Optionally look for other packages from the same Studio release, to see what else is installed.

Generally the multiple packages that make up Sun Studio will use a similar naming convention. In the currently available releases, these package names are cryptic.

Sun Studio 12 update 1 installed on Solaris 10

What version is built into the binary?

% /opt/sunstudio12.1/bin/cc -V
cc: Sun C 5.10 SunOS_sparc 2009/06/03
usage: cc [ options] files.  Use 'cc -flags' for details

Which package is that binary in?

% pkgchk -l -p '/opt/sunstudio12.1/bin/cc'
NOTE: Couldn't lock the package database.
Pathname: /opt/sunstudio12.1/bin/cc
Type: symbolic link
Source of link: ../prod/bin/cc
Referenced by the following packages:
SPROcc
Current status: installed

What other packages are installed?

% pkginfo | grep SPRO
application SPROatd                          Sun Studio 12 update 1 Advanced Tools Development Module
application SPROcc                           Sun Studio 12 update 1 C Compiler
application SPROcmpl                         Sun Studio 12 update 1 C++ Complex Library
application SPROcpl                          Sun Studio 12 update 1 C++ Compiler
application SPROcplx                         Sun Studio 12 update 1 C++ 64-bit Libraries
...

Sun Studio 12 update 1 installed on OpenSolaris

What version is built into the binary?

% /opt/sunstudio12.1/bin/cc -V
cc: Sun C 5.10 SunOS_i386 2009/06/03
usage: cc [ options] files.  Use 'cc -flags' for details% /opt/sunstudio12.1/bin/cc -V

Which package is that binary in?

% pkg search -lp /opt/sunstudio12.1/bin/cc
PACKAGE                                   PUBLISHER
pkg:/developer/sunstudio12u1@12.1.1-0.111

What other packages are installed?

% pkg list | grep -i studio
developer/sunstudio12u1                       12.1.1-0.111    installed  -----

Sun Studio 12 update 1 installed on SuSE 11 Linux

What version is built into the binary?

% /opt/sun/sunstudio12.1/bin/cc -V
cc: Sun C 5.10 Linux_i386 2009/06/03
usage: cc [ options] files.  Use 'cc -flags' for details

Which package is that binary in?

% rpm -qf /opt/sun/sunstudio12.1/bin/cc
sun-cc-12.1-1

What other packages are installed?

% rpm -qa | grep sun- | head
sun-lang-12.1-1
sun-idext-12.1-1
sun-mr3m-12.1-1
sun-prfan-12.1-1
sun-stl4h-12.1-1
sun-cplx-12.1-1
sun-dbxx-12.1-1
sun-pls-12.1-1
sun-dwrfs-12.1-1
sun-rtmx-12.1-1
...

Notes

The excessively terse naming convention is because of the ancient restrictions in AT&T System V UNIX that limited package names to 9 characters. Sun also made an early decision to prefix packages names with 4 letters to mark the part of the company that was releasing the packages. In all fairness, Sun was trying to invent a scheme where outside software vendors could reasonably choose package names without accidentally conflicting with any of the Sun packages. That’s difficult to do in only 9 characters. On OpenSolaris, you can see that we merged everything into one package. Because the friendly new packaging system is one of the highlights of OpenSolaris, we didn’t want to confuse new users with the multitude of small packages we have for Sun Studio.

Hopefully, this information will be useful in a variety of circumstances. Inside the Studio team, we need to go back and forth between all three packaging systems, and it’s not easy to remember the right system commands to work with the packages on a given system. In the support team, one of the first things they ask a customer is which version of the Sun Studio software they are running. It’s also possible to install subsets of Sun Studio, so you may want to know which tools are currently installed.

Note: Studio will actually run fine on lots of different versions of Linux, including distributions that don’t use RPM as their native package format (like Ubuntu). The tarball downloads are useful for those Linux distributions.

OpenSolaris VNC update

Chris Quenelle — Thu, 22 Apr 2010 18:44:23 +0000

I use VNC connections as part of my regular daily routine, and I ran into another bug a few days ago, so I figured I’d post a summary of the current bugs that are currently afflicting me, and the workarounds for them.

First is a really interesting bug where the less-than-sign turns into a greater-than-sign. Really. I’m not joking. The comma and period work fine, it’s just the less-than-sign. The public bugid for this is 14729. The workaround is to run xmodmap when you start your VNC session (I put it in ~/.vnc/xstartup).

xmodmap -e 'keycode 94 = comma less'

The second bug is that the server crashes whenever you turn OFF the capslock key. The public bug for this one is 14397. The workaround I found for that is the following:

vncserver ... -RemapKeys 0xffe5->0x00

In other words, you try to disable the capslock key, and the result is a capslock key that works correctly. I have no idea what the implementation is doing.

I’m using a Mac client these days as a desktop, so I’d be interested in hearing from anyone using windows VNC clients to see if they’ve hit these same bugs or not. Of course, I do live on the bleeding edge of OpenSolaris. The last few biweekly dev releases have been internal-only, but I think both of these bugs are in public OpenSolaris releases.

OpenSolaris build 129 has problems with filemerge fonts

Chris Quenelle — Wed, 16 Dec 2009 02:48:08 +0000

I’ve been staying more up-to-date with Solaris dev builds since they started publishing builds nigh-biweekly. That means I get to experience all the joyful brokenness of a system in development. Build 128 introduced a problem with VNC which can be worked around by disabling the “tight” encoding (bugid 6906660). And Build 129 introduced a problem with filemerge fonts (bugid 6909040). Both have pretty easy workarounds described in the bug descriptions. The release notes for opensolaris are not widely redistributed, so if you’re using the dev repository from outside Sun, make sure you’re on the osol-announce@opensolaris.org mail alias.

Downgrade OpenSolaris

Chris Quenelle — Fri, 13 Nov 2009 16:06:01 +0000

This is a fairly frequently asked question inside Sun, and now that the biweekly builds are available outside Sun, there are probably more people who are interested. If you want to create a Boot Environment with a specific build (not the latest one) in OpenSolaris, here are the steps to use:

beadm create snv122
beadm mount snv122 /mnt
pkg -R /mnt install pkg:/entire@0.5.11-0.122
bootadm update-archive -R /mnt
beadm umount snv122
beadm activate snv122

Caveat Emptor and all that stuff. This recipe comes with no warranty, but I’d be glad to make it more accurate or up-to-date if you let me know of improvements.

Installing Solaris patches on top of a tarball.

Chris Quenelle — Wed, 03 Jun 2009 23:47:12 +0000

There are times when you have a copy of some Solaris packages installed, but the pkg database is not available. This can happen when you relocate the installation directory manually, or if you’ve placed the files on a non-Solaris NFS server, or if you backed up your /opt directory, and restored it aftera complete OS reinstall. So what happens if you want to update your software with the latest Solaris patches? It’s actually not too hard to tear apart a Solaris patch and get at the files, so I wrote a script to help out in these situations.

The script can compare your install directory with a patch to see if the files in the patch match the versions in your install directory. It can also install the files from the patch into your install directory. The patch arguments can be either zip files or unzipped patch directories.

This checks if any patches look like they’re not applied.

% patchraw -n DIR PATCH PATCH ....

This installs all files in all patches

% patchraw DIR PATCH PATCH ...

I developed the script using the sparc version of Sun Studio 12 as an example, and did no other testing. So caveat emptor. If you want to use the script, make a backup copy of the install tree. There are some comments in the source, and some commented out print statements that might be useful.

In order to get the latest patches for Sun Studio, I recommend using the Patch Check Advanced script by Martin Paul. You can get it here: http://www.par.univie.ac.at/solaris/pca/

The pca script is a swiss army knife for Solaris patches. In our case we only need to make limited use of it download a set of patches. It wants to use ‘showrev’ to check your system, so you can’t run it on OpenSolaris for now. To get the list of the latest patches available for Sun Studio, there are two ways. There is a page listing Sun Studio 12 patches., but it can be out of date. At the time of this writing, it’s missing a locale related patch.

You can also use the pca script to download the latest index of patches and search through it. Sun’s patch index doesn’t split out platforms very clearly, so I had to tack on some greps. Here is the command that worked for me to find sparc patches. This process only works if the patch synopsis follows some sort of mostly sensible convention.

% pca -l -p 'Sun Studio 12: ' total | grep -v Linux | grep -v RHEL

Put that output into a file like “patchlist”, then use pca to download the patches. You’ll need a free login/password for the sunsolve site.

% pca -a -d patchlist
Please enter Sun Online Account User:  (your sunsolve/SDN login)
Please enter Sun Online Account Password:  (your password)

If you find this useful please drop me a line…

The script is here: SCRIPT.

Extra Caveat: The Sun Studio 12 tarball that’s available from sun.com leaves out the performance libraries. So installing all the patches carries the risk of installing part of the performance libraries, but not all the files. This may cause incorrect linking of programs that try to use the performance libraries.

Visualizing dynamic library dependencies

Chris Quenelle — Fri, 22 May 2009 15:59:01 +0000

Darryl Gove has been working on graphical display of shared library dependencies. It seems useful for performance analysis and debugging of dynamically linked applications.

He did one for StarOffice and for Firfox and Thunderbird.

StarOffice : http://blogs.sun.com/d/entry/drawing_libraries_neater_eye_candy
Firefox, Thunderbird : http://blogs.sun.com/d/entry/graph_of_libraries_used_by

Finding the right package in OpenSolaris?

Chris Quenelle — Thu, 08 Jan 2009 03:47:53 +0000

Ubuntu has a cool feature called command-not-found where they publish an index of all the binaries on the system, and the package that each one belongs to. When you combine this with the bash command-not-found hook, you get a default environment that responds to a missing command by telling you what package to download. (Sorry, too lazy for screenshots…)

Anyway, this feature would be a good use for the new pkg command. You could write a script that creates an index of all binaries, and then store that index in the user’s home directory. Another script could react to the command-not-found hook in bash and tell the user something useful.

In fact, you could almost do this interactively by querying the pkg database, except that when I query ‘dbx’ (for example), I get 37 hits. Most are for the different versions of the right package, and a few are for a package-alias of some kind. A script that got that answer would need to apply some clever heuristics to do the right thing.

It seems like a good afternoon project, but I haven’t had a spare afternoon in quite a while. The pkg-discuss@opensolaris.org alias can offer help with pkg issues.

Any takers?

Goodbye Solaris 9 (for Sun Studio)

Chris Quenelle — Wed, 20 Aug 2008 23:48:58 +0000

We’re making the internal transition to building Sun Studio on Solaris 10 (instead of Solaris 9). This is a big deal because the product bits immediately become useless on any Solaris 9 system. There’s a new libm.so.2 library that became available on Solaris 10, and if you depend on it, you can’t run on Solaris 9. It’s a challenge making sure our vast ocean of loosely maintained lab machines is ready for the change. The good news is we get to use newer, faster hardware.

I’ll make this post short because I’m using ScribeFire for the first time in forever, and I don’t trust it. I can’t believe blogging is still this hard.

SXDE 5/07 — Solaris Express Developers Edition

Chris Quenelle — Tue, 12 Jun 2007 02:00:22 +0000

A new update to Solaris Express Developer Edition is out. It has the latest and greatest Solaris Nevada build, along with all the development tools you could ever want. Well, I dunno, I can want a lot, but there’s a bunch of tools in there. My favorite parts are Sun Studio 12 and Solaris Nevada, but on the same DVD you get Netbeans 5.5 fully loaded, including the Creator RAD web development tool and other goodies.

I’ll take this opportunity to review some of my own personal favorite features of Sun Studio 12 and Solaris Nevada.

Sun Studio 12

The part of Sun Studio 12 that I was most directly involved with recently is the compiler support for the dwarf debugging format. We finished support for C++ and Fortran using dwarf, and now it will be much easier to maintain compatibility between Sun tools and the gcc tool suite. Dwarf also has “location lists” which we can use in the future to support local variables in optimized code. We were hoping to get this feature in for Sun Studio 12, but it didn’t make it in time. Look for optimized locals support in a future Sun Studio Express release.

One of the most visible changes in Sun Studio 12 is the new IDE. The debugger GUI has been rewritten to use the latest Netbeans, and to be much more like Visual Studio in some ways. Boy you should have heard the discussions we had about that! But it turns out that Visual Studio seems to be geared towards beginning developers, and that’s something we need to get better at. There are lots of changes in the new version, and lots of things for us still to implement, but give it a try, and let us know what you think.

Other major new features include a new Thread Analyzer tool for finding data races in threaded programs, and the Linux version of our product now includes all the compilers! Having Linux compilers allows more people than ever to give Sun Studio a try, and see what they’ve been missing. Especially a “real” Fortran compiler!

Solaris Nevada

A lot of the Nevada features I like are also in Solaris 10 by now, but I’m not sure which. I’ve been starting to use ZFS filesystems on my group’s file server, and I just remembered one of my favorite new features. It’s the -h option. That’s it, just one option. In reconfiguring filesystems, I end up running ‘du’ and ‘df’ commands over and over and over. It’s always a pain in the butt because the output of those commands is normally in blocks. What the hell is a “block”? I never quite trusted the OS to give me an implementation independent answer. But the -h option stands for “human” format. You get size numbers displayed as 27.3K for kilobytes or 52G for gigabytes, and the number is rounded to an appropriately small numbr of significant figures. I never realized how annoying those block counts and sig figs were until I started to get used to the -h output.

I also like a lot of the Gnome desktop features in Solaris. There’s a feature called gDesklets that I was reading about the other day. I had them confused with panel applets (which are also cool), but desklets let you put clever little clocks or graphs on your background in a similar fashion to what Apple popularized with their “Gadgets” feature. You can code your gDesklets in a scripting language (python), so they should be easy to create.

Another thing that comes to mind is the remote desktop support in Gnome (aka Vino). Lots of people where I work use VNC for working from home. But if you go back and forth between home and office, it’s not really seamless to use the vncviewer program when you’re already on your local desktop. So people will shut down all their apps when they go back and forth. There’s a ‘remote desktop’ feature that allows you to start up a VNC session and transfer all your desktop windows into the remote server. Then when you shut down the VNC session they go back to being local windows. That feature makes moving between the office and another location so much easier.

Of course, these are the just some of the features. The new Network Automagic configurator, wider support for wireless drivers, Gnome 2.16, etc. For a list of lots more new stuff in SXDE, look here.