Solaris

I had an interesting discussion with a blog reader and customer via icq (oh wonderful new world of business). It started with a question about the J4500 and it ended in the the discussion, if MAID is really a viable way for storing data of if it´s just one of this hypes working their way through the storage business every year.

MAID? What is MAID? MAIDs are Massive Arrays of Idle Disk. The idea is to save power by switching of disks in the case they aren´t in use. You can buy such components from companies like Copan.

I´ve uploaded another early pdf of the tutorial. I´ve added some text and the first to figures. Kerberos is just one of the topics where you should explain only the configuration or you have to explain it almost completly. You can´t explain just a few things and still hope your explanation makes sense.

In the Webshop of Dell you can choose Solaris x86 as an option. This is an example from the Dell PowerEdge R905:


But it costs $231 more than the subscription at Sun

With the integration of PSARC 2007/347 NFS/RDMA Solaris got a new implementation for NFS over RDMA:

This integration brings the NFS/RDMA Solaris Implementation up to datewith the latest IETF Drafts. Specifically, it removes support for the initial NFS/RDMA implementation (version 0) and adds support for rpcrdma [1] and nfsdirect [2] (version 1).

I forgot to post the conclusion and the table of content when i published the JET tutorial some weeks ago. Okay ... here they are. This is a real long tutorial. 16 parts in total - the longest tutorial so far:

• Part 1: Introduction
• Part 2: Basic Jumpstart
• Part 3: Files controlling the installation
• Part 4: Jumpstart FLASH
• Part 5: The Toolkit
• Part 6: Prerequisites
• Part 7: Installing the JET Server
• Part 8: Preparing the first installation
• Part 9: Basic installation of a new system
• Part 10: A polished basic installation
• Part 11: Mirrored boot disks
• Part 12: Automatic hardening with SST
• Part 13: Deep dive to post installation
• Part 14: Using flash archives
• Part 15: Using flash archives for recovery
• Part 16: Conclusion

I hope this tutorial is useful for you!

JET and Jumpstart are incredible powerful tools to ease the installation of your solaris systems. You can install your systems via network, configure and customize them. With tools like Jumpstart FLASH it´s easy to clone systems for large computing cluster.

Even when you have only a few systems, it´s a good practice to install you systems by a Jumpstart server. A bare metal recovery of your system is much easier, you can use a common template to setup your system and beside the hard disk space the jumpstart server needs only resources at the moment of the installation.

I´ve integrated the CacheFS and the JET tutorials in the standard release of the LKSF book, thus now all links refering to the released version of the book should deliver the version 0.28 of the LKSF book (with the date of the 18.08.2008 on the titlepage). You can download it here.

CacheFS is one of this hidden gems in Solaris, it´s even that hidden, that it´s in sustaining mode since 1999. That means, that only bugfixes are made for this features but no new features found their way into this component since that day. In the recent days there was some discussion about the declaration of the End-of-Feature status for CacheFS which will lead to the announcement of the removal of CacheFS. After a few days of discussion the ARC decided in favour of the removal.

Let´s take an example: Let´s assume you have 50 webservers that serve static content or that serves and executes .php files. You may have used scp or rsync to syncronize all this servers.

With CacheFS the workflow is a little bit different: You simply put all you files on a fileserver. But you have noticed, that there was a large load on this system. To reduce load to the fileserver you use your new knowledge to create a cache on every server. After this you saw a lot of requests for directory and file metadata for the documents directory. You know that this directory is provisioned with the changes only at 3 o´clock in the morning. Thus you write a little script, that checks for the file please_check_me and starts cfsadmin -s all on the client in the case of it´s existence and it will check at the next access to the files if there´s a newer version.

Let´s assume you share a filesystem with static content (for example the copy of a cdrom) or a filesystem that changes on a regular schedule (for example at midnight every day). So it would pose unnessesary load to network to check the consistency everytime again.

CacheFS knows a special mode of operation for such an situation. It´s called on demand consistency checking. It does exactly what the name says: It checks only the consistency of files in the cache, when you tell the system to do it.

Okay, we have a working CacheFS mount but where and how is the stuff cached by the system. Let´s have a look at the cache.

Okay ... using CacheFS is really easy. Let´s assume, you have an fileserver called theoden We use the directory /export/files as the directory shared by NFS. The client in our example is gandalf.

The history of CacheFS

Sun didn´t introduced this feature for webservers. Long ago, admins didn´t want to manage dozens of operating system installations. Instead of this they wanted to store all this data on a central fileserver (you know ... the network is the computer). Thus netbooting Solaris and SunOS was invented. But there was a problem: Swap via Network was a really bad idea that days (it was a bad idea in 10 MBit/s times and it´s still a bad idea in 10 GBit/s times). Thus the diskless systems got a disk for a local swap. But there was another problem. All the users started to work at 9 o´clock ... they switched on their workstations ... and the load on the fileserver and the network got higher and higher. They had a local disk ... local installation again? No ... the central installation had it´s advantages. Thus the idea of CacheFS was born.

CacheFS is a really old feature of Solaris/SunOS. It´s first implementation dates back into the year 1991. I really think you can call this feature matured

CacheFS in theory

The mechanism of CacheFS is pretty simple. As i told you before, CacheFS is somewhat similar to a caching web proxy. The CacheFS is a proxy to the original filesystem and caches files their way through CacheFS. The basic idea is to cache remote files locally on a harddisk, so you can deliver them without using the network when you access them the second time.

Of course the CacheFS has to handle changes to the original files. So CacheFS checks the metadata of the file before delivering the copy. If the metadata has changed, the CacheFS loads the original file from the server. When the metadata hasn´t changed it delivers the copy from the cache.

The CacheFS isn´t just usable for NFS, you could use it as well for caching optical media like CD or DVD.

There is a hidden gem in the Solaris Operating Environment called CacheFS, solving a task many admins solve with scripts.

Imagine the following situation. You have a central fileserver, and let´s say a 40 webservers. All this webservers deliver static content, this content is stored on the harddisk of the fileserver. Later you recognize, that your fileserver is really loaded by the webservers. Harddisks are cheap, thus most admins will start to use an recursive rcp or an rsync to put a copy of the data to the webserver disks.

Well ... Solaris gives you a tool to solve this problem without scripting, without cron based jobs, just by using NFSv3 and this hidden gem: CacheFS. CacheFS is a really nifty tool. It does exactly what the name says. It´s an filesystem, that caches data of an other filesystem. You have to think about it like a layered cake. You mount a CacheFS filesystem with an parameter that tells CacheFS to mount another one in the background.

An interesting new feature had it´s flag day. With PSARC 2008/393 a feature called "zfs primarycache and secondarycache prop" found it´s way into Solaris for build 96. With this properties you can control, what ZFS caches in the ARC and the L2ARC.

The introduction of the 'primarycache' property will allow administrators the ability to control what is cached in the ARC. The possible choices are: all, none, and metadata. If 'all' is chosen, then both user data and metadata are cached. If 'none' is chosen, then neither user data nor metadata is cached. If 'metadata' is chosen, then only metadata is cached. The default is 'all'.

The introduction of the 'secondarycache' property will allow administrators the ability to control what is cached in the l2ARC. The possible choices are the exact same as 'primarycache'. The default is also 'all'.

This is interesting for situations, when you know that your application is caching data as well (thus it makes no sense to cache it twice) or you aware of the fact that caching is just waste of time because of the access pattern.

You can find details about this feature in the PSARC discussion for 2008/393 and in the specs.