Random Ramblings About BSD on MacOS X (Part 2)

By Jeffrey Carl and Matt Loschert

Daemon News, December 2001

This is the first chapter in a series of observations, representing the adventures of a couple of BSD admins (one with a lot of prior MacOS experience, the other with more on the BSD side) poking around the command line on an iBook laptop running Apple’s  Mac OS X Public Beta. We’ll attempt to provide a few notes and observations that may make a BSD admin’s work with Mac OS X easier.

Note: Again, as members of the FreeBSD Borg collective MIND-STABLE, we’ll refer to our various comments/sections by the singular “I.” This also prevents either of us from admitting which dumb parts of the article were ours specifically. 🙂

New Information from Last Time

The quality and quantity of the feedback that we (OK, Borg segfault here) received after part one of this series was fantastic. Thanks to everyone (you know who you are) who wrote in to clear up points or to show us a better way to do things. Aside from a few flames on Slashdot (“You are stupid and blow goats!”, “Duh!”, and “N4t4li3 P0rtm4n r00tz y00”, etc.) the feedback was very helpful.

The number of NeXT gurus (and some BSD overlords, as well) out there who came to the rescue to correct mistakes and offer answers to the questions posed in the article was amazing. This time around, our readers who are l33t NeXT h4Xors, BsD r00tz or k3wl m4c d00dz are still invited to help clear up the questions or postulations herein on Mac OS X. So, to follow up and answer some of the questions posed by the first article, here are some of the best responses received:

------------------------------------------------
From: Dwarf
Subject: Daemon News Article
Not sure if any of this is new info to you guys....
OSXPB inherits a lot of "philosophy" from OS X Server. Thus, the lack of logging that occurs. Apple seems to have come down in favor of system responsiveness versus use monitoring. Their rationale for turning off logging(for almost everything) by default is that it impacts network thruput. If the logs are something you need, they can all be enabled from the command line, but network (and probably GUI) responsiveness will likely suffer as a result Apple also seems to have made several assumptions about how OS X (any flavor) will be used.
Apparently their idea is that it will provide services to a LAN and be hidden from the world by a firewall of some sort, thus the default enabling of NFS and having NetInfo socketed to the network by default. Since NetInfo is a multi-tiered database, your "local" NI Server may also be the "master" NI Server for subnetted machines, while being either a clone or a client of a still higher level NI Server. So, they hook it to the net by default. This also provides the mechanism by which other machines can automatically be added to your network. At bootup, each machine tries to learn who it is and where it lives by querying the network for its "mommy" (a Master NI Server). If it finds one it accepts the name and IP that server furnishes and initializes itself accordingly. If it doesn't, it uses the defaults from its initialization scripts. Getting this all to work painlessly is one of the things about which the NetInfo documentation is pretty obscure. Owing primarily to the fact that it is written in terms of tools that no longer exist as separate entities, but have been combined into more powerful tools. Further, if NFS is properly setup, each machine will automount the appropriate NFS volumes at startup. Another area where making it work is not clearly explained. I will only touch on the confusion that exists about setting up MailApp and making it work. Another documentation shortcoming.
Another facet of operation that isn't clearly explained is the Apple philosophy about how the file tree is organized. Their thinking is that users should only install things into the /Local tree. /System should be reserved for those things that administrators add. My guess is that naïve users will be fine so long as they confine themselves to operating within the GUI, as the GUI tools seem to be pretty smart about where they put things. But, if those users start installing things from the CLI....
A problem area about which not much has been written is the fundamental incompatibility between the Mac HFS and HFS+ filesystem and BSD. Mac files are complex, containing both a Data Fork and a Resource Fork. BSD knows nothing about complex files and thus when BSD filesystem tools are used to manipulate Mac files, the resource forks get orphaned. See: http://www.mit.edu/people/wsanchez/papers/USENIX_2000/ for a better explanation of this. This may be the source of a longstanding OS X Server problem whereby the Desktop Database process eventually goes walkabout and consumes over 90% of the CPU.

Authors’ Note: I’ve received a large number of comments about how the existing state of Mac OS X/Darwin documentation sucks. Frankly, I agree – that’s why I/we wrote these articles. While there’s a certain thrill to “spelunking” a new OS, it’s not what an administrator would like to have to be doing in their spare time. However, it’s hard to point a finger at Apple, since they’re currently under a hiring freeze after their recent absurd stock devaluation (post-Q3 results), and they would be perfectly right to have every man/woman/droid/vertebrate/etc. working on developing the OS rather than documenting it. 

Nonetheless, there is still a significant problem lurking. There are tens of thousands of otherwise non-super-technical folks who have become MacOS gurus through inclination and experience, able to roam around a school or office and fix traditional MacOS problems. At the moment, the folks who (working with the current paltry documentation) can do this for MacOS X are incredibly few, since it requires significant knowledge of MacOS as well as Unix experience – and even then, it’s only NeXTstep mavens who will be truly at home with some of its aspects.

The good folks at Daemon News have provided a space here to try to answer some of these questions, but it’s up to the knowledgeable folks already out there to contribute to sites like Daemon News, Darwinfo, Xappeal, MacFixit, etc. to make whatever knowledge is out there available to the soon-to-be Mac OS X community. If Apple can’t document this OS thoroughly while rushing every available resource to develop it, it’s up to the folks who (at least marginally) understand it to do so, for the good of all its users.

From: Brian Bergstrand <[email protected]>
Subject: Re: Daemon News : Random Ramblings about BSD on MacOS X (part 1)
In your article you said: "(as mentioned, etc, tmp and var are all links to inside /private; I refuse to speculate why without having several drinks of Scotch first)".
The /private dir. is again a part of Mac OS X's NeXT heritage. Originally, the thought behind /private was that it could be mounted as a local drive for NeXT stations that were Net booted. That way you would not have to mount volumes for /etc, /var, or whatever else needed write perms. This also worked well if you booted from a CD. /private, meant data that was to be used only on a specific machine.
HTH.
Brian Bergstrand
Systems Programmer Northern Illinois University
Imagination is more important than knowledge. - Albert Einstein

Authors’ Note: We’re discovering more and more that Mac OS X seems very much like the next revision of OpenStep – with MacOS 9 compatibility and a new GUI thrown in. Not that this is necessarily a bad thing; it just seems like NeXT was the “Addams Family” member of the BSD clan that nobody else noticed, and we’re not sure why. If anyone would like to speculate on the reasons that NeXT’s new ideas were largely ignored by the industry (aside from the typical Steve Jobs-ian tendency to make your computers too expensive for normal people to buy), we’d love to find out more.

------------------------------------------------
From: Daniel Trudell <[email protected]>
Subject: random bsd os x ramblings...netinfo and ipconfigd
[...]
Netinfo is interesting. One thing I noticed is that most of the stuff in the utility applications are like mini versions of netInfoManager. IE I can edit/add/delete users in netInfoManager including root and daemon, and those changes are present in multiple users after the fact, and vice-versa. However, some things still depend on /etc/passwd even in multiuser mode. I installed samba, and I needed an /etc/passwd file for it. i used "nidump passwd .> /etc/passwd" to generate one from netinfo....but there was a twist...some of the users were shadowed, some were not. I s'pose that might be an issue. Also, I was conforming UID's on my box to the company UID's....if somebody with a UID of 500 logs in remotely the machine forgets how to handle users...a reboot fixes this.
[...]
In general, i think there's a consensus about acceptance of netinfo. When you first run around in tcsh, a geek asks themself "what the f**k...this is jacked up, what's up with /etc?", but once you figure out netinfo, a geek says "hey, check this out, it's nifty!"
tack

Authors’ Note: I agree. Still, it will take a while (and some seriously improved documentation [see above]) to get used to.

------------------------------------------------
From: Rick Roe
Subject:Re: Mac OS X article
Well, I'm not the foremost expert on Darwin, but I've learned a few things from "this side of the playground" that might help...
- The "Administrator" issue is Apple's compromise between the single-powerful-user paradigm of Classic Mac OS and the Unix/NeXT multiuser system with it's too-powerful-for-your-own-good root.
An "administrator" has the privileges an upgrading Mac user expects: ability to change system settings and edit machine-wide domains on the disk (like /Applications). However, it still protects them from the dangers of running as root all the time, since they don't get write access to the likes of /etc (except through configuration utilities), or to /System (which is a partitioning that keeps the Apple-provided stuff separate from the stuff you install, like /usr vs. /usr/local).
The inability of "administrator" users to make changes to items at the top level of the filesystem is a bug in the current version.
- Actually, we got NTP support back in Mac OS 8.5, not 9.0 :)
- The developer tools are available separately from Mac OS X through Apple's developer program. The basic membership level is free, and gets you access to not only the BSD/GNU developer tools, but also the cool GUI tools, headers, examples, and documentation out the wazoo. Of course, you can also get a lot of this stuff from the Darwin distribution, too.
- Regarding the list of top-level files and directories in .hidden:
- Desktop DB and Desktop DF are used by Mac OS 9 to match files to their "parent" applications. OS X maintains them for the sake of the Classic environment but only uses them as a fallback, as it has a more sophisticated per-user system for this purpose.
- Desktop Folder is where native OS 9 stores items that show up on the desktop. In OS X, they're in ~/Library/Desktop.
- SystemFolderX was where the BootX file (a file containing info for Open Firmware and some bootstrap stuff to get the kernel started) was kept in previous developer releases. It's elsewhere now.
- Trash is the Mac OS 9 version. OS X uses /.Trash, /.Trashes, and ~/.Trash.
- So you've discovered how cool NetInfo is. I got tired of reading reviews that were just complaints about not being able to edit stuff in /etc to change things. :) Here's some extra info for you:
- There's a convenient GUI utility for editing NetInfo domains,
/Applications/Utilities/NetInfoManager.
- root's password can be changed in NetInfoManager, or the Password panel in System Preferences, in addition to the command line.
- NetInfo is a pretty cool "directory service" for administering groups of computers... one of those unfortunate "best kept secrets" of NeXT... but what's cooler is that, in OS X, it's just one possible backend to a generic directory services API. So it's also possible to run your network using LDAP, or Kerberos/WSSAPI (er, whatever that acronym was), or NDS, or (god help you) Active Directory -- and the user experience for Mac OS X will be the same.
- You might like this... try entering ">console" at the login window.

Authors’ Note: Typing “>console” at the login window (no password necessary) and hitting “Return [Enter]” boots you directly into Darwin, skipping the Mac OS X GUI layer entirely. Sooper cool.

------------------------------------------------
From: Larry Mills-Gahl <[email protected]>
Subject: NetInfo and changing network settings
[...]
One bit that I've been sending feedback on since the Rhapsody builds (pre-OS X Server) is the suggestion that you must reboot to have network settings changes take effect. This is one area in NT that drives me absolutely nuts and I feel like billing Bill G for the time it takes for multiple restarts of every NT or 9X machine you setup!!! Unix seems to have figured this out long ago. The Mac OS has figured this out long ago!!! I appreciate the engineers being conservative because their market is notoriously unforgiving about issues that work, but are not clairvoyant and anticipate how each luser wants the system to work. I hope that they will have this cleared up by release time.
In the interim, here is a script that HUPs netinfo services to get a hot restart.
#!/bin/sh
case `whoami` in
root)
;;
*)
echo "Not Administrator (root). You need to be in order to restart the network."
return
;;
esac
echo "Restarting the network, network will be unavailable."
kill `ps aux | grep ipconfigd | grep -v grep | awk '{print $2}'`
echo " - Killed 'ipconfigd'."
ipconfigd
echo " - Started 'ipconfigd' right back up."
sleep 1
ipconfig waitall
echo " - Ran 'ipconfig waitall' to re-configure for new settings."
sleep 1
kill -HUP `cat /var/run/nibindd.pid`
echo " - Killed 'nibindd' with a HUP (hang up)."
sleep 2
kill -HUP `cat /var/run/lookupd.pid`
echo " - Killed 'lookupd' with a HUP (hang up)."
echo "The network has successfully been restarted and/or re-configured and is now available."

Authors’ Note: Larry gives credit to “Timothy Hatcher” as the original author of this script. You can find the original script at the bottom of the page at http://macosrumors.com/?view=archive/10-00, as well as another script about 3/4 of the way down the page which reproduces (roughly) the very useful MacOS 8-9 “Location Manager” functionality. I don’t like to cite the site MacOS Rumors as any kind of source of reliable info, since it’s 90 percent pretentiously uninformed speculation that doesn’t admit itself as such, but one of its readers did give the Mac community “the scoop” here, as far as I can tell. If Timothy Hatcher or anyone else out there wants to speak up as the original author of this script, please let me know – I’d love to ask you some questions about NetInfo. 😉

————————————————

From: Dag-Erling Smorgrav <[email protected]>
Subject: mach.sym in Darwin
I'll bet you a dime to a dollar the mysterious mach.sym in MacOS/X's root directory is simply a debugging kernel, i.e. an unstripped copy of mach_kernel.

————————————————

From: Paul Lynch <[email protected]>
Subject: MacOS X Daemon News
I can give you a few updates on some parts that might be of interest. In no particular order:
- the kernel (Mach) is only supplied in binary. Most MacOS X admins won't be expected to be able to build a new kernel; that requires a BSD/Mach background (and who's got that outside of Apple?) and a Darwin development system. So building it with firewall options enabled is reasonably smart.
- as well as /.hidden, you will notice that dot files aren't visible in the Finder. .hidden should be looked for in the root of any mounted filesystem, not just /.
- /private is a hangover from the old days of diskless workstations. NeXT had a good netboot option, which meant that you could stash all the local configuration and high access files (like swap - /private/vm/swapfile) in a locally mounted disk. This is all part of the Mach, as opposed to BSD, option.
- MacOS X doesn't only support HFS. It also supports UFS, and that may shed some light on some of the "but HFS does this" quirks.
Paul

Authors’ Note: The inclusion of a distinct /private now seems to make a lot of sense, especially for those who are willing to believe in grand computer-industry conspiracy theories. 🙂 I saw a Steve Jobs keynote in which he showed 50 iMacs net-booting from a single server, showing their abilities as (relatively) low-cost “Network Computers.” And who is a greater believer in NCs than Apple board member (and reputed best buddy of Steve Jobs) than Oracle chief Larry Ellison? No specific documentation of the role of /private has thus far been provided by Apple (as far as I can tell), but the above explanation seems very plausible, leaving open the door for future uses as described.

------------------------------------------------
From: Peter Bierman
Subject: http://www.daemonnews.org/200011/osx-daemon.html
Try 'nicl .'
Then try combinations of cd, ls, and cat.
cd moves around the netinfo directory structure
cat prints the properties of the current directory
ls prints the subdirectories of the current directory
A few minutes in nicl, and NetInfo will make a lot more sense.
Unfortunately, there's no man page yet.
Another tidbit:
In X-GM, volumes will mount under /Volumes instead of at /

————————————————

From: James F. Carter <[email protected]>
Subject: Comments on Random Ramblings about BSD on MacOS X
Why a firewall with no rules? Because the firewall code has to be selected at compile time, but when the CD is burned they don't know what rules the end user will want, and they don't want to lock out any "traditional" behavior, such as the ability to play host to script kiddies with a port of Trinoo for the Mac. I agree that a set of moderately restrictive default rules would be a good idea for the average grandmother, but I can understand the developers' attitude too.
Why have a plain file /tmp/console.log in addition to the syslog? In case syslogd dies. I have this problem on Linux: there's a timing dependency which if violated kills syslogd, and I'm running a driver (you don't want to know the gory details when I suspend the laptop to RAM and restart an hour later). If "sync" got done, and if the file is rotated by the code that opens it, you have a chance to see your machine's death cry when it crashed and burned. I've hacked my Linux startup files to do this partially to catch (sysadmin-induced) screwups during the boot process.
Why put resolv.conf in /var/run? ppp and dhcpd often obtain the addresses of the ISP's DNS servers during channel setup, and have to write them into resolv.conf. Modern practice, at least on SysV-ish systems like Solaris and Linux, is that /etc is potentially mounted read-only on a diskless workstation or CDROM, and dynamic info goes in /var/something.
Running the NFS daemon: Agreed that it's a security hole. Solaris only starts nfsd if /etc/dfs/sharetab (the successor of /etc/exports) contain the word "nfs". I've hacked my Linux startup files to do something similar.
/private/Drivers: I assume this contains drivers, similar to /lib/modules/$version on Linux. You wouldn't want to intermix code segments with device inodes, would you? :-) Or does recent BSD do something weird and wonderful along these lines? I've thought about a hypothetical UNIXoid operating system in which the device inode is [similar to] a symbolic link to its driver. (Paraquat on the grass?)
James F. Carter 
Internet: [email protected] (finger for PGP key)

Authors’ Note: The NFS inclusion seems like yet another attempt by Apple to include functionality “in the background” that they may or may not make use of. It’s an opposite to their attempts in recent versions of Mac OS (via the “Extension Manager”) to allow users to enable or disable anything that patches traps or otherwise alters the functions of the base OS, it still makes sense. The current Extension Manager functionality was most likely included because third-party utilities included this functionality, rather than because Apple really wanted end-users to have fine-grained control over the OS, and because so many poorly-written current MacOS extensions could interfere with Apple-provided OS functionality (if not hose the OS completely).

The prevailing attitude at Apple regarding OS X may very likely be that since it would be very difficult for typical  users to modify their kernel (at least with existing tools), it’s best to open up everything that might be needed at some current or future point. However, this holds only until someone creates a kernel extension/module interface as easy as the current MacOS Extension Manager (something that, despite FreeBSD’s /stand/sysinstall attempts is still far away for any other *nix).

————————————————

Further Exploration: Das Boot

Last time, we mentioned that holding down the “v” key at startup shows a BSD-esque text console startup rather than the standard MacOS X GUI startup. Considering that the hardware on a revision-A iBook is pretty different than the hardware in your average x86 Free/Open/NetBSD box, we thought it would be interesting to see just what XNU (the Darwin kernel) does on startup.

Let’s look at what happens at boot time (as shown in the message buffer using dmesg just after boot time). Comments are shown on following lines (after “<=”).

minimum scheduling quantum is 10 ms

<= Haven’t seen this before on any BSD boot.

vm_page_bootstrap: 37962 free pages

<= RAM on this iBook is 160 MB.

video console at 0x91000000 (800x600x8)

<= The iBook’s screen, 800×600 resolution (x 8 bit color?)

IOKit Component Version 1.0:
Wed Aug 30 23:17:00 PDT 2000; root(rcbuilder):RELEASE_PPC/iokit/RELEASE
_cppInit done

<= IOKit is Apple’s Darwin device driver scheme

IODeviceTreeSupport done
Copyright (c) 1982, 1986, 1989, 1991, 1993
      The Regents of the University of California. All rights reserved.

<= It’s nice that they included this BSD-style, without any “copyright Apple blah blah” stuff

AppleOHCI: config @ 5505000 (80080000)
AppleUSBRootHub: USB Generic Hub @ 1

<= This is the iBook’s one built-in USB port

AppleOHCI: unimplemented Set Overcurrent Change Feature

<= I believe that OHCI is the USB Open Host Controller Initiative, a generic standard for USB devices. This option appears to be a standard USB driver parameter which is not currently implemented (?).

AppleUSBRootHub: Hub attached - Self powered, power supply good
PMU running om NonPolling hardware
IOATAPICDDrive: Using DMA transfers
IOCDDrive drive: MATSHITA, CD-ROM CR-175, rev 5AAE [ATAPI].

<= The iBook’s ATAPI 24x CD-ROM drive

IOATAHDDrive: Using DMA transfers

<= The iBook’s HD is UltraDMA/66, if I recall correctly

IOHDDrive drive: , TOSHIBA MK3211MAT, rev J1.03 G [ATA].
IOHDDrive media: 6354432 blocks, 512 bytes each, write-enabled.

<= The iBook’s 3.2 GB hard drive; there are three logical volumes on this one.

ADB present:8c

<= Not sure about this. ADB in Apple-speak generally refers to the legacy Apple Desktop Bus, which was a low-speed serial bus used for connecting keyboards/mice. The iBook does not have an ADB port; so this probably just indicates the presence of the ADB driver.

struct nfsnode bloated (> 256bytes)
Try reducing NFS_SMALLFH
nfs_nhinit: bad size 268

<= Not sure why it’s reporting errors against its own default settings for NFS (asking the average Mac user to recompile their kernel with this option is like asking the average driver to rebuild their engine with an extra cylinder). This is presumably a “beta” bug.

devfs enabled

<= The Unix devfs (separate from MacOS X drivers?) is enabled.

IP packet filtering initialized, divert enabled, rule-based forwarding enabled, default to accept, logging disabled

<= The packet filtering that we mentioned last time. 

IOKitBSDInit
From path: "/[email protected]/[email protected]/[email protected]/@0:11,\\mach_kernel", Waiting on <dict ID="0"><key>IOProviderClass</key>
<string ID="1">IOMedia</string><key>IOPath Separator</key>
<string ID="2">:</string><key>IOPath Extension</key>
<string ID="3">11</string><key>IOLocationMatch</key>
<dict ID="4"><key>IOUnit</key>
<integer size="32" ID="5">0x0</integer><key>IOLocationMatch</key>
<dict ID="6"><key>IOPathMatch</key>
<string ID="7">IODeviceTree:/[email protected]/[email protected]/[email protected]</string></dict></dict></dict>

<= System preferences in MacOS X are set with XML files. Kudos to Apple for this forward-looking use of XML. See below for more on this and the “defaults” command.

UniNEnet: Debugger client attached
UniNEnet: Ethernet address 00:0a:27:92:04:3a

<= I think that “UniN” here refers to the “UniNorth” Apple MoBo chipset used in the iBook, which has a 10/100BT RJ-45 interface, among other things, built into it (and Ethernet network set up as the default under this configuration).

ether_ifattach called for en

<= Presumably “en” is the device driver type for this NIC

Got boot device = IOService:/Core99PE/[email protected]/AppleMacRiscPCI/[email protected]/KeyLargo/[email protected]/AppleUltra66ATA/IOATAStandardDevice/IOATAHDDrive/IOATAHDDriveNub/IOHDDrive/TOSHIBA MK3211MAT Media/IOApplePartitionScheme/Hard [email protected]
BSD root: disk0s11, major 14, minor 11
bsd_init: rootdevice = 'disk0s11'.

<= Finding the boot device; unsure why it calls it “BSD root” rather than “Darwin root” or just “MacOS X root.”

devfs on /dev
Ethernet(UniN): Link is up at 10 Mbps - Half Duplex

<= Yep, it’s plugged into the 10BT/half-duplex hub in my Netopia SDSL router.

Resetting IOCatalogue.
kext "IOFWDV" must change "IOProbe Score" to "IOProbeScore"

<= This appears to be a debugging (?) warning in a Darwin kernel extension.

kmod_create: ATIR128 (id 1), 23 pages loaded at 0x5878000, header size 0x1000

<= This appears to describe a kernel module/driver for the ATI Rage 128 chipset, although this Rev. A iBook uses only an ATI Rage Pro chipset. Perhaps this is a driver for the ATI family up to the Rage 128 series?

kmod_create: com.apple.IOAudioFamily (id 2), 16 pages loaded at 0x588f000, header size 0x1000
kmod_create: com.apple.AppleDBDMAAudio (id 3), 5 pages loaded at 0x589f000, header size 0x1000
kmod_create: com.apple.AppleDACAAudio (id 4), 9 pages loaded at 0x58a4000, header size 0x1000

<= Loading drivers for the audio chips on the iBook MoBo.

PPCDACA:setSampleParameters 45158400 / 2822400 =16
kmod_create: com.apple.IOPortServer (id 5), 13 pages loaded at 0x58be000, header size 0x1000
kmod_create: com.apple.AppleSCCSerial (id 6), 9 pages loaded at 0x58cb000, header size 0x1000

<= More drivers for Apple MoBo chipsets.

creating node ttyd.irda-port...
ApplePortSession: not registry member at registerService()

<= This looks like the IrDA infrared transfer port drive attempting to create a connection and failing.

creating node ttyd.modem...
ApplePortSession: not registry member at registerService()

<= It looks like it’s trying to create a connection to the modem port and failing.

.Display_ATImach64_3DR3 EDID Version 1, Revision 1
Vendor/product 0x0610059c, Est: 0x01, 0x00, 0x00,
Std: 0x0101, 0x0101, 0x0101, 0x0101, 0x0101, 0x0101, 0x0101, 0x0101,
.Display_ATImach64_3DR3: user ranges num:1 start:91800480 size:ea680
.Display_ATImach64_3DR3: using ([email protected],16 bpp)

<= These appear to be setting GUI resolution at 800 x 600 in 16-bit color (which is what they had been set to in the GUI controls) 

kmod_create: SIP-NKE (id 7), 7 pages loaded at 0x59b8000, header size 0x1000
kmod_destroy: ATIR128 (id 1), deallocating 23 pages starting at 0x5878000

<= Not sure about this; probably unloading the ATI Rage 128 driver from the kernel (?)

MacOS X’s Hardware Drivers and Support

Following on the above: many people, when they think about hardware and drivers that Apple needs to create for Darwin/Mac OS X, have one of two thoughts. It’s either “That should be really easy since Apple has all standardized hardware,” or “Won’t that be hard, since Macs use a bunch of whacked-out hardware that nobody else has ever heard of?” The answer is somewhere in between.

One of Apple’s few built-in advantages has always been that, since it creates its own hardware as well as software, it needs to support only a small fraction of the devices that any commodity x86-based OS might need to support. Furthermore, Apple’s dictum that Mac OS X will support only “Apple G3-based computers” makes it seem that hardware driver support would be relatively straightforward. This, however, is not the case. Apple’s “G3” support actually involves support for two wildly differing branches of hardware (and some “in-between” models).

Apple (circa 1996 or so) suffered in terms of hardware manufacturing and compatibility because of the amount of “non-standard” hardware it used. Aside from the obvious use of Motorola/IBM PowerPC CPUs instead of commodity x86 CPUs, (some) Apple’s desktops used the Texas Instruments NuBus expansion system instead of PCI, AGP or ISA; a proprietary serial bus for printers/modems; the Apple Desktop Bus (ADB) for keyboards and mice; external SCSI-1 for all other peripherals; and a variety of other custom Apple MoBo (motherboard) components.

However, Apple in the “Age of Steve” has moved to a more industry standard-compliant position. When Apple ditched beige colors (starting with the Jobs-directed iMac in 1998), it moved to a legacy-free environment, ditching a lot of its older custom hardware. Apple also effected a number of other hardware changes, moving more of the Mac OS “Toolbox” routines from custom ROM chips into software (MacOS X shouldn’t need them at all) and moving to unify all of its lines with a unified motherboard architecture (UMA-1). 

The new machines (the ever-fly Steve Jobs’ “kickin’ it new-school legacy-free style”) ditched floppy disk drives and the old Apple Desktop Bus and serial ports in favor of USB for keyboards, mice and low/medium-speed peripherals. Built-in external SCSI was soon eliminated in favor of a mixture of USB and IEEE 1394 (“FireWire”) for higher-speed peripherals. With the introduction of the G4-based desktops, 2xAGP replaced PCI as the video card slot, leaving three 33-MHz PCI slots for internal expansion.

Drawing OS X’s “supported models” cut-off line at Apple’s G3s (which excludes Apple or clone-vendor models with G3 CPU upgrade cards) eliminates needing to support much of the legacy hardware that Apple has used in the past. However, there are a few Apple G3 models that bridge both technology generations, creating a notable thorn in Apple’s side. Because the original G3 desktops and PowerBooks included legacy technology (ADB, Apple serial ports, older chipsets), Apple must support these devices in Darwin and Mac OS X.

Type I/II PC card support does not appear to be available in Mac OS X Public Beta (possibly the reason there is no *official* support for Apple’s IEEE 802.11 “Airport” cards); IEEE 1394 (FireWire) support does not appear to be available, either. Apple’s new UMA-2 chipset is rumored to be introduced with new models at January’s Macworld expo; however, the specs of this chipset can only be guessed at now.

The Mysteries of “defaults”

Much of this article and the previous one have been the result of aimlessly exploring the file system of MacOS X from the command line, finding things that seemed odd or interesting and wondering, “Hmm, what will this do when I poke it?” I almost wish I didn’t stumble onto this next item. As I investigated its operation and the history of its implementation, I just became more curious about certain design decisions. Sort of like … well, NetInfo.

The item in question is the defaults command. After finding it, reading its man page (defaults(1)), and playing with it a little, it appeared sort of cool, but pretty mundane. The command allows for the storage and retrieval of system and application level user preferences. Please forgive the reference if it’s too far off, but it’s like a sort of “Windows registry” for MacOS X.

To get an idea of what information the system stored, I experimented with the command to see what it would tell me. Typing ‘defaults domains’ spit back the following list of information categories, or “domains” in Apple parlance:

% defaults domains
NSGlobalDomain ProcessViewer System%20Preferences com.apple.Calculator com.apple.Console com.apple.HIToolbox com.apple.Sherlock com.apple.Terminal com.apple.TextEdit com.apple.clock com.apple.dock com.apple.finder com.apple.internet com.apple.keycaps loginwindow

Those domains that related to applications illustrated Apple’s suggested domain naming convention of preceding the application name with the software vendor’s name. In this case, on a system containing solely Apple software, the application domains all began with ‘com.apple’.

I found that there were a variety of ways to view the data contained in these domains. In order to view the data for all domains, simply type ‘defaults read’. To query instead for information specific to a domain, use the form ‘defaults read <domain name>’ (without the arrows), or to grab a specific value, use ‘defaults read <domain name> <key>’. The command also allows the setting of values using the form ‘defaults write <domain name> <key> <value>’. The man page describes a variety of other ways to use the command.

Having read about XML-based plists (property lists) in MacOS X (originally from John Siracusa’s excellent series of Ars Technica articles on the OS, indexed at http://arstechnica.com/reviews/4q00/macosx-pb1/macos-x-beta-1.html), I assumed that this service might be based on an XML back-end. A quick search through the filesystem confirmed this suspicion. Per user preferences were found under ~/Library/Preferences, with each application having its own plist file. System-wide preferences showed up under /Library/Preferences, and although they were not present on this non-networked machine, I would be willing to bet that network-wide preferences could be found under /Network/Library/Preferences.

I did a little comparison using the preference data from the system clock application. First, I read the data using the defaults command, then I located the plist version in my ~/Library/Preferences directory. The results below show how the system translates the clock data into the XML plist format.

% defaults read com.apple.clock 
{
 24Hour = 0; 
 ColonsFlash = 0; 
 InDock = 0; 
 "NSWindow Frame Clock" = "-9 452 128 128 0 4 800 574 "; 
 ShowAnalogSeconds = 1;  
 Transparancy = 4.4;  
 UseAnalogClock = 1;  
 UseDigitalClock = 0;  
} 

% cat Library/Preferences/com.apple.clock.plist 
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist SYSTEM "file://localhost/System/Library/DTDs/PropertyList.dtd">
<plist version="0.9">
<dict>
 <key>24Hour</key>
 <false/>
 <key>ColonsFlash</key>
 <false/>
 <key>InDock</key>
 <false/>
 <key>NSWindow Frame Clock</key>
 <string>-9 452 128 128 0 4 800 574 </string>
 <key>ShowAnalogSeconds</key>
 <true/>
 <key>Transparancy</key>
 <real>4.4e+00</real>
 <key>UseAnalogClock</key>
 <true/>
 <key>UseDigitalClock</key>
 <false/>
</dict>
</plist>

I was a bit intrigued by the date of the defaults man page. It was dated March 7, 1995, which as far as I knew was prior to the advent of XML. Apparently, there was some amount of history to this command and database that was not immediately obvious. A little research revealed that the defaults command and database date back at least to (yes, you guessed it) NeXTstep. The information found confirms that it was subsequently adopted for OpenStep and finally MacOS X. It also appears that the “back-end” evolved over time, from straight ASCII string based configuration files to today’s XML-based plist files. (As usual, if you know more about the history of this database, please feel free to share your knowledge, and please forgive my lack of knowledge of all things NeXT.)

With this history information in hand, I became curious about the present implementation. I was also curious if an API existed. Obviously, normal applications would not use this command to store and retrieve persistent data. However there was no mention of an API in the defaults man page, nor was there a link to any other information. A quick trip to the Apple developer website provided the information I was looking for. The API was called “Preference Services” and was a part of MacOS X’s “Core Foundation (http://developer.apple.com/techpubs/corefoundation/PreferenceServices/Preference_Services/Concepts/CFPreferences.html). Apple provides two sets of APIs for use based on the developer’s needs, a high-level one that allows the developer to quickly set and retrieve data in the default domain (defined as the current user, application, and host), and a low-level API for setting and retrieving data in very specific domains (specific user(s), application(s), and host(s)). The API also provides a method of storing data for suites of applications such as the standard office productivity apps we all know and love to hate.

During this exploration, I have been having a hard time deciding whether I think this is all cool or not. On one hand the idea of a system-(and even network-)wide preferences database seems pretty cool, especially to a BSD-er like myself, but at the same time, it is really nothing new. In the same way, at first glance, the idea of an XML-based back-end seems pretty innovative, but is it? Sure it’s cool to look at, but so what? The existence of the defaults command and an access API mean that the actual plist files are not intended for public consumption.

The defaults command and Preference Services API indicate that the whole database is supposed to be a black box to the system, the application, and especially the user. If this is the case, why not go with a high-horsepower back-end, one that offers more robust searchability and speed than that which could be achieved via the “crapload of text files” approach. I think the argument from Apple was supposed to be that the files should be architecture-neutral in order to be easily portable. If this was the case, why not just leverage an existing architecture independent binary database format. I know, for instance, that MySQL can do it, why not Apple?

The other argument I can think of might be that the XML format is essentially patch-able. Assuming the data is not customized too much by the running application, updates could be distributed and installed from small plist patch files. However, that doesn’t seem like a very convincing argument. All this having been said, this article will probably be published and the next day, a reader will say, “Well, <profoundly obvious answer succinctly stated>. Please unset your $LUSER variable.” The first person to write in with this wins a prize. 😉

Random Ramblings about BSD on MacOS X (Part 1)

By Jeffrey Carl and Matt Loschert

Daemon News, November 2001

This is the first chapter in a series of observations, representing the adventures of a couple of BSD admins (one with a lot of prior MacOS experience, the other with more on the BSD side) poking around the command line on an iBook laptop running Apple’s  Mac OS X Public Beta. We’ll attempt to provide a few notes and observations that may make a BSD admin’s work with Mac OS X easier.

Mac vs. Unix

Right up front, I should mention what a strange animal Mac OS X is. Since 1984, the Mac OS has always been about the concept that “my OS should work easily, without my knowing how it works.” Unix has always been about the idea that “I should be able to control every aspect of my OS, even if it isn’t always easy to figure out how.” Apple – whose very name is anathema to many Unix admins – is now trying to combine both, in the world’s first BSD-based OS that is likely to ship 500,000 copies in its first year.

Apple isn’t alone in this hybridization effort; Microsoft will presumably ship “Whistler,” its NT/2000-based consumer OS, sometime in mid-2001. It appears that everywhere, the rush is on to deliver consumer OSes based on the kernels of server operating systems. In the long run, this is probably a Good Thing®, but Mac OS X and MS Whistler will have to introduce millions of desktop users to the advantages and problems of server OS design that we as server OS admins have been dealing with for years. Real protected memory and pre-emptive multitasking, sure … but also real multi-user systems and all of the permission and driver complexities that requires, as well. However, both Microsoft and Apple are investing the time and effort into developing a real user-friendly interface that your grandmother could configure (sorry, KDE and GNOME). It should be interesting to watch.

If you, like most other Internet server admins with a fully-functional brain stem, prefer Unix over the NT kernel, then Mac OS X will be the first true consumer OS that you will ever feel comfortable with. And, whether you love or hate Apple, it’s worth your time to get acquainted with Mac OS X and what it offers to a BSD Unix administrator.

Right now, there are a lot of questions out there about Mac OS X and its attempts to marry a BSD / Mach microkernel with an all-singing, all-dancing Mac GUI. This is largely because there is unfortunately a comparatively small crossover between Mac administrators and Unix administrators (much like the slim crossover between supermodels and serious “Dungeons and Dragons” players). Though neither Jeff nor Matt is a total über-guru in Mac OS or BSD, we’ll attempt to bridge the gap a little and give the average BSD admin an idea of what he or she will see if they peek around under the hood of Mac OS X Public Beta. Much of this is “first impression” notes and questions, without too much outside research behind it; explanations or clarifications from those who have more detailed answers are gratefully appreciated. 

On a side note, Matt and Jeff will both use “I” rather than “we” in this article as we go, since we are both members of the FreeBSD Borg Collective and think of ourselves as one entity. 😉

Poking Under the Hood: First Impressions

To examine the hidden Unix world on OS X, navigate in the GUI to Applications >> Utilities >> Terminal. The first thing you may notice is that – as you might expect – the default shell is tcsh. You’ll quickly find that many of the user apps you’d expect in a typical Unix installation are there, plus a few extras and minus a few others (showing the refreshing influence of many developers, each pushing for the inclusion of their favorite tools). Interestingly, pico is there, although Pine is not; Emacs and vi are there, also (ed may be there, but I didn’t check since I thought everyone who might use ed was fossilized and hanging on display in a museum by now). Other fun inclusions are gawk, biff, formail, less, locate, groff, perl 5.6, wget, procmail, fetchmail, and an openssh-based client and server.

The first thing that I do when beginning to work on a new system is to set my shell and copy over my personal config files.  When presented with the Mac OS X shell prompt, I was pleased to see that it was tcsh; but at the same time I was a little confused at how to customize it. Usually, I just drop my .cshrc file in ~/, and off I go. Well, ~/ in this case is /Users/myusername.  This is unlike the normal /usr/home or /home that most admins are used to.  Would tcsh be able to find my .cshrc there?

Well, since I didn’t have the machine network-connected while I was originally exploring, and I was too lazy to hand-copy parts of my personal .cshrc file over to this directory, I instead began poking around /etc looking at how the default .cshrc worked.  I quickly found a small csh.cshrc which contained a single line sourcing /usr/share/init/tcsh/rc.  This is where it gets interesting.  

This rc file sets up a very cool method of providing standard system shell features that can be overriden by each Mac OS X user (assuming he/she realizes that the “shell” does not refer to the funky computer case).  The rc file first checks to see if the user has a ~/Library/init/tcsh directory.  If so, it sets this as the tcsh initialization directory; otherwise it sets the /usr/share/init/tcsh directory as the default.  It then proceeds to source other files in the default directory including, for instance, environment, aliases, and completions files which each in turn source files (if they exist) found in the abovementioned tcsh initialization directory.

In this way, the system provides some powerful “standard” features, but still gives the experienced user the ability to override anything and everything.  I dropped some customizations in my personal ~/Library/init/tcsh directory and immediately felt at home.  Without a doubt, the old school UNIX curmudgeons will hate the built-in shortcuts, but I still must applaud the developers for making an attempt at providing a useful set of features for the “new” user.  No one will ever agree on a standard set, but it’s good to see that the average Mac “What is this, some kind of DOS prompt?” user will have a very functional command line, should he/she choose to explore it.  I must admit that some of these completions can be a little annoying though, while typing quickly.

Digging Around

When you drop to the shell, you’ll notice that you’re logged in with the “user name” you selected for yourself when you installed the OS. When you create a user during the installation process, you are informed that “you will be an administrator of this computer.” Yet, when in the shell, you take a look at /etc/passwd, you’ll see no entry for your username. Nonetheless, if you su to “root,” you’ll notice that root’s password is the same as the administrative user you created. Even stranger, although you are an “administrator,” as long as you’re logged in as that user, you still don’t have the permissions of “root.” What’s going on here?  Well, it turns out that this functionality has been co-opted by Apple’s NeXT-inherited Net Info (which I will describe below).

I couldn’t resist running dmesg to see what would come up.  To my surprise, the output was pretty boring.  The most bizarre thing was that a packet filter was enabled, but with a default to open policy and no logging.  An open firewall … why?  If they weren’t planning to enable firewall rules, why compile packet filtering code into the kernel in the first place?  At this point, I felt I had to do at least a little searching for some sign that this had been considered at one point.  Well, /etc held no clues, and there was nothing obvious in /var (the only other probable location I could think of).  I guess they simply wanted to leave the door conveniently ajar should they choose to go back and reconsider the decision.

When I visit a machine, I am always curious to check out /etc.  This directory usually reveals some interesting information about the operating system and/or the administrator.  In this case, while touring the usual suspects, I noticed something that, again, I did not expect.  The wheel group listed in /etc/group did not contain any users (it seemed like some Communist Linux system!).  You typically expect to find root along with at least one administrative user account in /etc/group.  In this case, wheel was empty, leaving me to wonder why they even kept the concept, since this functionality had apparently been folded into an alternate privilege manager. As I found out, this functionality is also handled by NetInfo.

While checking out /etc/syslog.conf, I noted that quite a bit of system information gets routed to /dev/console, as one would expect.  What was strange was that later while exploring /var/tmp (to look for tell-tale temp files), I found a file named console.log.  It appeared to contain a file version of the most recent couple of console messages.  I verified that identical (albeit fewer) entries appear when using the GUI-based console viewer.  I’m no super OS guru … but this strikes me as a strange hack.  Why not simply send the same information to a standard log file via syslog and have the GUI console app read from that?  That’s what syslog’s there for!  Maybe someone else out there can shed some light on this one.

Another oddity was /etc/resolv.conf.  It was symlinked to /var/run/resolv.conf, which seemed a little strange.  Even more so when /var/run/resolv.conf turned out to be non-existent. Okay … well, there was no named running (I checked) and the resolver man pages gave no clue.  So what was going on here?  Apparently something odd, since a quick nslookup responded with the DNS server listed as being the local machine.  No named, no resolv.conf, how about /etc/hosts?  Well, /etc/hosts had no special magic in it, but it did have a note mentioning that the file is only consulted while the machine is in single-user mode.  At all other times, the request is handled by lookupd working through the facilities provided by NetInfo.  Hmm … NetInfo was beginning to sound very interesting.

After a bit of roaming around, I became curious as to whether root would have a home directory on the machine.  Given the difference in user and administrator handling on Mac OS X, I wasn’t even sure that root would have a home directory.  But, lo and behold, there was ~root under /var (or more properly /private/var since it’s a symlink) with a set of default files and directories resembling any other user in /Users.  None of this should have come as a surprise to me since /etc/passwd contained this information.

Strange Services

While exploring the system, you’ll find some interesting services enabled.  I was surprised to see the nfs daemon running. Thus, it didn’t faze me when I found the portmap daemon (a service required in order to run nfs) hanging around.  Finding nfs active was unexpected, since I would only run it if necessary, due to potential security concerns.

Another surprise was seeing a full-fledged NTP daemon running.  It was cool to see this service as a standard part of Mac OS; but I thought it a bit strange though to do continuous high accuracy time synchronization via ntp for a desktop, consumer OS.  Why not use ntp’s little brother, ntpdate, on a periodic basis for this same functionality?  Do Mac OS users really need the accuracy of continuous ntp, when a functionally similar result could be obtained without the security risk of running another network daemon?

The answer is that, for now, “yes they do.” Since the introduction of Mac OS 9, Mac users have been given the default option of using NTP to synchronize their desktop clocks via NTP servers, and it appears that Apple wants to give Mac OS X users (presumably in a server setting) the option of running an NTP server for other Macs on a LAN/WAN.

Speaking of network services, a little netstat lovin’ showed a couple of other interesting tidbits.  The machine was not as quiet network-wise as I had expected.  Along with the nfs, portmap, and ntp network activity, I found listening sockets open on ports 752 and 755, with an active connection to port 752 from another local privileged port.

After some man page reading and poking at a few of the running daemons, I was able to narrow things down a little.  I found that HUP-ing lookupd (an interface daemon to NetInfo) caused the active connection to be disconnected and reestablished (obvious since the new connection originated from a new port).  I also found that HUP-ing nibindd (the NetInfo binder, a sort of librarian for NetInfo) caused the listening sockets to be closed and reappear on new ports. That struck me as quite odd.  I would have expected them to re-bind to the same ports.  Even with these addresses changing, lookupd somehow knew where to find the ports, as I saw new connections established shortly after the HUP signal was received.

Not knowing much about NetInfo, I was curious why this service was implemented using tcp sockets.  I assumed that the service must be distributed, available to and from remote hosts.  Otherwise, the developers probably would have implemented this using Unix domain sockets since they are substantially faster and are safer for local-only protocols. Not having much background on NetInfo at the time, I was somewhat puzzled.

Welding the Hood Shut

Apple’s longstanding (since Mac System 1.0 in 1984) policy has been to prevent users from screwing up their systems by hiding important items from them, or making it difficult to access the cool things that they might play with and hose their hard drive. This attitude has extended to OS X, and things like header files, gcc and gdb are not present (although these were included in Apple’s earlier “developers-only” previews of Mac OS X), not that the average Mac user would be able to “accidentally” damage their system with gcc.

Not to worry, though; the Mach / BSD portion of Mac OS X is maintained separately as an open-source operating system that Apple calls “Darwin.” In fact, a uname -a at the command line in MOSXPB reveals:

Darwin ibook 1.2 Darwin Kernel Version 1.2: Wed Aug 30 23:32:53 PDT 2000; root:xnu/xnu-103.obj~1/RELEASE_PPC  Power Macintosh powerpc

You can get the abovementioned developer goodies (and much more) by downloading Darwin or following the CVS snapshots for various packages. 

At the root of the filesystem, it’s not hard to see that most of what you would expect to find is there somewhere; it may just be in a different place than you expected. The first thing you’ll likely notice is that a lot of what’s there is being hidden from the user by the MacOS X GUI. The Mac OS has always had its share of hidden files (for volume settings, the desktop database, etc.) However, you’ll notice a file in / called “.hidden” which lists the items in / that the GUI hides. The contents of .hidden are:

bin (the Unix directory for binaries needed before /usr is mounted)

cores (a link to /private/cores; a directory – theoretically, a uniform location for placing Unix core dumps)

Desktop DB (the GUI desktop database for files, icons, etc.)

Desktop DF (don’t remember what this does)

Desktop Folder (for items that are shown on the desktop)

dev (the Unix devices directory)

etc (a link to /private/etc; contains normal Unix configuration files and other normal /etc stuff)

lib (listed, but not present in /)

lost+found (the Unix directory for “orphaned” files after an unclean system shutdown. Classic Mac OS users are used to looking in the Trash for temp files saved after a crash or other emergency shutdown)

mach (a link to mach.sym)

mach_kernel (the Mach kernel itself)

mach.sym (listed as a Mach-O executable for PowerPC, but I’m not sure exactly what it does. Any Darwin users out there with more knowledge should correct me.)

private (contains var, tmp, etc, cores and a Mac OS X directory called Drivers, whose relationship to /dev is unclear)

sbin (Unix system binaries traditionally needed before /usr mounts)

SystemFolderX (not present in /; may be a typo or used in the future?) 

tmp (a link to /private/tmp; the Unix temporary files directory)

Trash (files selected in the GUI to be deleted)

usr (just about everything in BSD Unix these days 😉 )

var (Unix directory meant once upon a time for “variable” files – mail, FTP, PID files, and other goodies)

VM Storage (the classic MacOS virtual memory filesystem – usually equal to the size of physical memory + swap space)

A brief digression about .hidden: Editing this file and adding any directory or file name will cause the GUI to hide the item. For example, edit this file in the shell and add ‘Foo,’ create a directory in / with that name, log out and log back in, and it won’t be seen. 

However, it isn’t as simple as it sounds. Strangely, there’s an item in / called “TheVolumeSettingsFolder” whose name isn’t in .hidden, but still doesn’t show up in the GUI. This indicates that there is more to what’s shown and hidden in the GUI than just the “.hidden” file. Also, adding a “.hidden” file to another directory than “/” does not appear to cause files of that name to be hidden in that directory. Furthermore, /.hidden does not prevent files with those names from appearing in lower directories, nor does it hide directories in lower directories when an absolute path is placed in .hidden. If anyone can clear this up for me, I’d love to know what’s at work here.

Getting back on track: You’ll notice that all of the traditional BSD file hierarchies are present, although the GUI hides them from the user so that they are unaware that these Unix directory structures exist. Furthermore, some of the directories you see are in fact soft links to unusual places (as mentioned, etc, tmp and var are all links to inside /private; I refuse to speculate why without having several drinks of neat Scotch first). Also, unlike previous incarnations of the Mac OS, the user is unable to rename the “System” directory from the GUI.

As a brief aside, it should be noted that Apple’s great struggle here (and the immensity of their effort should be appreciated) is to combine the classic Mac OS “I can move or rename damn near anything I want” ethos with Unix’s “it’s named that way for a reason” ethos. How Apple chooses to resolve this dilemma in the final release will speak volumes about their dilemma over (I’m ashamed to admit I’ve forgotten who put it this way, but it’s brilliant) an OS that “the user controls,” versus an OS that “allows you to use it.” And, let’s face it, Unix has always been the latter.

Also, logical or physical volumes are listed under “/” using their “name.” Rather than use a Unix filesystem hierarchy, DOS’s A: or C: drives or even Windows 9x’s “My Computer,” Mac users have always been able to name their hard drives or partitions arbitrarily. Each drive or partition thereof was then always mounted and visible on the desktop. If I have a drive in my computer which I’ve named “Jeff’s Drive,”  you’ll see it from the shell as: /Jeff’s\ Drive, and all of its file and directory contents are viewable underneath it. Similarly, if a user installs MOSX PB on a drive with Mac OS 9 already installed, at installation time their old files and directory hierarchies are all moved to a directory called “Mac OS 9” in /.

** What is strange about the above?? It sounds very UNIXy.  As an example, many people mount their floppy drives as ‘/floppy’, but nothing is preventing them from mounting the drives as ‘/Cool\ Ass \Plastic\ Coaster\ Partition’. **

NetInfo & Friends

Well, by the time I got this far, I realized that this article wouldn’t really be complete without some discussion of the mysterious NetInfo.  I also knew that a little bit of research was in order.  While searching, I learned that NetInfo is a distributed configuration database consisting of information that would normally be queried through a half-dozen separate sub-systems on a typical UNIX platform.  For example, NetInfo encompasses user and group authentication and privilege information, name service information, and local and remote service information, to name just a few.

When the NetInfo system is running (i.e. when the system is not in single-user mode), it supercedes the information provided in the standard /etc configuration files, as well as being favored as an information source by system services, such as the resolver.  The Apple engineers have accomplished this by hooking a check into each libc system data lookup function to see if NetInfo is running.  If so, NetInfo is consulted, otherwise the standard files or services are used.

The genius of NetInfo is that it provides a uniform way of accessing and manipulating all system and network configuration information.  A traditional UNIX program can call the standard libc system lookup functions and use NetInfo without knowing anything about it.  On the other hand, MacOSX-centric programs may directly talk to NetInfo using a common access and update facility for all types of information.  No longer does one have to worry about updating multiple configuration files in multiple formats, then restarting one or more system daemon or daemons as necessary.

The other benefit of the system is that it is designed to be network-aware from the ground up.  If information cannot be found on the local system, NetInfo may query upward to a possibly more knowledgable information host.

NetInfo also knows how to forward requests to the apropriate traditional services if it does not have the requisite information.  It can hook into dns, nis, and other well-known services, all without the knowledge of the application making the initial data request.

NetInfo is a complex beast, easily worth an article of its own.  If you want more information, here are a few tips.  I found that reading NetInfo man pages was frustrating.  Most of the pages tended to heavily use NetInfo-related terms and concepts with little to no definition.  Nevertheless, if interested, check out netinfo(5) (netinfo(3) is simply the API definition), netinfod(8), nibindd(8), and lookupd(8).  However, the best information that I found was on Apple’s Tech Info Library at http://til.info.apple.com/techinfo.nsf/artnum/n60038?OpenDocument&macosxs.

Getting More Information

Finally, for a great resource on all things Darwin, don’t go to Apple’s website (publicsource.apple.com/projects/darwin/). Instead, go to www.darwinfo.org, and you’ll find lots of great stuff, including the excellent “Unofficial Darwin FAQ.”

If you don’t have access to MacOS X Public Beta but would like to read its man pages to get a better idea of how some of these commands are implemented, you can find a collection of MOSXPB/Darwin man pages online at www.osxfaq.com/man.

Jeff and Matt hope this little tour has been semi-informative and has raised the curiosity of the few brave souls who have made it to the end of this MacOS travelogue.  Next time we will take a look at … and discuss ….  See you next time.

Darwin Evolves – Apple’s BSD Hits the Prime Time

By Jeffrey Carl

Boardwatch Magazine
Boardwatch Magazine, May 2001

Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among ISPs, particularly for its annual ranking of Tier 1 ISPs and through the ISPcon tradeshow. Writing and speaking for Boardwatch was one of my fondest memories of the first dot-com age.

In a Nutshell: DarwinOS, the core of Apple’s just-released MacOS X, is open-source and available as a free download. While it inherits many characteristics from current BSD Unixes, it’s also radically different – revealing its true identity as the direct descendant of the ahead-of-its-time NeXTSTEP operating system. Darwin has some rough edges and missing documentation, but offers some very interesting possibilities. Unfortunately, the fact that it’s intended for Apple hardware will limit its appeal to many ISPs and sysadmins.

By the time you read this, Apple’s MacOS X should be on the street – the first-ever consumer OS with a Mach microkernel/BSD Unix core. That core is called Darwin, and it’s an open-source project – available under the BSD-ish Apple Public Source License – that can be downloaded for free (MacOS X, which includes the new Mac GUI and lots of other goodies, costs $129).

I’ve talked about Darwin here before, when it was in its earlier stages, but wasn’t able to go into many specifics about what Darwin was really like for a Unix admin. Now that the version that ships with MacOS X 1.0 is here, let’s take a look at this remarkable OS change for Apple.

The Politics Behind Darwin

Darwin is the triumph of NeXT over Apple. The GUI-only, outmoded classic MacOS of Apple’s last 17 years is gone, and a tough, sexy Unix core has replaced it (although MacOS X includes a compatibility layer to run existing MacOS apps). To understand why – and to understand why a Unix admin might be very interested in an Apple OS (other than the “research project” MkLinux) for the first time in many years – it helps to know the politics behind the situation.

In case you haven’t been paying attention to Apple’s corporate soap opera for the last few years (or, more likely, just don’t care), here’s the short version. Apple was at its lowest depths in late 1997, having wasted more than six fruitless years trying to create a modern operating system to replace MacOS (which was still building on foundations laid in 1984). Then-Apple CEO Gil Amelio instead looked to buy a company that already had an advanced OS, and settled on NeXT, the failing company run by Apple’s exiled cofounder Steve Jobs. The brilliant, mercurial, occasionally tantrum-prone Jobs was instrumental in building Apple from Steve Wozniak’s garage into an industry giant. But in 1985, he was fired by his own board of directors, essentially for being a major jerk to everyone within a 50-yard radius. It was a very nasty “divorce,” and Jobs left in a huff to found NeXT.

The NeXTSTEP (later called OPENSTEP) operating system which powered NeXT’s computers was years ahead of its time – but shipped on underpowered, grossly overpriced computers (sound familiar?) and was rejected by the marketplace. NeXTSTEP was based on Unix and the Mach microkernel, and included a radical new GUI and a pretty cool object-oriented development framework. 

Microkernels (of which Mach is the most famous example) provided a much more elegant OS design than most OSes used (and still use), by moving everything but memory management and other lowest-level tasks out of the kernel. However, the overhead of this elegant and scalable design provided reduced performance in many “everyday” situations (like replacing a simple Excel spreadsheet with a full relational database), and microkernels were sidelined as academic curiosities. The NeXT object-oriented development kit was very advanced, but required knowledge of the relatively obscure Objective-C language and was largely ignored as well.

For Apple’s $400 million purchase of NeXT in 1997, they got not only the company’s OS but CEO Jobs as well. Then-Apple CEO Gil Amelio thought he was getting a valuable figurehead/consultant in Steve Jobs. But an ex-Apple employee who knew Jobs better than Amelio did predicted to Apple CTO Ellen Hancock that “Steve is going to f*** Gil so hard his eardrums will pop.” Sure enough, on July 6 1998, Gil Amelio was forced to resign and Steve Jobs once again was in charge of Apple. (Hancock resigned when Amelio did, and went on to become president of web hosting and datacenter giant Exodus.)

The rest is history. In the foreground, Jobs was introducing the fruit-colored, legacy-free  iMac to the world, sparking Apple’s sales resurgence. In the background, nearly all of Jobs’ loyal NeXT troops were assuming the top posts at Apple and changing the company’s technology and direction.

In 1999, the hype about Linux and open source was at its height, and Apple felt the pressure to join the crowd. Since BSD and Mach – which formed the core of NeXTSTEP – were already open source, it wasn’t hard for the normally ultra-proprietary Apple to take the step of officially open-sourcing the core of the forthcoming MacOS X. The NeXTSTEP core of MacOS X officially became “Darwin,” and a developer community of Apple engineers, Macolytes and Unix hackers began to form around the project. Along the way it saw contributions from Quake designer John Carmack, a basic port to Intel x86 hardware, and a “1.0” version that has evolved significantly as MacOS X neared release.

It’s noteworthy to keep in mind how much of a departure Darwin is from the old MacOS. It’s as if the “House that GUI Built” was taken over by the Unix geeks from Carnegie-Mellon and handed over to the hacker community for safekeeping. NeXT was the “ugly step-child” of the BSD family that nobody else noticed; and now, it will claim more users than the rest of the family combined – probably in less than six months.

Getting In-Depth with Darwin

After all that, it’s time to get under the hood of Darwin as a Unix admin would approach it. Many reports have claimed erroneously (at times, I have said this as well) that Darwin was basically “FreeBSD on Mac.” In fact, it’s a bit of updated Mach, a bit of traditional Free/Open/NetBSD, and a lot of NeXTSTEP flavoring. 

Darwin uses the Mach microkernel, but attempts to solve some of its performance problems by moving much of the BSD functionality into the same “kernel space” (rather than in “userland” as a pure microkernel would). As such, it merges the two worlds in a way that is designed to keep the architectural elegance of a microkernel design while minimizing the performance overhead that microkernel process scheduling causes.

The first thing a BSD Unix admin will notice upon logging into Darwin is its familiarity. The default shell is tcsh, and nearly all of the customary /bin, /usr/bin/ and /usr/sbin utilities are there. In addition, you’ll find vipicoperlsendmailApache, and other typical goodies. Ports to Darwin of popular Unix open-source Unix apps – from mysql to XFree 4.0.2 are proliferating rapidly. From a typical user’s perspective, it’s almost indistinguishable from *BSD.

It’s only once you become root and muck around in the system’s administration internals that you start to notice what makes the system a true child of NeXTSTEP. You’ll notice in /etc (actually a link to /private/etc; see below for more information) that /etc/resolv.conf doesn’t contain what you would expect. Nor does /etc/hosts, /etc/group or /etc/passwd. Why? It’s because Darwin wraps many of the functions contained in separate system and network configuration files in *BSD into a single service (inherited from NeXT) called NetInfo. Why is this useful rather than just annoying?

In *BSD and Linux, a number of different services derive their information from separate text configuration files – each with its own syntax and options. There’s no global preferences database – for example, any application that doesn’t automatically know how to read all the items in /etc/resolv.conf can’t find out what name servers your computer is using, or a program that can’t parse the syntax of /etc/passwd doesn’t know which users are on your system. 

Somewhat like the Microsoft Windows Registry, Darwin’s NetInfo provides a database of network and user settings that can be read by any other NetInfo-aware application. NetInfo supersedes the information in the traditional /etc/* configuration files, as well as being favored by system services. NetInfo is consulted not only by MacOS X-native applications, but also by traditional BSD/Unix applications as well (making it much easier to port these apps to Darwin). The Apple engineers have accomplished this by hooking a check into each libc system data lookup function to consult NetInfo if it’s running (by default, it’s only “off” in single-user mode). 

MacOS X’s GUI provides graphical tools for manipulating the NetInfo database; in Darwin, this can be done using the niutil and nicl commands (use man niutil and man nicl to see the syntax and options; it’s interesting to note that these man pages are dated from NeXT days). NetInfo can also “inherit” its settings from a “parent” NetInfo server, so you can create one server which has everyone’s account information on it, and all of its client machines will have their login info, network setup, et cetera (imagine a “family” of servers where users can interchangeably log in with the same accounts, settings, etc.).

Like NetInfo settings, application preferences are stored in a global XML database; they can be manipulated from the command-line defaults program. Typing man defaults from the command line will give you an idea of how its structure works.

One area that has been changed since the MacOS X Public Beta is that it is no longer necessary to reboot or log out/log back in to change network settings. Anyone who has used *BSD/Linux on mobile computers, or changed network profiles in Windows NT will appreciate this difference.

Darwin/MacOS X includes a /private directory at the root filesystem level, which includes the normal BSD /etc, /var, /tmp and the Darwin /cores and /Drivers). It appears that (in behavior inherited from NeXT) this directory is filled with info that is machine-specific, so that the rest of the filesystem can be booted from a parent network server. This clearly plants the roots for Darwin or MacOS X-based systems to serve as “thin clients.”

As for performance, recent builds of Darwin work admirably on mid-range Mac hardware. The only real complaint I have about Darwin (my gripes with the MacOS X user interface could fill a small book, however) is its woeful lack of documentation. While many BSDs suffer a similar problem, their user communities have had time to fill in the gaps; the changes that Darwin makes to the traditional BSD model are largely known only to the Darwin community and old-school NeXT gurus.

The closer I look at it, the more Darwin (and MacOS X) is appearing to take up where Novell left off in the race to compete with Windows NT/2000 in the corporate network space. It might be possible that Apple is looking to go head-to-head with Windows Whistler/XP while nobody is looking. And any victory in that space for *nix is something to be cheered.

Darwin’s Dilemma

Unfortunately, most of the work that has gone into Darwin thus far (understandably) has been in developing it for recent Apple hardware. While an Intel x86 port exists (but is still as of this writing in embryonic stage for hardware support), and older PowerPC Mac hardware support will undoubtedly extend over time, the current release of Darwin is not officially supported for Mac hardware older than G3 Power Macs. This sadly eliminates (at least for now) using Darwin to make a great server out of any old Mac hardware you have sitting on a shelf.

When you buy a new Mac, you’re paying for things (like full MacOS X, an ATI Radeon or nVidia GeForce video card, optical mouse, etc.) that you aren’t going to care about as a server admin. While new Mac systems are surprisingly powerful considering their CPU clock speed (I would sooner compare a G4 to an UltraSPARC III than a Pentium III), you still won’t get the same performance dollar-for-dollar as you would with commodity x86 hardware and a free OS. 

As a result, buying a new Mac just to make into a Darwin server simply isn’t worth the money. If you love the GUI tools of MacOS X, it may be worthwhile (I’m personally salivating over the new Titanium PowerBook G4); otherwise, it still doesn’t make bottom-line dollars and sense to purchase a new Mac as a Darwin server.

As Darwin expands its processor base, this may change. In the meantime, it’s well worth your while to keep an eye on the Darwin project, and to get to know it, since some of its features are well worth adopting by the other BSDs and Linux.

Some of proceeding items stem from a series of articles that BSD guru Matt Loschert and I wrote for the BSD news site Daemon News (see www.daemonnews.org/200011/ and www.daemonnews.org/200012/ for excessively long and drawn-out versions). 😉 For great info on Darwin, you can skip its home page (www.opensource.apple.com) and go straight to Darwin Info (www.darwinfo.org). Also, check Xappeal (www.xappeal.org) and Apple’s Darwin lead Wilfredo Sanchez’s updates page (www.advogato.org/person/wsanchez). As always, let me know about any comments, corrections or suggestions you have, and I’ll publish them in a future column.

Webhosting with Free Software Cheat Sheet

By Jeffrey Carl

Boardwatch Magazine
Boardwatch Magazine, April 2001

Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among ISPs, particularly for its annual ranking of Tier 1 ISPs and through the ISPcon tradeshow. Writing and speaking for Boardwatch was one of my fondest memories of the first dot-com age.

So you want to run a webserver without paying a dime for software, eh? Or you want to make sure you have the source code to all your webserving applications in case you get bored and decide to try and port them to your networked office Xerox copier? Well, you’re in luck; webhosting with free (open-source, or free as in “free speech”) and free (no cost, or free as in “free beer”) software isn’t just possible, it also provides some of the best tools out there at any price.

In case you’re new to the game, or you’re looking for alternatives to packages you’re using now, the following is a brief guide to some of the more popular options that are out there. Trying to condense the features of any OS or application down to a couple sentences is inherently a dangerous thing; and I’m sure that many fans of the software listed below will disagree with elements of my “Reader’s Digest Condensed (Large Print Version)” summaries. Still, the following – based on my experiences and those of others – should provide at least a basic idea of what’s out there and why you might – or might not – want to choose it.

Operating Systems

Linux vs. BSD:

These OSes show the characteristics of their development styles: BSD was developed by small teams, largely focused on server hardware. Linux has been developed by many more people with wider uses, focusing more on desktop/workstation uses. 

BSD has been around longer and is (in some ways) more optimized for server use. Due to its hype, Linux has many more developers, and almost all new third-party software is available for Linux first. Linux has the edge in user-friendliness, because distributions are targeting new users; BSD is, for the most part, more for the “old school.” Linux has also been adopted by a number of server hardware vendors producing “integrated” solutions.

Ultimately, it’s a matter of what you feel most comfortable with. Either way, with commodity x86 hardware, your server components (RAM, drives, etc.) and network connection will affect your performance much more than your choice of Linux vs. BSD will.

• FreeBSD (www.freebsd.org)

Best known among the BSDs. Concentrates on x86 architecture, server performance, integration of utilities. Standout features include ports collection, sysinstall admin utility, Linux binary compatibility, frequent releases.

• NetBSD (www.netbsd.org)

BSD with a focus on porting it to as many platforms as possible and keeping code portable. Great for using old/odd hardware as a server. Infrequent releases, not as popular as other BSDs.

• OpenBSD (www.openbsd.org)

BSD with a focus on security. Still in the process of line-by-line security audit of the whole OS. Infrequently released, utilities/packages lag behind other OSes because of security audits, but it’s the #1 choice if security is your primary concern.

• Red Hat Linux (www.redhat.com) 

The number one U.S. distro, considered by many (rightly or wrongly) as “the standard.” As a result, it’s what many third-party/commercial Linux apps are tested against/designed for. Early adopter of new features in its releases; is on the cutting edge, but sometimes buggy until “release X.1.” Standout features: Red Hat Package Manager (RPM) installation, third-party support.

• SuSE Linux (www.suse.com)

The number one European distro. Favored by many because its six-CD set includes lots and lots of third-party software to install on CD. Less “cutting-edge” than Red Hat. Standout features include the YaST/YaST2 setup utility and the SaX X Windows setup tool.

• Slackware Linux (www.slackware.com)

Designed for experts: Slackware has no training wheels, and is probably the most “server-oriented” of Linux distros (maybe because of its close relationship to the BSDs). Not cutting-edge, few frills, but designed to be stable and familiar to BSD administrators.

• Linux Mandrake (www.linux-mandrake.com/en)

A solid, user-friendly distribution with good (but not great) documentation. Standout features include the DrakX system configuration utility and the DiskDrake disk partitioning utility.

• Debian GNU/Linux (www.debian.org)

The ideological Linux – totally supported by users rather than a corporation, and free (as is the GNU definition) software only is included. This is “ideologically pure” Linux – GNU-approved, but infrequent releases and not necessarily a good choice for beginners.

• Caldera OpenLinux (www.caldera.com/eserver)

Very user-friendly for new users. Standout features include LIZARD, its setup/configuration wizard.

• Corel LinuxOS (linux.corel.com)

By the time you read this, Corel will have sold its LinuxOS product to someone else, but the distro should remain the same. Ease of use for Windows converts is stressed, includes great SAMBA integration. Good for new users. Focus is mainly on desktop use.

Essentials

• Perl (www.perl.com)

From CGI to simple administration tasks, Perl scripts can cover a lot of territory. Perl is a must, and is practically part of Unix now. Check the Comprehensive Perl Archive Network (www.cpan.org) to find modules to extend Perl’s functionality.

• Perl’s cgi-lib.pl (cgi-lib.berkeley.edu) and/or CGI.pm (stein.cshl.org/WWW/software/CGI)

These are also “must-haves” for CGI scripts, whether you’re writing your own or using scripts found “out there” on the web.

• Sendmail (www.sendmail.org) or Qmail (www.qmail.org)

Free mailservers. Sendmail has the history and the documentation (a good thing, since its internals are famously complex), but Qmail has a less-complicated design, and a strong and growing band of followers.

• wu-ftpd (www.wu-ftpd.org)

A significant improvement in features over the classic BSD FTP daemon – for both BSD and Linux. Despite an older security flaw that was recently exploited by the “Ramen” Linux worm, it’s a very good program.

• OpenSSH (www.openssh.com/)

In this day and age, Telnet has become a liability for security reasons. There’s no reason not to migrate users who need a shell account to SSH. See www.freessh.org for a list of clients.

Web Servers

• Apache 1.3.x (www.apache.org/httpd.html)

The current king of web servers. Very good performance, stable enough to run on mission-critical systems. Very user-friendly to install and configure (due to comments in httpd.conf), but not always as easy as it should be to debug problems.

• Apache 2.x (www.apache.org/httpd.html)

Still in beta development, but may be final by the time you read this. It probably shouldn’t be used for mission-critical systems until it’s had a few months of time “out there” to find bugs after its final release. Version 2.0 will be much easier to add new protocols into (like FTP or WAP), and should have significantly better performance because of its multi-threaded nature.

• Roxen (www.roxen.com/products/webserver)

Roxen is much more than just a simple webserver – it includes its own web admin interface, secure server, and more. Used by real.com; shows promise but doesn’t have the acceptance level yet of Apache.

Secure Web Servers

Note: You may receive a “security warning” in most web browsers about your secure certificate if you generate your own secure certificate (free). For a non-free certificate created by an authority that most web browsers will accept without a warning, see VeriSign (www.verisign.com/products/site/ss/index.html), Thawte (www.thawte.com), Baltimore (www.baltimore.com/cybertrust), ValiCert (www.valicert.com/), Digital Signature Trust Co. (www.digsigtrust.com) or Equifax (www.equifaxsecure.com/ebusinessid) for more information.

• ApacheSSL (www.apache-ssl.org)

Focused on stability/reliability, and lacking in “bells and whistles” features. It’s simple and it works, but it lacks some features of mod_ssl and it isn’t updated very often.

• mod_ssl (www.modssl.org)

Originally based on ApacheSSL, mod_ssl is now largely rewritten and offers a number of extra features, plus better documentation. 

Microsoft Web Compatibility

• FrontPage Extensions for Unix (www.rtr.com/fpsupport)

On one hand, it allows you to host sites built and published with FrontPage on a Unix server. On the other hand, it’s possibly the biggest piece of junk Unix software ever created. Use it if you have to; avoid it if you can.

• Improved mod_frontpage (home.edo.uni-dortmund.de/~chripo)

Addresses a number of problems with mod_frontpage (www.darkorb.net/pub/frontpage), with extra security and documentation, support for Dynamic Shared Objects (DSOs), better logging, as well as (unverified) claims of increased performance.

• Apache::ASP (www.nodeworks.com/asp)

An alternative to the very expensive ChiliSoft or Halcyon ASP Unix solutions, using Perl as the scripting language for ASPs. Requires the Apache mod_perl.

• asp2php (asp2php.naken.cc)

As its FAQ says, “ASP2PHP was written to help you correct the mistake of using ASP.” Converts ASP scripts to PHP scripts for use with Apache/PHP.

Application Building

• Zope (www.zope.org)

A complete tool for building dynamic websites; there’s a (somewhat) stiff learning curve that may be too much for basic needs. Zope offers incredible functionality, and is well-suited to large projects and web applications; it may be overkill for simple scripting that could be done with PHP or Perl CGIs.

• PHP (www.php.net)

The favorite open-source tool for building dynamic websites, and the open-source alternative to ASP. Reliable, uses syntax that seems like a cross between Perl and C, and features native integration with Apache. Version 4 is thread-safe, modular, and reads then compiles code rather than executing it as it reads (making it much faster with large, complex scripts).

Database Software

Note: for a more in-depth comparison, I highly recommend the O’Reilly book MySQL and mSQL, as well as the article “MySQL and PostgreSQL Compared” (www.phpbuilder.com/columns/tim20000705.php3).

• MySQL (www.mysql.com)

The “Red Hat” of free relational database software. Well-documented, and its performance for most users is excellent, designed around fast “read” rather than “write” operations. It doesn’t offer “subselect” functionality, and tends to buckle under very heavy loads (more than 15 concurrent users per second), but is very fast and reliable for most sites.

• PostgreSQL (www.postgresql.org)

Has an active developer community, especially popular among the “GPL-only” crowd. Offers advanced features that MySQL doesn’t (subselects, transactional features, etc.), but traditionally wasn’t as fast for common uses and sometimes suffered data corruption. New versions appear to have remedied most of these deficiencies.

• mSQL (www.hughes.com.au) 

The first of the bunch, but appears to have fallen behind. More mature than MySQL or PostgreSQL, but may not have all of the features of its rapidly developing brethren.

Administration

• Webmin (www.webmin.com/webmin)

Fully featured web-based administration tool for web, mail, etc. Offers excellent functionality, but can present a potential security risk (I get really nervous about anything web-accessible which runs with root permissions).

Java Servlets

• Tomcat (jakarta.apache.org) and JServ/mod_jserv (java.apache.org)

Tomcat is an implementation of the Java Servlet 2.2 and JavaServer 1.1 specifications that works with other browsers as well as Apache. JServ is an Apache module for the execution of servlets. The two work together to serve JSPs independently of Apache.

Website Search

• ht://Dig (www.htdig.org)

ht://dig is relatively simple to set up, and (with a few quirks) offers excellent searching capabilities. Easily customizable, and has a good “ratings-based” results engine.

• MiniSearch (www.dansteinman.com/minisearch)

A simple Perl search engine, which can also be run from the command line. Not as fully featured as ht://dig, but good enough for basic needs.

Web Statistics

• analog (www.statslab.cam.ac.uk/~sret1/analog)

Analog is extremely fast, reliable and absolutely filled with features. Its documentation is a bit confusing for beginners, however, and it takes some configuration to make it pretty.

• wwwstat (www.ics.uci.edu/pub/websoft/wwwstat)

A no-frills, simple statistics analysis program that delivers the basics.

Other Goodies

Configuration Scripts:

• Install-Webserver (members.xoom.com/xeer)

• Install-Qmail (members.xoom.com/xeer)

• Install-Sendmail (members.xoom.com/xeer)

Shopping Carts:

• Aktivate (www.allen-keul.com/aktivate)

Aktivate is an “end-to-end e-commerce solution” for Linux and other Unixes. It is targeted at small-to-medium-sized businesses or charities that want to accept credit card payments over the Web and conduct e-commerce. 

• OpenCart (www.opencart.com)

OpenCart is an open source Perl-based online shopping cart system. It was originally built to handle the consumer demands of Walnut Creek CDROM, was later expanded to also work with The FreeBSD Mall, and was finally developed to be used by the general public.

• Commerce.cgi (www.careyinternet.com)

Commerce.cgi is a free shopping cart program. Included is a Store Manager application to update program settings, and you can add/remove products from the inventory through a web interface.

Message Boards:

• WaddleSoft Message Board (www.ewaddle.com)

WaddleSoft is a message board system that includes polls, user registration, an extensive search engine, and sessions to track visitors.

• MyBoard (myboard.newmail.ru)

MyBoard is very easy and light-weight web messageboard system. It also has some extended features such as search and user registration.

• NeoBoard (www.neoboard.net)

NeoBoard is a Web-based threaded message board written in PHP. It includes a wide variety of advanced features for those comfortable with PHP.  

• PerlBoard (caspian.twu.net/code/perlboard)

PerlBoard is a threaded messageboard system written in Perl. It is very easy to use and set up, and has been time-tested for the past several years on the site it was originally written for.

• RPGboard (www.resonatorsoft.com/software/rpgboard)

RPGBoard is a WWWBoard-style message board script. It includes a list of features as long as your arm, and is well worth checking out for those who need a rather advanced message board.

Notes from the Underground

If you see a favorite package here that I’ve overlooked, or would like to offer comments on any of the package descriptions, e-mail me at [email protected]. I’ll update this list with more information for a future column.

Running Linux Programs on FreeBSD

By Jeffrey Carl

Boardwatch Magazine
Boardwatch Magazine, January 2001

Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among ISPs, particularly for its annual ranking of Tier 1 ISPs and through the ISPcon tradeshow. Writing and speaking for Boardwatch was one of my fondest memories of the first dot-com age.

Even though many server admins prefer BSD Unix, there’s no denying that Linux is “where it’s at” for third-party software development. So, what’s a BSD admin to do?

In the bygone misty past of Unix (in the era historians call “the early ‘90s”), acceptance and market share were hampered by the diverging standards that forced application developers to write for only one version of Unix. This (as the youngsters today say) “really sucked,” and resulted in the Unix wars that left many syadmins hospitalized with “flame” wounds from heated Usenet exchanges. However, the tide has fortunately turned in favor of compatibility among the many *nixes.

To commercial software vendors, market share is everything. As such, if *BSD forced Linux vendors to develop for their OSes, it would be like your average Unix sysadmin saying, “If Cindy Crawford wants to go out, she can call me.” Fortunately, Jordan Hubbard of FreeBSD has indicated a prudent recognition that Linux’s API is becoming the default for third party developers (Indpendent Software Vendors, or ISVs) to develop for free *nixes. Therefore, it’s better for BSD users to accept the Linux ABI (Application Binary Interface) rather than to force ISVs to choose between Linux and BSD. This, in my opinion, is a very pragmatic and wise attitude.

FreeBSD, NetBSD and OpenBSD all include options to allow Linux binaries to run on their systems (at least for the Intel x86 architecture; on other architectures, your mileage may vary). For OpenBSD and NetBSD, you can check your system documentation or see their respective websites (www.openbsd.org and www.netbsd.org); for the purpose of this article, we’ll look at Linux binary compatibility on FreeBSD for x86. NOTE: Much of the information on the inner workings of Linux compatibility came from information graciously provided by Terry Lambert ([email protected]).

Under FreeBSD, Linux binary compatibility is managed by creating a “shadow” Linux file system with the requisite programs and libraries, and re-routing the Linux program’s system calls into this file system. These libraries then deal with a Linux kernel embedded into the FreeBSD kernel.

Setting Up Linux Compatibility

The development of an “executable class” loader for FreeBSD has been under development since 1994 (although when the project was started, Linux wasn’t one of the primary target OSes). There are three essential items that enable this functionality.

The first is a KLD (“Kernel LoaDable”) object called linux.ko, which is essentially a Linux kernel that can be loaded dynamically into the FreeBSD kernel. As a KLD, it can be loaded or unloaded without rebooting, using the kldload and kldunload commands. To check and see whether linux.ko is loaded properly on your system, use the kldstatcommand:

schnell# kldstat

Id Refs Address    Size     Name

 1    4 0xc0100000 1d9a60   kernel

 2    1 0xc1038000 3000     daemon_saver.ko

 3    1 0xc1822000 4d000    nfs.ko

 4    1 0xc1020000 10000    linux.ko

The second necessary item is the set of basic Linux libraries (pre-existing code that developers can link their programs to, so they don’t have to write every function from scratch) and binaries (like bash, cp, rm, etc.). These can be installed via the ports collection from /usr/ports/emulators/linux_base, which is based on a minimal Red Hat installation. As of this writing in late September, for the i386 architecture, the installed libraries matched a Red Hat 6.1-1 release including glibc 2.1.2-11, libc 5.3.12-31, glib 1.2.5-1, ld.so 1.9.5-11, libstdc++-2.9.0-24, gdbm-1.8.0-2 and XFree86 libs 3.3.5-3. 

To be able to use a wider range of Linux apps, you may also wish to install the Linux development tools, found in the ports collection at /usr/ports/devel/linux_devtools. Note that you’ll want to do this after installing linux_base, since linux_devtools requires rpm (the Red Hat Package Manager application, also available as a FreeBSD app through the ports collection) and /compat/linux/etc/redhat-release to be installed. This currently includes (also for i386) kernel-headers-2.2.12-20, glibc-devel-2.1.2-11, make-3.77-6, cpp-1.1.2-24, egcs (plus egcs-c++ and egcs-g77)-1.1.2-24, gdb-4.18-4, and XFree86-devel-3.3.5-3.

Also, any necessary binaries/libraries can be set up manually by installing them into their original paths under the FreeBSD directory /compat/linux (e.g., install the Linux /usr/X11R6/lib/libX11.so.6.1 library onto FreeBSD as /compat/linux/usr/X11R6/lib/ libX11.so.6.1). This process is also necessary for installing libraries that aren’t part of the default ports collection sets.

The third item is “branding” Linux ELF (“Executable and Linking Format”) binaries. Although the GNU toolchain now brands Linux ELF binaries with the OS name (so newer Linux binaries should work without branding), some older binaries may not include this information (although ELF has replaced a.out as the primary binary format since Linux kernel 2.x and FreeBSD 3.0). To do this, use the brandelf command (which sounds like a move from “Dungeons and Dragons” to poke elves with hot sticks) with the “-t” (type) flag like:

schnell# brandelf -t Linux /usr/local/games/doom

How Linux Compatibility Works

Originally, Unix supported only one “binary loader.” When you ran a program, it examined the first few bytes of the file to see if it was a recognized binary type. If the file type that was attempted to being loaded and run wasn’t the singular binary type that it recognized, it would pass the file in question to the basic shell interpreter (/bin/sh) and try to run the program with that. Failing that, it would spit a nasty message back to you about how you couldn’t run whatever it was that you were trying to run.

Nowadays, FreeBSD supports a number of loaders, which include an ELF loader (by the way, Linux binaries can be stored on either a FreeBSD UFS or Linux ext2 file system, assuming you have mounted the ext2 file system the app is located on using the mount -t ext2fs command). If the “brand” is seen in the ELF file as “Linux,” it replaces a pointer in the “proc” structure of the binary to point automatically to /compat/linux/, and the binary is executed using the Linux libraries and binaries, instead of the FreeBSD ones. If it isn’t a FreeBSD or Linux binary, and the list of loaders (including those examining the first line for “#!” lines like Perl, Python or shell scripts) is exhausted without recognizing a “type,” then it is attempted to be executed as a /bin/sh script. If this attempt to run is unsuccessful, it is spit back out as an “unrecognized” program.

Linux and BSD Binaries: Both “Native?”

If a binary or library is required by a Linux program that is not included under the Linux compatibility “area,” the FreeBSD tree is searched. Thus, a Linux application that is expected to run with the Linux version of /compat/linux/usr/lib/libmenu.so – if it doesn’t find it – is checked to see whether it can run under the FreeBSD /usr/lib/libmenu.so. Only if a compatible library is not found in either the Linux-compatible “re-rooted (/compat/linux/whatever)” filesystem or the regular FreeBSD libraries is an executable rejected. 

Because of this construction, it isn’t really accurate to say that Linux is “emulated” under FreeBSD, especially in the sense that, say, platforms are emulated under the MAME (www.mame.net) emulator, since it doesn’t involve a program which impersonates another platform on a binary level and translates all of its native system calls into local system calls. Linux binaries are simply re-routed to a different set of libraries, which interface natively with a (Linux kernel) module plugged into the FreeBSD kernel. Rather than “emulation,” this is largely just the implementation of a new ABI into FreeBSD.

Under the current implementation of binary compatibility, programs calling FreeBSD’s glue() functions are statically linked to FreeBSD’s libraries, and programs calling the Linux glue() function calls are statically linked to its own libraries, so their executions are dependent on completely different code libraries; however, the importance of glue()is on the wane. In the future, this might change, which would seriously open the question as to whether Linux apps under FreeBSD are any more “native” than apps written specifically for FreeBSD are. 

Running Linux Binaries on BSD

Currently, the major problem with running Linux binaries on FreeBSD is similar to the major problem with using new apps on Linux: discovering which libraries they require and downloading and installing them. Unfortunately for BSD users, the only certain way to do this right now is to have access to a Linux system which already has the app in question installed, and run the ldd command on the application to list the libraries needed. If you don’t have access to a Linux system that already has the program installed, your next best bet is to check info on the program’s website, or search for it at Freshmeat (www.freshmeat.net).

Under current FreeBSD Linux binary emulation, most Linux binaries should work without problem. However, due to operating system differences, it won’t support those that heavily use the Linux /proc filesystem (which operates differently from the FreeBSD /proc), ones that make i386-specific calls like enabling virtual 8086 mode, or ones that use a significant amount of Linux-specific assembly code. While some people have reported that some Linux binaries run faster in this BSD environment than they do natively in Linux, this is a highly subjective claim, and one that should be taken with a grain of salt (at least until some real standardized testing is done with it). While Linux binary emulation on FreeBSD shouldn’t be considered as a major security risk, it is worth noting that a buffer overflow bug was discovered in August 2000 (www.securityfocus.com/frames/?content=/vdb/bottom.html%3Fvid%3D1628).

Popular Uses for Linux Compatibility

The most popular binaries for using Linux compatibility are the ISV applications that have been developed for Linux from popular commercial/semi-commercial vendors. These include programs like Sun’s StarOffice, Wolfram’s Mathematica, Oracle 8, and games like Bungie’s Myth II and id Software’s Quake III (there are already tutorials on installing Oracle 8.0.5 and Mathematica 4.x included in the FreeBSD Handbook; see below for more). 

Other popular installations include the Netscape Navigator 4.6-7.x browser. Particular advantages to installing the Linux version (even though there is a FreeBSD version already) are that the Linux version under FreeBSD compatibility is reportedly more stable than the “native” FreeBSD version. Also, binaries for popular plug-ins (Flash 4, Real Player 7, etc.) are available, since “native” FreeBSD versions aren’t. Info on installing VMware for Linux on FreeBSD can be found at www.mindspring.com/~vsilyaev/vmware/.

Getting Help with Linux Compatibility

The primary reference on Linux binary emulation on FreeBSD is the “handbook” page at www.freebsd.org/handbook/linuxemu.html, including specific pages on Oracle (www.freebsd.org/handbook/linuxemu-oracle.html) and Mathematica (www.freebsd.org/handbook/linuxemu-mathematica.html). To keep up-to-date on Linux emulation under FreeBSD, subscribe to the mailing list freebsd-emulation (send e-mail to [email protected] with the text [in the body] subscribe freebsd-emulation).

Additional information on Linux compatibility (including instructions for older versions of FreeBSD), see www.defcon1.org/html/Linux_mode/linux_mode.html. For a list of Linux applications that are known to run on FreeBSD under Linux binary compatibility, see www.freebsd.org/ports/linux.html. For an example of setting up Word Perfect 8 in Linux compatibility – as well as an account of running Windows 2000 on top of VMware on top of Linux on top of FreeBSD, read the excellent article at BSD Today (www.bsdtoday.com/2000/August/Features252.html). You can find some (mildly outdated but still useful) instructions on installing StarOffice on FreeBSD at www.stat.duke.edu/~sto/StarOffice51a/install.html. And, for fun, there’s info on setting up an Unreal Tournament server on FreeBSD via Linux compatibility at www.doctorschwa.com/ut/freebsd_server.html.

The Continuing Evolution of Darwin – Apple’s BSD Unix Has Lots of Promise, and Lots of Rough Edges

By Jeffrey Carl

Boardwatch Magazine
Boardwatch Magazine, September 2000

Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among ISPs, particularly for its annual ranking of Tier 1 ISPs and through the ISPcon tradeshow. Writing and speaking for Boardwatch was one of my fondest memories of the first dot-com age.

There’s a new BSD out there, and it’s unquestionably the odd member of the family. “Darwin” is the BSD Unix core of Apple’s next generation OS, MacOS X (see page 39 for the lowdown on OS X). While MacOS X (which includes Darwin) is a proprietary commercial product, Darwin by itself is open-source and available for free.

So why should you care (or think about deploying it)? First, it’s being developed for both PowerPC and Intel x86 architectures. Second, it’s being developed not only by its user community by also by Apple – which, like ‘em or hate ‘em, has considerable manpower and money to throw at the project. 

Lastly, Apple is scheduled to start shipping MacOS X on new Macintoshes in January 2001, as well as making it available for current machines – meaning that there will probably be a couple million boxes out there with Darwin at their heart within a year. While that’s no guarantee that all of your favorite software will rush to support it or optimize for it, it will certainly have the user numbers to make it worth noticing.

What is Darwin?

To get a feel for where Darwin is and where it’s headed, I asked Ernie Prabhakar, Apple’s Developer Platforms Product Line Manager.

Q: Can you give me a brief history of Darwin (including its roots in NextStep/ Rhapsody/Mac OS X Server)?

Ernie Prabhakar: Darwin is based on the Mach/BSD technology that traces its roots back through Mac OS X Server, Rhapsody, OpenStep for Mach, NextStep, and ultimately the original Mach 2.0 work at CMU.  In all the NeXT/Apple products, this UNIX core provided the support for advanced GUIs and development environments.  Over time, we’ve evolved from BSD 4.3 to BSD 4.4, and from Mach 2.0 to Mach 2.5 and ultimate Mach 3.0 (based largely on OSF MK 7.3). 

Q: Who is Darwin’s intended audience of consumers?

EP: Darwin is targeted at Macintosh developers who want to leverage the power of open source, as well as Open Source developers who want to enjoy the performance, consistency, and advanced GUI available on the Apple platform. Certain large customers (e.g., research universities) will take advantage of the open core to customize it to their environments, as well as helping students and researchers studying open source.

Q: What is the relationship between Darwin, its BSD kernel, and the Mach microkernel? 

EP: Darwin is the entire product, including a hybrid Mach/BSD kernel plus assorted utilities (mostly from FreeBSD) and developer tools (primarily from the GNU project). 

Q: How does using Mach affect its characteristics or performance?

EP: Very little. We use a microkernel architecture but a monolithic implementation, so Mach and BSD normally run in the same address space.

Q: Where does Darwin come from? Which parts are built around each of the different BSDs and/or NextStep?

EP: The current 1.0 version of Darwin actually shares very little code with the original NextStep implementation.  The Mach microkernel is actually based on the OSF code used in MkLinux.   The BSD kernel is based on BSD 4.4Lite, primarily the FreeBSD 3.2 distribution, with a healthy dose of NetBSD.

Q: What does Darwin add to the mix? Are there any innovations new to *BSD in Darwin?  

EP: For an existing BSD developer, Darwin has the advantages of:

• a powerful kernel extension mechanism and I/O Kit for writing drivers

• traditional Apple technology (HFS+, AppleTalk)

•  a well-funded commercial developer with a single well-supported reference release 

•  a very cool optional (though proprietary) GUI called Aqua (i.e., Mac OS X) 

Q: Is there any other OS which Darwin is trying to match or beat in features?

EP: Darwin’s top priority is making Mac OS X the world’s best personal computer operating system.  Beyond that, we want to help our developers turn Darwin itself into a fully-functional standalone open source operating system, compatible with FreeBSD and comparable to Linux. 

Q: Will Darwin include features like FreeBSD’s sysinstall or ports tree?

EP: Actually, we already ship Darwin with the Debian packaging mechanism.

Q: Is Darwin POSIX-compliant? How well should existing open-source apps (especially server apps like Apache, sendmail, qmail, PHP, etc.) compile on it or port to it?

EP: Most BSD and GNU stuff compiles out of the box. The major limitation used to be pthreads, but that was pretty much fixed in Darwin 1.0. We have a very active developer community that is ensuring third-party UNIX software is becoming “Darwin-ready” which usually means just defining the appropriate #defines and makefiles.

Q: Does Darwin bring any particular optimizations or improvements that would make it good from an ISP/Internet server perspective?

EP: Well, we have a dedicated team of networking engineers focusing on giving it best-of-breed performance integrated with Apple hardware. As well as a really cool GUI for administration (if you buy Mac OS X). Other than that, we use generally the same code as FreeBSD and NetBSD, which is already pretty thoroughly optimized.

Q: Are there any reasons you could think of that someone might choose to use Darwin instead of FreeBSD/NetBSD/etc.?

EP: Several: 

1.  We will be shipping Darwin (as Mac OS X) on millions of machines next year, giving developers an enormous potential user base.

2.  You have the option of purchasing a very cool UI (Mac OS X).

3.  We have excellent support for the latest high-performance PowerPC hardware 

Q: How much of a priority is the x86 port of Darwin?

EP: A lot of Darwin developers are excited about being able to use the exact same open source OS on both PowerPC and Intel hardware, so we’re doing what we can to help.  We demonstrated some major progress at WWDC [the Apple World Wide Developers’ Conference], and I think the community is gearing up to finish the work on their own.

Where is Darwin Now?

As of this writing, the current version of Darwin is 1.0.2. Darwin currently boots on Intel hardware, but there are a number of unresolved issues right now which keep it from being genuinely functional. Apple provides a binary release and installer for Darwin on the PowerPC architecture (located at http://www.publicsource.apple.com/projects/darwin/release.html), and it current supports G3/G4 Macintoshes as well as a few older desktop models. The PowerPC version sits entirely inside a single UFS or Apple HFS+ disk partition (current limitations of the Darwin booter limit the boot volume size to 8 GB).

Once you install Darwin, you can tell pretty quickly that most of Apple’s efforts are (understandably) going towards making Darwin work with MacOS X, rather than on its own. (In fact, most of the Darwin projects in its CVS repository are live and being used by the Apple MacOS X team at the same time as the Darwin community.) This is most obvious in the fact that the Darwin-specific documentation and install/setup goodies included with the OS are about zero – Apple is working on the MacOS X GUI to include these things.

When you install Darwin 1.0.2, you’ll boot and be dropped off into a tcsh shell. From there, you’re on your own – don’t expect any sysinstall or LinuxConf-style goodies to help with setup and configuration. For most experienced administrators, this wouldn’t pose too much of a problem, except that Darwin has a lot of its files stashed in MacOS X-centric locations that take a while to puzzle out. For example, don’t bother looking in /etc/httpd or /usr/local for a httpd.conf file – you’ll find it as /Local/Library/WebServer/Configuration/apache.conf.

At this point, Darwin is certainly more of a toy for developers and “early adopters” than an OS worth implementing for an Internet server. OpenSSH hasn’t been finished up yet, nntpd and some other server staples seem to be MIA, there are plenty of device drivers missing, and there’s plenty of work yet to be done on the Intel side. Documentation is still pretty sparse, but some good stuff is developing out there, including a helpful “Unofficial Darwin FAQ” at http://www.synack.net/darwin/faq.html. You can keep up on Darwin’s progress by subscribing to any of the Darwin mailing lists at Apple (http://www.publicsource.apple.com/projects/mail.html) or, if you’re lazy like me, checking out project leader Wilfredo Sanchez’s updates to his Advogato page (http://www.advogato.org/person/wsanchez/).

Darwin is very new, and time will take care of many of its current deficiencies (many of the ones I’ve described will almost certainly be fixed by the time you read this). Despite its shortcomings, Darwin 1.0.2 is a fully functional BSD Unix system, and the default install includes several of the third-party packages you’d hope to find like Apache, Sendmail, Perl and BIND. Ports for XFree86 (much of the original work on this port was done by id Software’s John Carmack), OpenSSL and other common tools are available in source form from the Darwin CVS repository. 

Where is Darwin Going?

Darwin’s promise lies not in where it is now, but where it will be. Darwin can catch up on plenty of things by grabbing code from the other BSDs, and it is likely only a matter of time before Darwin develops rough feature parity with some of your favorite BSDs. The Apple Darwin team has said that future releases of Darwin will try to track FreeBSD; Jordan Hubbard was one of the speakers at the BSD session at May’s Apple World Wide Developer Conference. The IOKit driver framework that Apple has created for MacOS X has a lot of potential for (relatively) easy device driver development. 

The real promise lies in the possibility that the large installed user base of MacOS X systems that will be out there will drive developers (especially commercial ones) to bring their server apps natively to a BSD-based system for the first time. Of course, porting something to MacOS X isn’t the same as porting to BSD; most commercial applications coming to MacOS X will use its API rather than the Darwin/BSD API. Nonetheless, even if some of these apps which currently exist for [insert your least-favorite commercial Unix here] are ported to the Darwin layer of MacOS X, it will be a victory for BSD. 

The close relationship between Apple’s Darwin team, the Darwin community, and the FreeBSD community is an excellent sign for Darwin’s future. While it may not be much more than a curiosity right now, Darwin is a welcome addition to the free *BSD family, and will definitely be something to watch in the next year.

Trusting BSD FreeBSD for Security Ultra-Gurus

By Jeffrey Carl

Boardwatch Magazine
Boardwatch Magazine, August 2000

Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among ISPs, particularly for its annual ranking of Tier 1 ISPs and through the ISPcon tradeshow. Writing and speaking for Boardwatch was one of my fondest memories of the first dot-com age.

While most Freenix admins are used to securing their servers, there is a “higher” world of security that has never been touched by free *nixes. The realm of “trusted” operating systems, long the province only of military and other ultra-secure environments, represents a security level beyond that of all but a few operating systems. Now, however, the TrustedBSD (http://www.trustedbsd.org/) project is working to bring that security level to FreeBSD and other *BSD OSes. If you operate a multi-user environment and you’re looking for the optimum in security, then learning about TrustedBSD is a very exciting project. 

I asked Robert Watson, one of TrustedBSD’s lead developers, to describe the project for Internet providers and other BSD server users:

Q: What is the purpose of TrustedBSD? 

A: TrustedBSD provides a set of security extensions to the FreeBSD operating system under a liberal license. Targeting both military and commercial models, TrustedBSD includes traditional trusted operating system features, as well as an extensible authorization architecture to simplify development of new security features. The TrustedBSD project provides an organizational framework through which to discuss, design, develop, and document these extensions.

Q: What is the Common Criteria for Information Technology Security Evaluation (CCITS)? What does it mean to “real world” uses of the OS? 

A: The Common Criteria (“CC”) are a set of security description and evaluation documents developed by the United States and other governments, as well as ISO standards. Using the CC, you can use a common terminology to describe specific sets of features and degrees of evaluation. Many readers will be familiar with the Orange Book (and entire Rainbow Series) which can be considered precursors to the CC in the United States. While the Orange Book largely targeted military applications, the CC really provides a language for setting goals and determining if they have been achieved, rather than prescribing a particular set of features required for a given certification. Unlike the Orange Book, TrustedBSD specifically targets commercial and network-centric environments. 

TrustedBSD is being developed with this vocabulary and evaluation system in mind, as the CC by definition provides a common criteria for understanding security products. At this time, no formal evaluation is planned, as formal evaluation requires substantial investment of resources, including financial resources. However, TrustedBSD will make an excellent candidate for evaluation. It is possible that companies choosing to resell TrustedBSD may seek formal evaluation [as a “trusted” operating system].

Q: What benefits will the TrustedBSD extensions to FreeBSD provide to admins using FreeBSD as an Internet server? 

A: It is fair to say that some features of TrustedBSD will have an immediate impact on securing every-day server systems. Other features will come into play only in extremely security-centric contexts, such as military, electronic commerce, and banking environments. Regular systems may see the use of least privilege capabilities rather than volumes of less safe setuid and setgid programs. Similarly, they may take advantage of ACLs on files allowing more flexible discretionary access control. They may also take advantage of auditing features for intrusion detection. However, they are less likely to take advantage of more intrusive functionality, such as the confidentiality and integrity-oriented mandatory access policies that will be implemented. 

Q: Why should an administrator of a FreeBSD web/mail/etc. server use the TrustedBSD extensions to the OS? 

A: It is the intent of the TrustedBSD project to make almost all code developed for TrustedBSD available as part of the base FreeBSD operating system. Thus, all users of FreeBSD will benefit from the project by virtue of using FreeBSD. However, specific advantages, as described above, include the ability to more generally specify permissions on files and allow users to more easily manage resources, and to be able to run far less code with root privileges. As such, administrators taking advantage of this functionality will be able to run a more secure system. 

A recent paper from Argus Systems, a commercial provider of trusted extensions for Solaris, described how use of mandatory integrity policies could have protected against the recent penetration of the Apache Project’s web server. Many of the same protection services will be available as part of TrustedBSD. 

Q: What is the downside, if any, in installing the TrustedBSD extensions? Is administration significantly more complex? Are there any performance penalties involved? 

A: Improved security is almost always a tradeoff against improved usability. That said, one important goal of TrustedBSD is to make the feature set of a trusted operating system easily accessible to users of a widely distributed free operating system. Any changes in system behavior will require administrators to understand the differences, but in most cases the differences will not be as substantial. Administrators will need to learn how to read, manipulate, and back up Access Control Lists (ACLs) on files, and understand how capabilities on files behave. 

Performance implications depend on the feature in question: support for most new security checks introduces little or no overhead. However, features such as fine-granularity auditing require creation and management of large quantities of data. Performance-sensitive sites may wish to avoid using some features, such as auditing, as a result. We anticipate producing comprehensive evaluations of the performance impact as code becomes available. 

Q: Are the TrustedBSD extensions planned to be architecture-specific? e.g., are the efforts of TrustedBSD only for i386 machines, or are they portable to other architectures that FreeBSD may be ported to? 

The code base is intended to be architecture-neutral, and is written entirely in portable C. i386 platforms are the primary ones in use for development, but we also have access to Alpha-based systems for testing. As TrustedBSD’s supporting infrastructure allows for third-party extension, it is possible that third party security providers might distribute extension in binary-only form, but all base code will be freely available in source form.

Formal evaluation for certification requires selection of specific hardware, as whole systems are evaluated rather than purely software products. 

Q: Does the TrustedBSD project intend on making its extensions portable to OpenBSD/NetBSD/Apple Darwin? What are the barriers to porting them? 

A: TrustedBSD is made up of a number of components, some of which do require extensive modifications to the kernel structure. However, where such changes are made, they will improve kernel modularity and extensibility, supporting modular insertion of TrustedBSD components. If these modularity changes are introduced in other *BSD kernels, they can be used to support the higher level functionality. As the *BSD kernels are relatively close in structure, this task is well within the realm of possibility. The KAME IPv6/IPsec implementation is an example of a project that has successfully developed software for multiple BSD platforms. The BSD-style license used in TrustedBSD should pose no problem for integration with almost any other project, open source or commercial, and the TrustedBSD project would be glad to see and assist in integration into other operating systems. 

Q: Do the TrustedBSD extensions to *BSD open the doors to any new uses for the OS? e.g., Are there any tasks to which the existing BSD flavors were unsuited to before for which they are now suitable? 

A: Yes, it does, as it opens to doors to uses that have more demanding security requirements. Examples include banking, military, and electronic commerce. That is not to say that FreeBSD wasn’t used there before, as it was already a very qualified OS for these environments, but his provides a higher degree of assurance. An extensible security infrastructure will make FreeBSD/TrustedBSD a more appealing platform for security research and development, also. 

If formally evaluated, then it would be open for use on classified networks in ways in which it currently isn’t usable. 

Q: How does TrustedBSD compare to OpenBSD’s security goals? Are they competitive projects or complementary projects? 

A: Complementary. To our knowledge, OpenBSD has really taken quite a different approach to security: fine-grained source code auditing and integration of cryptography. TrustedBSD introduces new authorization models, as well as auditing. While we won’t be porting the changes over to OpenBSD at this time, this doesn’t mean others can’t, or that we won’t port them in the future. 

Q: How many active contributors are there to the TrustedBSD project? What areas are you looking for contributors to? 

A: TrustedBSD has a relatively small developer community, although we hope it will grow as interest grows. At this point, the number of active developers is around 6 to 10, but there is substantial design discussion on our mailing list from a far larger group bringing experience from a variety of other platforms. There is also substantial interest in cross-platform portability for applications. 

At this point, contributors should have strong kernel and application security experience; in particular, we’re interested in developers who have past experience with trusted operating systems. As the project progresses and the kernel component reaches greater maturity, we’ll be interested in application programmers to help adapt existing applications for any changes in the trusted OS environment. As there is interest in portable APIs, we hope to be able to leverage other work in this area. 

Q: What are TrustedBSD’s goals for the next six months/year/unspecified dates in the future? 

A: As with all operating system scale projects, TrustedBSD will involve an iterative development process, with features becoming available over time. We feel that the current goals can be broken down conceptually into a number of implementation phases: 

• In the first phase, the goal is to introduce TrustedBSD security changes directly and within the context of the existing FreeBSD security implementation. This phase is well underway, and will result in a usable, secure system. However, the goal of this phase is to gain both development and operational experience: while some of this code will be integrated into the base system (some already has – support for extended file attributes, and ACL interfaces), the majority will be made available via seperate distribution mechanisms. 

• In the second phase, the goal is to generate stronger security infrastructure in the kernel, aiming for greater generalization and modularity. In this phase, we build on the practical development experience in the first phase, having a strong understanding of the requirements base on having implemented the same features. One of the main targets of this phase is a generalized kernel authorization framework, allowing modular and pluggable security extensions to be introduced without substantial source modification (as required in the first phase). This will allow both the TrustedBSD project, and third party developers and vendors, to distributed FreeBSD security enhancements with substantially lower development and maintenance costs. 

The first phase is well underway — some interfaces (in particular, for ACLs) were included in 4.0-RELEASE, and supporting infrastructure such as extended file attributes have been committed to the FreeBSD 5.0-CURRENT development branch. Support for fine-grained least privilege (a variant of POSIX.1e capabilities), native ACL support in the FFS file system, fine-grained event auditing, and MLS MAC policy support is in the works. We see a time frame for the completion of the implementation of this component within 12 months; however, one aspect to trusted operating systems is strong supporting documentation, including design, implementation, and operation. Completing this documentation may take additional time. 

The second phase is currently under design–we’ve gained substantial experience in that which we have completed of the first phase, and are ready to begin discussing the stronger and more general abstractions in the second phase. By the time this interview is published, drafts for at least the modular authorization system will be under discussion on the TrustedBSD mailing lists. So far, the results look promising. We hope to have a first prototype of the generalized authorization system completed within six months, and to being migrating the security models implemented in the first phase over to this mechanism shortly thereafter. 

We hope to present a number of papers detailing aspects of the TrustedBSD project at the BSD Con 2000 conference in October.

BSDI, Walnut Creek CD-ROM Merge – Move Brings Two Biggest BSD Unix Variants, FreeBSD and BSD/OS, Under One Tent

By Jeffrey Carl

Boardwatch Magazine
Boardwatch Magazine, June 2000

Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among ISPs, particularly for its annual ranking of Tier 1 ISPs and through the ISPcon tradeshow. Writing and speaking for Boardwatch was one of my fondest memories of the first dot-com age.

Confirming a deal that had been whispered about among the BSD Unix community for months, Berkeley Software Design, Inc. (BSDI) and Walnut Creek CD-ROM announced on March 9 that they would merge to form a new company under the BSDI name. The significance of the announcement lies in that it brings the vendor of commercial BSD-based systems (BSDI’s BSD/OS operating system) together with the primary distributor of the largest free BSD variant (FreeBSD), signaling a major push toward unity among the BSD Unix flavors. 

The move also opens the door to more intermingling of code between the two OSes, since some of FreeBSD’s key developers were employed by Walnut Creek and will now be employed by BSDI, and members of BSDI’s BSD/OS engineering team may be contributing code to FreeBSD. In a separate move, Walnut Creek’s Slackware Linux distribution will be spun off into a separate company, run by longtime Slackware chief Patrick Volkerding.

In the short term, users of FreeBSD and BSD/OS won’t see much change to the current products (FreeBSD 4.0 and BSD/OS 4.1). BSDI will continue to offer their commercial BSD/OS and provide paid support and consulting, while FreeBSD will remain free. However, BSDI will soon offer commercial support options for FreeBSD – potentially paving the way for FreeBSD’s entry into the enterprise market in the same way that commercial support for Linux from RedHat, LinuxCare and others did for Linux. Just as important to the enterprise market, BSDI CEO Gary Johnson said that there are plans to implement a “BSD engineer” certification program by the end of Q2 2000.

In the longer term, the move offers a wealth of potential possibilities by placing the resources of a well-funded corporate entity behind promotion of BSD Unix and encouraging commonality between the OSes. BSDI has said that it will work to develop a common Application Binary Interface (ABI) between FreeBSD and BSD/OS, enabling an application written for one OS to run on the other without any modification. While specifics are still very much up in the air, it is likely that some elements from each OS will find their way into the other, over the course of the next few OS releases. 

Unconfirmed possibilities include extending FreeBSD’s hardware support with the addition of some BSD/OS code, and making user-friendly elements like the FreeBSD “ports collection” software installation scheme available as an optional add-on to BSD/OS. Still, BSDI is adamant that the merger won’t make users of FreeBSD or BSD/OS sacrifice any of the qualities they treasure in their OS of choice.

Most exciting for many *BSD users is the likelihood that many of the “old school” computer scientists who were part of the UC Berkeley Computer Science Research Group (CSRG) that developed BSD Unix and are currently affiliated with BSDI will be brought into the BSD community. While BSDI hasn’t firmed up its message about which OS to push to what constituency, it is likely that FreeBSD will continue to appeal to the open-source community while BSD/OS will be aimed at commercial and enterprise users. 

Why was the merger done? To paraphrase Benjamin Franklin, the BSDs must all hang together, or surely they will hang separately. BSDI says that, between the two operating systems, there are at least two million BSD servers running out there. BSDI cites their collective market share as 15 percent of all Internet sites, as well as being used by nine out of ten ISPs or NSPs. Between the two OSes, customers include Microsoft Hotmail (which reportedly tried to convert to Windows NT but reverted to FreeBSD after numerous problems), UUnet and Yahoo! (which also made an equity investment into the new BSDI). 

Still, the public presence of BSD Unix outside these groups is nearly nil, while Linux has garnered massive mindshare. “BSD is pervasive throughout the Internet … but the world at large doesn’t know it,” said BSDI Marketing Manager Kevin Rose. While this can be attributed to large number of factors, it lies largely in the fact that commercial BSD/OS has never had mass mindshare, while the free BSD variants with more users have never had the money to undertake real promotional campaigns. 

While BSDI won’t discuss what the relative portions of those two million servers are FreeBSD or BSD/OS, it is relatively certain that the majority are running FreeBSD – which has never had the funding to promote its offerings on a level equivalent to what Linux distribution makers have done. The merger will likely put significant cash resources behind the promotion of BSD Unix for the first time. BSDI CEO Johnson said, “everything we’re doing is for the betterment of BSD …  the idea is to promote not BSD/OS, not FreeBSD, not NetBSD, not OpenBSD, but BSD.”

FreeBSD Chief Evangelist Jordan Hubbard said that the merger also leaves the door open to cooperation with the other free BSD variants, NetBSD and OpenBSD. BSDI will be working to bring more third-party applications to BSD, as well as promoting publication of BSD books and developing user groups. 

The fact that many of the technical details of the merger are still undecided leaves the door open to speculation that philosophical differences (or developer egos) may complicate the implementation of the full possibilities of the merger. Still, the merger announcement is a true turning point in the history of BSD Unix, and a much-needed encouraging sign for the platform.