{"id":189,"date":"2001-05-25T12:37:10","date_gmt":"2001-05-25T19:37:10","guid":{"rendered":"http:\/\/www.jeffcarl.com\/?p=189"},"modified":"2020-07-08T16:28:09","modified_gmt":"2020-07-08T23:28:09","slug":"darwin-evolves-apples-bsd-hits-the-prime-time","status":"publish","type":"post","link":"https:\/\/www.jeffcarl.com\/index.php\/2001\/05\/25\/darwin-evolves-apples-bsd-hits-the-prime-time\/","title":{"rendered":"Darwin Evolves – Apple’s BSD Hits the Prime Time"},"content":{"rendered":"\n

By Jeffrey Carl<\/strong><\/p>\n\n\n\n

\"Boardwatch
Boardwatch Magazine, May 2001<\/figcaption><\/figure><\/div>\n\n\n\n

Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among ISPs, particularly for its annual ranking of Tier 1 ISPs and through the ISPcon tradeshow. Writing and speaking for Boardwatch was one of my fondest memories of the first dot-com age.<\/em><\/p>\n\n\n\n

In a Nutshell:<\/em><\/strong> DarwinOS, the core of Apple\u2019s just-released MacOS X, is open-source and available as a free download. While it inherits many characteristics from current BSD Unixes, it\u2019s also radically different \u2013 revealing its true identity as the direct descendant of the ahead-of-its-time NeXTSTEP operating system. Darwin has some rough edges and missing documentation, but offers some very interesting possibilities. Unfortunately, the fact that it\u2019s intended for Apple hardware will limit its appeal to many ISPs and sysadmin<\/em>s.<\/p>\n\n\n\n

By the time you read this, Apple\u2019s MacOS X should be on the street \u2013 the first-ever consumer OS with a Mach microkernel\/BSD Unix core. That core is called Darwin, and it\u2019s an open-source project \u2013 available under the BSD-ish Apple Public Source License \u2013 that can be downloaded for free (MacOS X, which includes the new Mac GUI and lots of other goodies, costs $129).<\/p>\n\n\n\n

I\u2019ve talked about Darwin here before, when it was in its earlier stages, but wasn\u2019t able to go into many specifics about what Darwin was really like for a Unix admin. Now that the version that ships with MacOS X 1.0 is here, let\u2019s take a look at this remarkable OS change for Apple.<\/p>\n\n\n\n

The Politics Behind Darwin<\/h3>\n\n\n\n

Darwin is the triumph of NeXT over Apple. The GUI-only, outmoded classic MacOS of Apple\u2019s last 17 years is gone, and a tough, sexy Unix core has replaced it (although MacOS X includes a compatibility layer to run existing MacOS apps). To understand why \u2013 and to understand why a Unix admin might be very interested in an Apple OS (other than the \u201cresearch project\u201d MkLinux) for the first time in many years \u2013 it helps to know the politics behind the situation.<\/p>\n\n\n\n

In case you haven\u2019t been paying attention to Apple\u2019s corporate soap opera for the last few years (or, more likely, just don\u2019t care), here\u2019s the short version. Apple was at its lowest depths in late 1997, having wasted more than six fruitless years trying to create a modern operating system to replace MacOS (which was still building on foundations laid in 1984). Then-Apple CEO Gil Amelio instead looked to buy a company that already had<\/em> an advanced OS, and settled on NeXT, the failing company run by Apple\u2019s exiled cofounder Steve Jobs. The brilliant, mercurial, occasionally tantrum-prone Jobs was instrumental in building Apple from Steve Wozniak\u2019s garage into an industry giant. But in 1985, he was fired by his own board of directors, essentially for being a major jerk to everyone within a 50-yard radius. It was a very<\/em> nasty \u201cdivorce,\u201d and Jobs left in a huff to found NeXT.<\/p>\n\n\n\n

The NeXTSTEP (later called OPENSTEP) operating system which powered NeXT\u2019s computers was years ahead of its time \u2013 but shipped on underpowered, grossly overpriced computers (sound familiar?) and was rejected by the marketplace. NeXTSTEP was based on Unix and the Mach microkernel, and included a radical new GUI and a pretty cool object-oriented development framework. <\/p>\n\n\n\n

Microkernels (of which Mach is the most famous example) provided a much more elegant OS design than most OSes used (and still use), by moving everything but memory management and other lowest-level tasks out of the kernel. However, the overhead of this elegant and scalable design provided reduced performance in many \u201ceveryday\u201d situations (like replacing a simple Excel spreadsheet with a full relational database), and microkernels were sidelined as academic curiosities. The NeXT object-oriented development kit was very advanced, but required knowledge of the relatively obscure Objective-C language and was largely ignored as well.<\/p>\n\n\n\n

For Apple\u2019s $400 million purchase of NeXT in 1997, they got not only the company\u2019s OS but CEO Jobs as well. Then-Apple CEO Gil Amelio thought he was getting a valuable figurehead\/consultant in Steve Jobs. But an ex-Apple employee who knew Jobs better than Amelio did predicted to Apple CTO Ellen Hancock that \u201cSteve is going to f*** Gil so hard his eardrums will pop.\u201d Sure enough, on July 6 1998, Gil Amelio was forced to resign and Steve Jobs once again was in charge of Apple. (Hancock resigned when Amelio did, and went on to become president of web hosting and datacenter giant Exodus.)<\/p>\n\n\n\n

The rest is history. In the foreground, Jobs was introducing the fruit-colored, legacy-free  iMac to the world, sparking Apple\u2019s sales resurgence. In the background, nearly all of Jobs\u2019 loyal NeXT troops were assuming the top posts at Apple and changing the company\u2019s technology and direction.<\/p>\n\n\n\n

In 1999, the hype about Linux and open source was at its height, and Apple felt the pressure to join the crowd. Since BSD and Mach \u2013 which formed the core of NeXTSTEP \u2013 were already open source, it wasn\u2019t hard for the normally ultra-proprietary Apple to take the step of officially open-sourcing the core of the forthcoming MacOS X. The NeXTSTEP core of MacOS X officially became \u201cDarwin,\u201d and a developer community of Apple engineers, Macolytes and Unix hackers began to form around the project. Along the way it saw contributions from Quake designer John Carmack, a basic port to Intel x86 hardware, and a \u201c1.0\u201d version that has evolved significantly as MacOS X neared release.<\/p>\n\n\n\n

It\u2019s noteworthy to keep in mind how much of a departure Darwin is from the old MacOS. It\u2019s as if the \u201cHouse that GUI Built\u201d was taken over by the Unix geeks from Carnegie-Mellon and handed over to the hacker community for safekeeping. NeXT was the \u201cugly step-child\u201d of the BSD family that nobody else noticed; and now, it will claim more users than the rest of the family combined \u2013 probably in less than six months.<\/em><\/p>\n\n\n\n

Getting In-Depth with Darwin<\/h3>\n\n\n\n

After all that, it\u2019s time to get under the hood of Darwin as a Unix admin would approach it. Many reports have claimed erroneously (at times, I have said this as well) that Darwin was basically \u201cFreeBSD on Mac.\u201d In fact, it\u2019s a bit of updated Mach, a bit of traditional Free\/Open\/NetBSD, and a lot of NeXTSTEP flavoring. <\/p>\n\n\n\n

Darwin uses the Mach microkernel, but attempts to solve some of its performance problems by moving much of the BSD functionality into the same \u201ckernel space\u201d (rather than in \u201cuserland\u201d as a pure microkernel would). As such, it merges the two worlds in a way that is designed to keep the architectural elegance of a microkernel design while minimizing the performance overhead that microkernel process scheduling causes.<\/p>\n\n\n\n

The first thing a BSD Unix admin will notice upon logging into Darwin is its familiarity. The default shell is tcsh<\/strong>, and nearly all of the customary \/bin, \/usr\/bin\/ and \/usr\/sbin utilities are there. In addition, you\u2019ll find vi<\/strong>, pico<\/strong>, perl<\/strong>, sendmail<\/strong>, Apache<\/strong>, and other typical goodies. Ports to Darwin of popular Unix open-source Unix apps \u2013 from mysql to XFree 4.0.2 are proliferating rapidly. From a typical user\u2019s perspective, it\u2019s almost indistinguishable from *BSD.<\/p>\n\n\n\n

It\u2019s only once you become root and muck around in the system\u2019s administration internals that you start to notice what makes the system a true child of NeXTSTEP. You\u2019ll notice in \/etc (actually a link to \/private\/etc; see below for more information) that \/etc\/resolv.conf doesn\u2019t contain what you would expect. Nor does \/etc\/hosts, \/etc\/group or \/etc\/passwd. Why? It\u2019s because Darwin wraps many of the functions contained in separate system and network configuration files in *BSD into a single service (inherited from NeXT) called NetInfo<\/strong>. Why is this useful rather than just annoying?<\/p>\n\n\n\n

In *BSD and Linux, a number of different services derive their information from separate text configuration files \u2013 each with its own syntax and options. There\u2019s no global preferences database \u2013 for example, any application that doesn\u2019t automatically know how to read all the items in \/etc\/resolv.conf can\u2019t find out what name servers your computer is using, or a program that can\u2019t parse the syntax of \/etc\/passwd doesn\u2019t know which users are on your system. <\/p>\n\n\n\n

Somewhat like the Microsoft Windows Registry, Darwin\u2019s NetInfo provides a database of network and user settings that can be read by any other NetInfo-aware application. NetInfo supersedes the information in the traditional \/etc\/* configuration files, as well as being favored by system services. NetInfo is consulted not only by MacOS X-native applications, but also by traditional BSD\/Unix applications as well (making it much easier to port these apps to Darwin). The Apple engineers have accomplished this by hooking a check into each libc<\/strong> system data lookup function to consult NetInfo if it\u2019s running (by default, it\u2019s only \u201coff\u201d in single-user mode). <\/p>\n\n\n\n

MacOS X\u2019s GUI provides graphical tools for manipulating the NetInfo database; in Darwin, this can be done using the niutil<\/strong> and nicl<\/strong> commands (use man niutil and man nicl to see the syntax and options; it\u2019s interesting to note that these man pages are dated from NeXT days). NetInfo can also \u201cinherit\u201d its settings from a \u201cparent\u201d NetInfo server, so you can create one server which has everyone\u2019s account information on it, and all of its client machines will have their login info, network setup, et cetera (imagine a \u201cfamily\u201d of servers where users can interchangeably log in with the same accounts, settings, etc.).<\/p>\n\n\n\n

Like NetInfo settings, application preferences are stored in a global XML database; they can be manipulated from the command-line defaults<\/strong> program. Typing man defaults from the command line will give you an idea of how its structure works.<\/p>\n\n\n\n

One area that has been changed since the MacOS X Public Beta is that it is no longer necessary to reboot or log out\/log back in to change network settings. Anyone who has used *BSD\/Linux on mobile computers, or changed network profiles in Windows NT will appreciate this difference.<\/p>\n\n\n\n

Darwin\/MacOS X includes a \/private directory at the root filesystem level, which includes the normal BSD \/etc, \/var, \/tmp and the Darwin \/cores and \/Drivers). It appears that (in behavior inherited from NeXT) this directory is filled with info that is machine-specific, so that the rest of the filesystem can be booted from a parent network server. This clearly plants the roots for Darwin or MacOS X-based systems to serve as \u201cthin clients.\u201d<\/p>\n\n\n\n

As for performance, recent builds of Darwin work admirably on mid-range Mac hardware. The only real complaint I have about Darwin (my gripes with the MacOS X user interface could fill a small book, however) is its woeful lack of documentation. While many BSDs suffer a similar problem, their user communities have had time to fill in the gaps; the changes that Darwin makes to the traditional BSD model are largely known only to the Darwin community and old-school NeXT gurus.<\/p>\n\n\n\n

The closer I look at it, the more Darwin (and MacOS X) is appearing to take up where Novell left off in the race to compete with Windows NT\/2000 in the corporate network space. It might be possible that Apple is looking to go head-to-head with Windows Whistler\/XP while nobody is looking. And any victory in that space for *nix is something to be cheered.<\/p>\n\n\n\n

Darwin\u2019s Dilemma<\/h3>\n\n\n\n

Unfortunately, most of the work that has gone into Darwin thus far (understandably) has been in developing it for recent Apple hardware. While an Intel x86 port exists (but is still as of this writing in embryonic stage for hardware support), and older PowerPC Mac hardware support will undoubtedly extend over time, the current release of Darwin is not officially supported for Mac hardware older than G3 Power Macs. This sadly eliminates (at least for now) using Darwin to make a great server out of any old Mac hardware you have sitting on a shelf.<\/p>\n\n\n\n

When you buy a new Mac, you\u2019re paying for things (like full MacOS X, an ATI Radeon or nVidia GeForce video card, optical mouse, etc.) that you aren\u2019t going to care about as a server admin. While new Mac systems are surprisingly powerful considering their CPU clock speed (I would sooner compare a G4 to an UltraSPARC III than a Pentium III), you still<\/em> won\u2019t get the same performance dollar-for-dollar as you would with commodity x86 hardware and a free OS. <\/p>\n\n\n\n

As a result, buying a new Mac just to make into a Darwin server simply isn\u2019t worth the money. If you love the GUI tools of MacOS X, it may be worthwhile (I\u2019m personally salivating over the new Titanium PowerBook G4); otherwise, it still doesn\u2019t make bottom-line dollars and sense to purchase a new Mac as a Darwin server.<\/p>\n\n\n\n

As Darwin expands its processor base, this may change. In the meantime, it\u2019s well worth your while to keep an eye on the Darwin project, and to get to know it, since some of its features are well worth adopting by the other BSDs and Linux.<\/p>\n\n\n\n

Some of proceeding items stem from a series of articles that BSD guru Matt Loschert and I wrote for the BSD news site Daemon News<\/em> (see www.daemonnews.org\/200011<\/a>\/<\/em> and www.daemonnews.org\/200012\/<\/em> for excessively long and drawn-out versions). \ud83d\ude09 For great info on Darwin, you can skip its home page (www.opensource.apple.com<\/a><\/em>) and go straight to Darwin Info (www.darwinfo.org<\/a><\/em>). Also, check Xappeal (www.xappeal.org) and<\/a><\/em> Apple\u2019s Darwin lead Wilfredo Sanchez\u2019s updates page (www.advogato.org\/person\/wsanchez<\/a><\/em>). As always, let me know about any comments, corrections or suggestions you have, and I\u2019ll publish them in a future column.<\/p>\n","protected":false},"excerpt":{"rendered":"

By Jeffrey Carl Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among … Continue reading Darwin Evolves – Apple’s BSD Hits the Prime Time<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":22,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,66],"tags":[34,29,33,32],"class_list":["post-189","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-boardwatch-writing","category-tech","tag-apple","tag-bsd-unix","tag-darwin","tag-macos-x"],"jetpack_featured_media_url":"https:\/\/www.jeffcarl.com\/wp-content\/uploads\/2020\/04\/bwatch.gif","_links":{"self":[{"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/posts\/189","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/comments?post=189"}],"version-history":[{"count":2,"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/posts\/189\/revisions"}],"predecessor-version":[{"id":256,"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/posts\/189\/revisions\/256"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/media\/22"}],"wp:attachment":[{"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/media?parent=189"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/categories?post=189"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.jeffcarl.com\/index.php\/wp-json\/wp\/v2\/tags?post=189"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}