Webhosting with Free Software Cheat Sheet

By Jeffrey Carl

Boardwatch Magazine
Boardwatch Magazine, April 2001

Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among ISPs, particularly for its annual ranking of Tier 1 ISPs and through the ISPcon tradeshow. Writing and speaking for Boardwatch was one of my fondest memories of the first dot-com age.

So you want to run a webserver without paying a dime for software, eh? Or you want to make sure you have the source code to all your webserving applications in case you get bored and decide to try and port them to your networked office Xerox copier? Well, you’re in luck; webhosting with free (open-source, or free as in “free speech”) and free (no cost, or free as in “free beer”) software isn’t just possible, it also provides some of the best tools out there at any price.

In case you’re new to the game, or you’re looking for alternatives to packages you’re using now, the following is a brief guide to some of the more popular options that are out there. Trying to condense the features of any OS or application down to a couple sentences is inherently a dangerous thing; and I’m sure that many fans of the software listed below will disagree with elements of my “Reader’s Digest Condensed (Large Print Version)” summaries. Still, the following – based on my experiences and those of others – should provide at least a basic idea of what’s out there and why you might – or might not – want to choose it.

Operating Systems

Linux vs. BSD:

These OSes show the characteristics of their development styles: BSD was developed by small teams, largely focused on server hardware. Linux has been developed by many more people with wider uses, focusing more on desktop/workstation uses. 

BSD has been around longer and is (in some ways) more optimized for server use. Due to its hype, Linux has many more developers, and almost all new third-party software is available for Linux first. Linux has the edge in user-friendliness, because distributions are targeting new users; BSD is, for the most part, more for the “old school.” Linux has also been adopted by a number of server hardware vendors producing “integrated” solutions.

Ultimately, it’s a matter of what you feel most comfortable with. Either way, with commodity x86 hardware, your server components (RAM, drives, etc.) and network connection will affect your performance much more than your choice of Linux vs. BSD will.

• FreeBSD (www.freebsd.org)

Best known among the BSDs. Concentrates on x86 architecture, server performance, integration of utilities. Standout features include ports collection, sysinstall admin utility, Linux binary compatibility, frequent releases.

• NetBSD (www.netbsd.org)

BSD with a focus on porting it to as many platforms as possible and keeping code portable. Great for using old/odd hardware as a server. Infrequent releases, not as popular as other BSDs.

• OpenBSD (www.openbsd.org)

BSD with a focus on security. Still in the process of line-by-line security audit of the whole OS. Infrequently released, utilities/packages lag behind other OSes because of security audits, but it’s the #1 choice if security is your primary concern.

• Red Hat Linux (www.redhat.com) 

The number one U.S. distro, considered by many (rightly or wrongly) as “the standard.” As a result, it’s what many third-party/commercial Linux apps are tested against/designed for. Early adopter of new features in its releases; is on the cutting edge, but sometimes buggy until “release X.1.” Standout features: Red Hat Package Manager (RPM) installation, third-party support.

• SuSE Linux (www.suse.com)

The number one European distro. Favored by many because its six-CD set includes lots and lots of third-party software to install on CD. Less “cutting-edge” than Red Hat. Standout features include the YaST/YaST2 setup utility and the SaX X Windows setup tool.

• Slackware Linux (www.slackware.com)

Designed for experts: Slackware has no training wheels, and is probably the most “server-oriented” of Linux distros (maybe because of its close relationship to the BSDs). Not cutting-edge, few frills, but designed to be stable and familiar to BSD administrators.

• Linux Mandrake (www.linux-mandrake.com/en)

A solid, user-friendly distribution with good (but not great) documentation. Standout features include the DrakX system configuration utility and the DiskDrake disk partitioning utility.

• Debian GNU/Linux (www.debian.org)

The ideological Linux – totally supported by users rather than a corporation, and free (as is the GNU definition) software only is included. This is “ideologically pure” Linux – GNU-approved, but infrequent releases and not necessarily a good choice for beginners.

• Caldera OpenLinux (www.caldera.com/eserver)

Very user-friendly for new users. Standout features include LIZARD, its setup/configuration wizard.

• Corel LinuxOS (linux.corel.com)

By the time you read this, Corel will have sold its LinuxOS product to someone else, but the distro should remain the same. Ease of use for Windows converts is stressed, includes great SAMBA integration. Good for new users. Focus is mainly on desktop use.

Essentials

• Perl (www.perl.com)

From CGI to simple administration tasks, Perl scripts can cover a lot of territory. Perl is a must, and is practically part of Unix now. Check the Comprehensive Perl Archive Network (www.cpan.org) to find modules to extend Perl’s functionality.

• Perl’s cgi-lib.pl (cgi-lib.berkeley.edu) and/or CGI.pm (stein.cshl.org/WWW/software/CGI)

These are also “must-haves” for CGI scripts, whether you’re writing your own or using scripts found “out there” on the web.

• Sendmail (www.sendmail.org) or Qmail (www.qmail.org)

Free mailservers. Sendmail has the history and the documentation (a good thing, since its internals are famously complex), but Qmail has a less-complicated design, and a strong and growing band of followers.

• wu-ftpd (www.wu-ftpd.org)

A significant improvement in features over the classic BSD FTP daemon – for both BSD and Linux. Despite an older security flaw that was recently exploited by the “Ramen” Linux worm, it’s a very good program.

• OpenSSH (www.openssh.com/)

In this day and age, Telnet has become a liability for security reasons. There’s no reason not to migrate users who need a shell account to SSH. See www.freessh.org for a list of clients.

Web Servers

• Apache 1.3.x (www.apache.org/httpd.html)

The current king of web servers. Very good performance, stable enough to run on mission-critical systems. Very user-friendly to install and configure (due to comments in httpd.conf), but not always as easy as it should be to debug problems.

• Apache 2.x (www.apache.org/httpd.html)

Still in beta development, but may be final by the time you read this. It probably shouldn’t be used for mission-critical systems until it’s had a few months of time “out there” to find bugs after its final release. Version 2.0 will be much easier to add new protocols into (like FTP or WAP), and should have significantly better performance because of its multi-threaded nature.

• Roxen (www.roxen.com/products/webserver)

Roxen is much more than just a simple webserver – it includes its own web admin interface, secure server, and more. Used by real.com; shows promise but doesn’t have the acceptance level yet of Apache.

Secure Web Servers

Note: You may receive a “security warning” in most web browsers about your secure certificate if you generate your own secure certificate (free). For a non-free certificate created by an authority that most web browsers will accept without a warning, see VeriSign (www.verisign.com/products/site/ss/index.html), Thawte (www.thawte.com), Baltimore (www.baltimore.com/cybertrust), ValiCert (www.valicert.com/), Digital Signature Trust Co. (www.digsigtrust.com) or Equifax (www.equifaxsecure.com/ebusinessid) for more information.

• ApacheSSL (www.apache-ssl.org)

Focused on stability/reliability, and lacking in “bells and whistles” features. It’s simple and it works, but it lacks some features of mod_ssl and it isn’t updated very often.

• mod_ssl (www.modssl.org)

Originally based on ApacheSSL, mod_ssl is now largely rewritten and offers a number of extra features, plus better documentation. 

Microsoft Web Compatibility

• FrontPage Extensions for Unix (www.rtr.com/fpsupport)

On one hand, it allows you to host sites built and published with FrontPage on a Unix server. On the other hand, it’s possibly the biggest piece of junk Unix software ever created. Use it if you have to; avoid it if you can.

• Improved mod_frontpage (home.edo.uni-dortmund.de/~chripo)

Addresses a number of problems with mod_frontpage (www.darkorb.net/pub/frontpage), with extra security and documentation, support for Dynamic Shared Objects (DSOs), better logging, as well as (unverified) claims of increased performance.

• Apache::ASP (www.nodeworks.com/asp)

An alternative to the very expensive ChiliSoft or Halcyon ASP Unix solutions, using Perl as the scripting language for ASPs. Requires the Apache mod_perl.

• asp2php (asp2php.naken.cc)

As its FAQ says, “ASP2PHP was written to help you correct the mistake of using ASP.” Converts ASP scripts to PHP scripts for use with Apache/PHP.

Application Building

• Zope (www.zope.org)

A complete tool for building dynamic websites; there’s a (somewhat) stiff learning curve that may be too much for basic needs. Zope offers incredible functionality, and is well-suited to large projects and web applications; it may be overkill for simple scripting that could be done with PHP or Perl CGIs.

• PHP (www.php.net)

The favorite open-source tool for building dynamic websites, and the open-source alternative to ASP. Reliable, uses syntax that seems like a cross between Perl and C, and features native integration with Apache. Version 4 is thread-safe, modular, and reads then compiles code rather than executing it as it reads (making it much faster with large, complex scripts).

Database Software

Note: for a more in-depth comparison, I highly recommend the O’Reilly book MySQL and mSQL, as well as the article “MySQL and PostgreSQL Compared” (www.phpbuilder.com/columns/tim20000705.php3).

• MySQL (www.mysql.com)

The “Red Hat” of free relational database software. Well-documented, and its performance for most users is excellent, designed around fast “read” rather than “write” operations. It doesn’t offer “subselect” functionality, and tends to buckle under very heavy loads (more than 15 concurrent users per second), but is very fast and reliable for most sites.

• PostgreSQL (www.postgresql.org)

Has an active developer community, especially popular among the “GPL-only” crowd. Offers advanced features that MySQL doesn’t (subselects, transactional features, etc.), but traditionally wasn’t as fast for common uses and sometimes suffered data corruption. New versions appear to have remedied most of these deficiencies.

• mSQL (www.hughes.com.au) 

The first of the bunch, but appears to have fallen behind. More mature than MySQL or PostgreSQL, but may not have all of the features of its rapidly developing brethren.

Administration

• Webmin (www.webmin.com/webmin)

Fully featured web-based administration tool for web, mail, etc. Offers excellent functionality, but can present a potential security risk (I get really nervous about anything web-accessible which runs with root permissions).

Java Servlets

• Tomcat (jakarta.apache.org) and JServ/mod_jserv (java.apache.org)

Tomcat is an implementation of the Java Servlet 2.2 and JavaServer 1.1 specifications that works with other browsers as well as Apache. JServ is an Apache module for the execution of servlets. The two work together to serve JSPs independently of Apache.

Website Search

• ht://Dig (www.htdig.org)

ht://dig is relatively simple to set up, and (with a few quirks) offers excellent searching capabilities. Easily customizable, and has a good “ratings-based” results engine.

• MiniSearch (www.dansteinman.com/minisearch)

A simple Perl search engine, which can also be run from the command line. Not as fully featured as ht://dig, but good enough for basic needs.

Web Statistics

• analog (www.statslab.cam.ac.uk/~sret1/analog)

Analog is extremely fast, reliable and absolutely filled with features. Its documentation is a bit confusing for beginners, however, and it takes some configuration to make it pretty.

• wwwstat (www.ics.uci.edu/pub/websoft/wwwstat)

A no-frills, simple statistics analysis program that delivers the basics.

Other Goodies

Configuration Scripts:

• Install-Webserver (members.xoom.com/xeer)

• Install-Qmail (members.xoom.com/xeer)

• Install-Sendmail (members.xoom.com/xeer)

Shopping Carts:

• Aktivate (www.allen-keul.com/aktivate)

Aktivate is an “end-to-end e-commerce solution” for Linux and other Unixes. It is targeted at small-to-medium-sized businesses or charities that want to accept credit card payments over the Web and conduct e-commerce. 

• OpenCart (www.opencart.com)

OpenCart is an open source Perl-based online shopping cart system. It was originally built to handle the consumer demands of Walnut Creek CDROM, was later expanded to also work with The FreeBSD Mall, and was finally developed to be used by the general public.

• Commerce.cgi (www.careyinternet.com)

Commerce.cgi is a free shopping cart program. Included is a Store Manager application to update program settings, and you can add/remove products from the inventory through a web interface.

Message Boards:

• WaddleSoft Message Board (www.ewaddle.com)

WaddleSoft is a message board system that includes polls, user registration, an extensive search engine, and sessions to track visitors.

• MyBoard (myboard.newmail.ru)

MyBoard is very easy and light-weight web messageboard system. It also has some extended features such as search and user registration.

• NeoBoard (www.neoboard.net)

NeoBoard is a Web-based threaded message board written in PHP. It includes a wide variety of advanced features for those comfortable with PHP.  

• PerlBoard (caspian.twu.net/code/perlboard)

PerlBoard is a threaded messageboard system written in Perl. It is very easy to use and set up, and has been time-tested for the past several years on the site it was originally written for.

• RPGboard (www.resonatorsoft.com/software/rpgboard)

RPGBoard is a WWWBoard-style message board script. It includes a list of features as long as your arm, and is well worth checking out for those who need a rather advanced message board.

Notes from the Underground

If you see a favorite package here that I’ve overlooked, or would like to offer comments on any of the package descriptions, e-mail me at [email protected]. I’ll update this list with more information for a future column.

Hosting FrontPage Sites on Unix Servers – Dealing with the Great Satan of Server Extensions

By Jeffrey Carl

Boardwatch Magazine
Boardwatch Magazine, November 2000

Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among ISPs, particularly for its annual ranking of Tier 1 ISPs and through the ISPcon tradeshow. Writing and speaking for Boardwatch was one of my fondest memories of the first dot-com age.

Ever since a version of FrontPage started getting bundled in with Microsoft Office 2000, FrontPage has quickly become the web site design tool of choice for people who enjoy being lectured by animated paper clips. If your hosting environment is Linux or *BSD, then FrontPage needs to be understood and dealt with.

FrontPage and How it Works

FrontPage is a simple GUI tool which automates the things that most basic web designers are looking for these days, including graphical page design, publishing their sites and CGI-type actions (forms, dynamic pages, etc.). Throughout the rest of this column, I don’t want to disparage what the creators of FrontPage have made – a truly easy, user-friendly program for the design of simple to very complex web sites – a real achievement.

But, for a FrontPage-created site to take use of all the easy-to-create “all-singing, all-dancing” features that make it such a hit with people who wouldn’t know an HTML tag if it crawled up and bit them, the FrontPage Server Extensions (FPSE) must be present on the hosting server. The FPSE are essentially Microsoft-built proprietary CGIs executed over HTTP – designed for Windows NT and Microsoft IIS, but ported to Unix and supported on Microsoft’s behalf by Unix porting house Ready-to-Run Software (RTR). This is where the trouble starts.

Can You Support FrontPage on a Freenix?

Supporting FrontPage on Unix is both a help and a hindrance to ISPs. It allows you to host sites for clients using FrontPage – and this faction is rapidly growing. It can provide you with a competitive advantage over ISPs that don’t support FrontPage sites.

The extensions are currently available for a number of platforms including FreeBSD and Linux (for x86 architectures only), and for a number of webservers including Apache, NCSA, Netscape and Stronghold (although only Apache will be discussed in this column). I’m sure that RTR has done a herculean task porting these server extensions to Unix, but the fact remains that this software is poorly documented and frequently difficult to troubleshoot.

The FrontPage extensions for Unix don’t allow you to support the full range of capabilities of FrontPage-designed sites. Active Server Pages (ASP) aren’t natively handled with the FPSE for Unix, and neither is native ActiveX nor ODBC/Microsoft Access database connectivity. 

You can, fortunately, look for outside support. Software vendor Chili!Soft (www.chilisoft.com/allaboutasp/) offers a Unix-native commercial ASP package, Halcyon (www.halcyonsoft.com/products/iasp.asp) offers a Java-based ASP commercial package, and mod_perl (perl.apache.org) with the Apache::ASP (www.nodeworks.com/asp/) module provides a free, Perl-based solution. Commercial ODBC resources are available at www.openlinksw.com, and open-source resources can be found at www.jepstone.net/FreeODBC.

The Pitfalls of FrontPage Extensions

Unfortunately, offering FPSE/Unix commits you to supporting what is, frankly, some occasionally very frustrating software. While security problems with FP-administered sites have decreased, it’s still not suitable for the truly paranoid (a full discussion would take up this whole magazine). The author.exe program that is part of the FPSE seems, on Unix, to eat up an unseemly amount of CPU cycles while it’s running. The .htaccess files that FrontPage installs for each of the directories in a FrontPage-enabled site may interfere with the server administrator’s security policies. And, in my experience, the FPSE will occasionally just stop working, with a reinstall required.

If you can’t find the free support resources you need, you can of course contact RTR for phone support ($245 per incident) or e-mail support ($195 per incident). And, if you don’t support it, FrontPage users are advised in the FrontPage/Unix FAQ to “find another ISP” (http://www.rtr.com/fpsupport/faq2000g.htm#7newISP). 

So much for “alternatives.” And the customer pressure to host FrontPage sites is quickly increasing. 

Before I go on to discuss how to support FrontPage on Unix, I should mention that there is another option – set up a server running Windows NT or Windows 2000. This has its own set of problems and complications (which could take up a book, not a column), but at least the FPSE work flawlessly. After two years and a multitude of problems supporting 50 or so FrontPage users on Red Hat Linux and FreeBSD, and seeking more scalability without headaches, I shamefully admit that this is what I ultimately did. Unfortunately, I think this is what Microsoft was hoping for all along.

Getting the FrontPage Extensions

For a detailed understanding of the installation process and some help with common problems, first visit the Server Extensions Resource Kit (“SERK”) page for Unix installation (www.rtr.com/fpsupport/serk4.0/inunix.htm). For a full list of the files included in an installation and their permissions structures, see www.rtr.com/fpsupport/serk4.0/apndx01.htm.

If you’re installing Apache on the server for the first time, you can use the apache-fp package (www.westbend.net/~hetzels/apache-fp/), available as a Linux RPM or through the FreeBSD ports collection, which integrates an Apache distribution with the Unix FPSE. If you’re installing on an existing Apache setup, go directly to RTR Software’s FPSE/Unix homepage (http://www.rtr.com/fpsupport/). Download the FrontPage 2000 SR 1.2 extensions for your platform (they’re backward-compatible with FP ’97 and ’98 clients), which come in a zipped or GNU-zipped tar file, as well as the shell scripts fp_install.sh and change_server.sh. 

The extensions tarball will be named like fp40.[osname].tar (version 4 is the FP 2000 extensions; version 3 is for FP ’98). The fp_install.sh script converts the results of the uname -a command to a $machine variable, and won’t install if the extensions tarball name doesn’t match the $machine name.

For a long time, it was necessary to get the extensions to install on FreeBSD by “hacking” the script. In this case, adding the line:

FreeBSD*)            machine="bsdi" ;;

as line 70 of the fp_install.sh script would get the BSDI extensions to install. This has been fixed if you download the most recent versions of the software.

Installing Unix FrontPage Extensions

The default install is into /usr/local/frontpage/version40, or in the directory of your choice, although it will create a link to the above. The installation script installs the FPSE, an HTML version of the SERK, the command-line administration tool (fpsrvadm.exe), the web-based administration tools (in the admcgi directory), the Apache-FP patch, and assorted server and site administrator support files. 

If you have already installed the version30 extensions, the script will move its currentversion symbolic link to point to the version40 directory while leaving the older version intact (actually a good choice, and one that many other software packages might adopt!). The installation script will also prompt you to upgrade the “stub extensions” (located inside each FrontPage-enabled host’s document root, linking to the central extension executables) for each virtual host which has had the older stub extensions installed.

While installing, note that each site’s FrontPage username and password (used for administering their site through the FrontPage client) have no relation to their Unix username and password. For security’s sake (unless the users will be administering their sites through secure server access), it is advisable to make these username/passwords different from their Unix user/password counterparts. Note that if each of your virtual hosts are associated with different Unix users/groups, those “subwebs” should be owned by the Unix user/group of the site maintainer, or they will be unable to administer their site properly.

Next, run the change_server.sh script to automatically replace the existing httpd daemon with a FPSE-supplied one which has the FrontPage patch/module installed (with only the mod_frontpage module and a few other basic modules compiled in; it moves the old one to httpd.orig). The script further creates a suidkey file, used when the FPSE exercise their suid (“set user ID”) bit to make root-level modifications. It also prompts you to upgrade any virtual hosts if necessary.

The Apache version used with the current extensions (as of this writing) is 1.3.12. Note, however, that using an older version of the extensions may replace your Apache daemon with an older Apache daemon. If you would like to compile your own Apache daemon instead of using the change_server.sh-installed one (recommended), you may wish to download the Improved Mod_Frontpage (home.edo.uni-dortmund.de/~chripo/) and then follow the installation/compilation directions listed at www.rtr.com/fpsupport/serk4.0/inunix.htm#installingtheapachepatch. If the patch fails, check out the info at home.edo.uni-dortmund.de/~chripo/faq.asp). 

Administering FrontPage Sites

For the server administrator, the command-line tool fpsrvadm.exe is your primary interface. This tool allows you to install or uninstall FPSE for virtual hosts, set authoring options for them, set directory/executable permissions, chown files, and create or merge subwebs. Running fpsrvadm.exe without any arguments brings you to a GUI-like interface; or, you can run the program directly with command-line options which are detailed at www.rtr.com/fpsupport/serk4.0/adfpsr_3.htm.

When adding FPSE to virtual hosts, choose Apache-FP for “server type” unless you have chosen not to install the FrontPage patch. During a host installation, “stub extensions” will be installed for the selected host, and its executable directories aliased appropriately.

Through fpsrvadm.exe, you can also run a “check and fix” option on existing hosts. While this option won’t tell you anything other than whether the extensions are installed or not (and which version), it can frequently be a valuable tool. When running a check or installing new virtual hosts, be sure to specify the port number (e.g., www.somehost.com:80) or it won’t be recognized.

If e-mailing FrontPage forms fails, be sure to edit your /usr/local/frontpage/we80.cnf file and add the line: 

SendMailCommand:/usr/sbin/sendmail %r

…replacing the above path with the correct path to your MTA of choice. Otherwise, FrontPage forms will not be able to send e-mail. I’m not sure why this isn’t part of the default installation, but it isn’t. Also, note that frequently users may use FTP rather than use their FrontPage client to upload files to their site. A common problem is that if an entire directory is uploaded by the user via FTP, it may overwrite the FPSE located in that directory, and it may be necessary to reinstall the FPSE for that host.

Documentation and Support

Your official first stop for documentation will be the “official” version of the SERK at officeupdate.microsoft.com/frontpage/wpp/serk/. The SERK is required reading for any sysadmin to understand FrontPage, especially those with training in Unix and not Microsoft systems. Unfortunately, the FP 2000 SERK (the FP 98 SERK did not share this problem) is the first Unix documentation package I have ever encountered which does not even mention the terms “problem” or “troubleshooting.”

For real-world administrators, your un-official first stops will be the RTR FrontPage 2000 FAQ (www.rtr.com/fpsupport/faq2000.htm) and discussion board (www.rtr.com/fpsupport/discuss.htm). While the discussion boards aren’t always much help, the FAQ is absolutely essential, answering most of the common questions that can make the difference between supporting FrontPage and chewing the Ethernet cable off the server in frustration. I’m not sure why elements of the FAQ haven’t been integrated with the SERK (by the way, would a few man pages be too much to ask?). It is essential that, because of how disorganized much of the FrontPage/Unix documentation is, you should scour everyavailable resource before considering a problem unsolvable.

Part of my problem with the Unix FPSE is that they use Microsoft’s seemingly proprietary way of describing things, which makes life difficult for Unix administrators. For example, the collections of a user’s web pages are normally known as a “web site,” which has a “document root” and may be a “virtual host.” In the Front Page Mirror Universe, the evil Captain Kirk has a goatee, and this is known as a “FrontPage web,” with a “root web” and “virtual webs.” To use my favorite acronym, “WTF?”

There is a newsgroup available at microsoft.public.frontpage.extensions.unix. However, a quick perusal of this newsgroup’s messages over the past several months shows that the postings appear to be mainly problems and questions – without many answers. There was a FrontPage/Unix mailing list ([email protected]), but as of this writing [email protected] no longer seems to be responding for subscriptions. I recommend searching Google (www.google.com) for FrontPage web resources.

Conclusions

Supporting FrontPage websites on Freenixes is very possible, and for basic purposes, it works very well. However, be prepared to do your homework, and know that it may not be easy. Administering a web server always requires serious research and work; depending on your tastes, dealing with FrontPage on Unix may require too much.