Tepatche -- Automatic OpenBSD System Patcher

Gunnar Wolf

OpenBSD is my preferred operating system for highly exposed or security-oriented servers. OpenBSD is designed specifically with security in mind and, with "only one remote hole in the default install, in more than 7 years" (according to www.openbsd.org) it becomes the obvious choice for any exposed server.

No code written by humans, however, can be considered perfect, and OpenBSD is no exception. A new OpenBSD release comes out every six months, and some flaws are always found. Because OpenBSD is a security-minded system, the core team releases a source code patch every time a bug is found and corrected. During the life of any given release, an average of 30 patches are issued (see sidebar "OpenBSD Releases' Life Cycle").

Patching an OpenBSD System the Standard Way

OpenBSD was designed with the idea that a simple implementation tends to be and stay more secure than a complex one. Updates are distributed as source code patches. To install them, the sys admin must patch the OpenBSD sources and recompile and install the relevant portions of the system. This process is easy to implement since patches come with complete instructions. Keeping the systems current can be done by subscribing to security-announce@openbsd.org, downloading the patches, then applying and compiling them. Keep in mind, however, that:

1. If you manage many OpenBSD machines, this process can be tedious. Applying the patches to a couple of machines is easy, but patching a whole data center can take hours, particularly if you check the status of each machine (as you should always do).

2. Patching may be postponed for days or weeks, or even forgotten despite our best intentions. This is especially a problem because automatic worms are scanning networks and compromising vulnerable systems.

Therefore, I wrote Tepatche to address these problems. I wrote the first version in June 2002, shortly after the first (and only) known remote vulnerability in the default configuration of OpenBSD surfaced. Some blackhat groups promptly released exploits for this vulnerability, and many systems administrators were left vulnerable. I decided to write Tepatche because periodically checking and applying simple patches is a task that a computer can do better than a human. See the sidebar "Why 'Tepatche'?".

Program Logic

Tepatche should be run automatically; I suggest running it daily as a cron job. Its operating logic is quite simple. Roughly:

• Tepatche reads the address of the FTP site for updates (almost always ftp.openbsd.org/pub/OpenBSD/patches/<version>/) from the configuration file and initiates a connection to the site.
• It obtains the list of files available in both the common and architecture-specific directories of the FTP site, compares it to our local copy, and if there are any new files, retrieves them, and schedules them for processing.
• It processes the patch instructions, executes each patch instruction, then checks whether it is in the allowed set of instructions. (This is explained in greater detail in the next section.)
• Tepatche notifies the administrator of any changes made by sending two emails -- one with the full output of the execution, and one with a short summary.

The FTP connection is handled using Perl module Net::FTP, which can be found either in the CPAN or as p5-libnet in OpenBSD's ports tree.

Important Design Considerations

As I mentioned, the program logic is simple. The whole program is slightly more than 400 lines of code, about 25% of which is comments. However, this program is intended to run as root because it has a lot of interaction with the operating system and is intended to modify vital system binaries.

I did my best to ensure that no error condition would go unnoticed, checking almost every statement that uses the FTP module, invokes an external command, or deals with the file system. This resulted in almost 60 possible causes for early program termination. Of course, early termination is always promptly reported to the systems administrator.

Because OpenBSD patches are distributed by FTP and they are not cryptographically signed, we can never be sure whether they were issued by a real OpenBSD developer or they are trojans, so we must trust that the file is legitimate for this process to be automated. Because the patches' headers are basically recipes on how they are to be applied, I decided to deal with this trust by being very selective about which commands Tepatche will automatically execute. The commands Tepatche will agree to run are:

• cd -- You must always change to the affected directory before patching.
• patch -- The command that applies the patch.
• make -- Compile and install the affected portion of the code.

This is enough for the vast majority of the patches, but not for all of them. One third of the patches in 3.2 involve killing a running process or removing older files. Although it is currently not implemented, I intend to include a configuration option (disabled by default) that allows Tepatche to execute any command listed on the patches' headers.

Finally, although most patches are applied to userland (i.e., the binaries, libraries, and other files available in a Unix environment), some patches must be applied to the kernel. I decided Tepatche would only patch the kernel source tree but not rebuild the kernel. Many people custom-configure their kernels, and it's not easy to come up with a standard version of compiling the kernel. The only way to apply the patches is to reboot the system, and no mission-critical system should automatically reboot itself, even more when faced with the danger of having an incorrectly built, unbootable kernel. Thus, the administrator must always rebuild the kernel, and Tepatche will only alert him that he must do so.

Conclusion

I have been running Tepatche for more than a year, and although I do not administer many OpenBSD boxes, it has clearly improved my life as a systems administrator. Knowing I am up-to-date with the patches and knowing that most patches will be automatically applied (or at least, I will get a notice from my computer telling me what to do) has made my life much easier. I have worked with Tepatche on OpenBSD versions 3.1 and 3.2 and 3.3, and have had no problems with it. Tepatche is free software, under the BSD license. You can download it from http://www.gwolf.cx/soft/tepatche/. See the sidebar "Similar Projects" for more information.

Gunnar Wolf has been a systems administrator since 1993, with a special interest in security. He has worked with and promoted free software since 1997 and is active in Mexico's free software community.