Improving SSH with Keychain

Todd A. Jacobs

Keychain is a wrapper for OpenSSH that simplifies key management by allowing use of a single authentication agent across virtual consoles and through multiple logins. Initially released in August 2001 for the Gentoo Linux distribution, keychain has matured into an essential tool for anyone who uses RSA or DSA keys for SSH authentication.

To reap the full benefits of this relatively new tool, a basic understanding of SSH is required. SSH (Secure Shell) is a replacement for services that transmit unencrypted data across the network. While it is most often used as a secure alternative to telnet, it can also replace other services such as rlogin, rsh, rexec, and ftp.

In its basic configuration, SSH creates an encrypted session before prompting the user to enter his system password (as defined on the remote system). If a user often connects to a server, this can become repetitive because he will be prompted for a password each time a connection is made. The problem becomes more pronounced if he opens sessions to multiple systems, each with a different password. Using the same password on multiple systems is a security risk, but keeping track of many passwords, and having to constantly re-enter them, is a real bother. SSH addresses these issues through the use of public key cryptography and an authentication agent named ssh-agent.

Introducing ssh-agent

With ssh-agent, a public/private key pair is generated using the ssh-keygen utility. The private key is encrypted with a password that, for security reasons, should differ from the password used for the user's system account. The public key is then uploaded to all the SSH servers with which the client may want to connect, while the private key is retained on the client system and is not shared with anyone.

If server-side support for public key authentication has been enabled, the SSH client will prompt the user for the password to the private key when initiating a connection. A correct password will unlock the secret key, which the SSH client then uses to complete the authentication process.

So far, this doesn't appear to be any different from authenticating with the user's regular password. The connection is initiated, the user is prompted for a password, authentication is performed, and a connection is then established. Even if things are left there, the ssh-agent approach offers two primary benefits:

1. Two-Factor Authentication -- To authenticate, the user needs to possess a token (the private half of the key pair) in addition to a password. This limits the potential for strictly password-based attacks.

2. Uniformity -- A user can safely use the same password to connect to multiple systems. Because of public key cryptography, the token (private key) and password are stored locally, and are never transmitted to the remote system.

However, the benefits don't stop there. Using ssh-agent can also simplify the login process by reducing repetition. It allows the user to cache the decrypted private key on the client side, so that future authentications can take place without prompting the user for a password. When the SSH client attempts a new connection, it will use the credentials provided by ssh-agent to complete the connection. See the sidebar, "A Few Ways to Launch ssh-agent".

Why Keychain

When ssh-agent launches a new process (or if the output of ssh-agent is evaluated in the current shell), it also creates two environment variables: SSH_AUTH_SOCK and SSH_AGENT_PID. The name of the UNIX-domain socket for the authentication agent is stored in SSH_AUTH_SOCK, while the process ID of the agent is held in SSH_AGENT_PID.

The various SSH tools use these two environment variables to communicate with the authentication agent. The ssh-add utility loads secret keys into the authentication agent using the defined socket, and the ssh client uses it again when connecting to a remote host.

The main drawback to using ssh-agent, especially under X, is that the information used to connect to an ssh-agent is stored in the current environment. If you open a new xterm that doesn't inherit its environment from the shell that originally spawned the agent process, an SSH client process won't know how to connect to the appropriate socket. You would need to launch another instance of ssh-agent just for that xterm, and then load your private keys yet again. Even if you run ssh-agent as part of your XDM/KDM/GDM login (see the sidebar), the authentication agent will exit when you log out.

Collectively, these are the problems that keychain addresses. By sharing information about the ssh-agent process with each login, keychain allows all your other processes access to the same authentication agent. You now only need one running agent per user on each machine, rather than one agent per login shell. And because keychain keeps your authentication agent running and accessible even after you log out, you can also use SSH tools from a crontab or at command without having to resort to dangerous tricks, such as creating secret keys with no defined password.

Installing Keychain

If you're ready to try keychain, you must first download the latest source code. At the time of this writing, the most recent version is 2.0.2. The source code can be found on the project's home page at:

http://www.gentoo.org/projects/keychain.html#doc_chap3

bzcat keychain-2.0.2.tar.bz2 | tar xv

Keychain is now installed and ready to use. However, to make the best use of keychain, you should modify at least one of your shell configuration files. The name of this file will depend on your shell (see Table 1) and how your shell environment has been configured. In general, the first file for each shell listed is most likely the correct one.

For Bourne-compatible (sh) shells, add the contents of Listing 1 to the file that controls your environment variables (e.g., .bash_profile). C-type shells, such as csh or tcsh, will look almost the same, except for the command and filename used to import the environment variables into the current shell. For C shells, use Listing 2 instead.

These commands run the keychain script, optionally load a list of secret keys into the authentication agent, and then import a set of shared environment variables into the current shell. Keychain supports a number of other command-line arguments, which can be found in the keychain documentation.

Note that there are currently no man or info pages distributed with keychain. The keychain documentation is contained in the script itself, and can be displayed with the --help command-line argument. The documentation is currently 73 lines long, so it's a good idea to pipe the output to a pager such as less or more. For example:

keychain --help | less

Now that your login script has been modified, keychain will be invoked whenever a new interactive shell is started. Keychain will first check the process list to see whether an authentication agent is already running under the current user ID. If not, a new agent is started, and the commands to add SSH_AUTH_SOCK and SSH_AGENT_PID to the environment are written to a pair of shell scripts in the .keychain directory -- one for Bourne shells, and one for csh-compatible shells -- so that this information can be shared with other sessions.

If ssh-agent is already running, then the appropriate shell script is sourced to import the existing values for SSH_AUTH_SOCK and SSH_AGENT_PID into the current environment. Remember, this is the secret to keychain: by sharing the values for the process ID and socket for ssh-agent, all login sessions belonging to that user have access to the same authentication agent and so do not require independent copies of the private keys to be reloaded and their passwords to be re-entered.

After having verified that an authentication agent is running for the current user, keychain runs attempts to load any private keys passed as arguments into ssh-agent (see Figure 1). You will be prompted for a password for each secret key in turn until all requested keys have been loaded or the script is aborted with CTRL-C.

Once your keys have been loaded into ssh-agent, keychain returns control to the shell. You can now connect automatically, without password prompting, to any host that has been configured with your public key(s).

At this point, I have covered the essential topics to get you started with keychain. However, there are some additional topics worth mentioning.

Keychain and NFS

The .keychain directory, which contains information about running agents, is located in the user's home directory by default (this can be overridden with the --dir option). In this directory, the filenames generated by keychain include the hostname of the machine to which the user is currently logged in (see Figure 2). Since each host will create its own pair of files in the .keychain directory, keychain is safe to use with NFS-mounted directories.

Key Management

As seen in Listings 1 and 2, keychain can be run with no private keys specified. This is a feature. While keychain certainly makes it easy to load multiple keys when starting an authentication agent, keys can also be added from the command-line after the login process has been completed; this is what keychain itself is doing during the key-loading phase anyway.

Once the socket has been shared, keys can be added to, or removed from, the authentication agent at any time (see Listing 3). The change will instantly affect all keychain-enabled sessions without any need to reinitialize the agent or the login session.

In some environments, it may not be appropriate to leave decrypted keys cached in memory for long periods of time. However, if you kill the authentication agent (see Listing 4), you will have to reinitialize each session in order to share a new agent. The solution is to remove the keys from the agent's cache, while leaving ssh-agent itself running (see Listing 5).

Security and Non-Interactive Key Access

As you can see, since keychain is simply a wrapper around the ssh-agent and ssh-add utilities, most key management tasks can be done either with keychain, or by calling ssh-add directly. However, there is one more value-added benefit to keychain worth mentioning -- interactive login protection.

Historically, if you had a crontab or at command defined that relied on public keys, you had to create a key with no password so that it could be run non-interactively. Keychain allows you to run such non-interactive processes without creating such a vulnerable key.

Since the authentication agent started by keychain continues running after logout, your crontab and at commands have access to any keys you have loaded into ssh-agent. Since keys cached by ssh-agent do not require a password, these non-interactive processes will not hang while waiting for a password. However, the keyfile on disk will remain encrypted, adding a measure of safety.

One vulnerability in this scenario is the potential for abuse by someone who knows your system password. This person could log onto the local system as you, and immediately have access to your authentication agent and, by extension, access to any system for which your agent will authenticate. This is obviously undesirable, so enter the --clear option. By adding --clear to the keychain statement in your login script as follows:

keychain --clear ~/.ssh/key1 ~/.ssh/key2 ~/.ssh/key3

Problems Under Cygwin

Keychain uses the nohup command to ensure that ssh-agent continues to run even after a login session terminates. However, this does not seem to work properly under Cygwin. While testing keychain for this article, I discovered that attempting to close the first session to call keychain would always hang. Other keychain-enabled Cygwin windows can open and close without problems, but the original window cannot close because of the ssh-agent process running under nohup in the background.

To resolve this problem, I wrote a .bash_logout shell script (see Listing 6) for Cygwin, which prompts the user to stop keychain before logging out, and prints a useful message to the screen that will tell the user what's going on in case the window does hang (see Figure 3).

Conclusion

Keychain adds a layer of convenience and control to the core SSH utilities by building on their existing UNIX-domain socket method of communication. I have found it to be a huge time-saver and an essential tool in my own work, and I hope that this article encourages others to try it.

Todd A. Jacobs, CISSP (articles@codegnome.org) is president of CodeGnome Consulting, LTD (http://codegnome.org). In addition to spending way too much of his free time researching obscure technical topics, he likes to ride his Honda Shadow and practice Tai Chi.