dec2003.tar

Domain Management Using LDAP

Divya Sundaram

If you're a systems administrator for a large network, you have to manage a wealth of information to keep the network running efficiently. Where is a server located, what rack, who owns it, when was it last patched, etc.? Sometimes we squirrel away data like that by including comments in the hosts map or in a knowledge database. Some organizations spend a tremendous amount of time and money developing or adapting asset management or change management software to track this information.

I adapted the LDAP server used for Unix naming services to manage all this information. By deploying an LDAP directory with a schema that conforms to RFC 2307, you can relatively easily transition to using LDAP as a naming service instead of NIS or even YP.

When we deployed the Sun ONE Directory Server as a component of Sun Microsystems' Solaris 9, it opened up some new opportunities to exploit LDAP Directories for managing the servers and services in our Unix domain. Having a well-designed and maintained LDAP Directory represented an opportunity to realize significant cost savings by replacing expensive and proprietary solutions for managing domain and server information and for asset management.

What Software You Need

To begin, you need an LDAP server. I use the Sun ONE Directory Server for all the examples in this article, because it's bundled with Solaris and is what I use at work. You could use Novell's eDirectory, OpenLDAP, or even the Microsoft Active Directory.

You also need the C LDAP SDK. You can use Mozilla's, or you can download one from Netscape or Sun. You can also use the software from the OpenLDAP Project at:

http://www.openldap.org

The SDK comes with two tools that are especially useful: ldapsearch and ldapmodify. These are respectively used to query and update an LDAP database. I chose to use these over the ones that are shipped with the LDAP server because they can be statically compiled and installed on all Unix systems.

Most of my scripts are written using Perl. There are two options for writing Perl scripts that interface with LDAP: PerLDAP from Mozilla and Net::LDAP (or, confusingly, Perl-ldap) by Graham Barr. There is also the now obsolete Net::LDAPapi by Clayton Donley. I use the PerLDAP modules from Mozilla, but the others are also quite adequate for most systems administration tasks.

The RFC 2307 Based LDAP Directory Service

The LDAP Directory Server is an object database capable of managing millions of entries. This means that, instead of the traditional key-value pairings used for traditional Unix maps, there is now the primary key and an arbitrary number of attribute values available as part of the map entry. The number and types of these attributes are essentially indefinitely extensible and allow the LDAP database to be used to replace or augment other domain management tools.

RFC 2307 lays out a method for storing the standard NIS maps into LDAP directories. It defines a schema for an LDAP database and a method for mapping NIS map entries into objects in the LDAP directory. This RFC has been adopted by most of the major Unix vendors. You can read the RFC at:

http://www.faqs.org/rfcs/rfc2307.html

For example, the following hosts map entry:

192.168.0.201 sol9login1 # Solaris 9 Login Host

becomes:

$ /opt/myOrg/bin/ldapsearch -h ldap0.avnika.com -b \
    "ou=hosts, dc=avnika, dc=com" cn=sol9login1
dn: cn=sol9login1+ipHostNumber=192.168.0.201, ou=Hosts, dc=avnika, dc=com
objectClass: ipHost
objectClass: device
objectClass: top
cn: sol9login1
ipHostNumber: 192.168.0.201

The comment information contained in the hosts map file was discarded. The other NIS maps are similarly mapped into LDAP database objects.

We have a large network consisting of servers used by various development groups, as well as servers dedicated to running business applications. We have a series of machines with the Sun ONE Directory Server 5.2 deployed on them. Most of the LDAP servers are multi-homed and strategically located across the network so that most NIS traffic does not leave the subnet of the NIS clients. This makes the LDAP directory a highly available and, through the use of replicas, robust and distributed repository for storing this data.

Schema Extensions

When extending schemas, it is important to ensure that the objectclasses and attributes are named uniquely so that they will not clash with the naming of the attributes and objectclass by other tools and applications that may be sharing the LDAP directory. Here I used the prefix "myOrg" for all of the attributes I added -- you can, and should, substitute the appropriate name or acronym that uniquely describes your organization to try to minimize the chances of naming collisions.

I first created an objectclass "myOrgHost" that would contain all the attributes I have added to the schema. For the Sun ONE Directory Server, objectclasses and attributes can be added from the Sun Console. This is shown in the dialog depicted in Figure 1. If I had chosen either the objectclass ipHost or Device as the parent objectclass, the attributes could only be associated with hosts. I chose to derive it from the objectclass top so that the LDAP database could be extended to track items that are not hosts -- like RAID arrays, Brocade switches for the LANs, etc.

Once the Objectclass has been created, another dialog is used to add attributes to this objectclass. Figure 2 shows the dialog used to add a given attribute. All attributes are created as "Case Insensitive String" type (also known as "DirectoryString"). I have made most of them multi-valued -- except where having multiple values does not make sense. For example, the attribute myOrgDefaultRouter is single-valued, because it does not make sense to have multiple default routers. Making these single-valued also means that any operation that tries to make them multi-valued will fail.

Asset Tracking Using the Hosts Map

Because every host in the hosts map will be represented in the LDAP directory, it makes sense to extend that map to track information that may otherwise be managed in asset tracking systems. The challenge is to decide what to track in the database and what to leave out. I was able to capture information based upon what I needed to track:

dn: cn=sol9login1+ipHostNumber=192.168.0.201, ou=Hosts, dc=avnika, dc=com
objectClass: ipHost
objectClass: device
objectClass: top
objectClass: myOrgHost
cn: sol9login1
ipHostNumber: 192.168.0.201
myOrgDefaultRouter: 192.168.0.254
myOrgTriJack: 1-22-34
myOrgTriJack: 1-22-35
myOrgTriJack: 1-23-08
myOrgTriJack: 1-23-09
myOrgRootMail: divya.sundaram@avnika.com
myOrgTerminalServer: mwtermserver1.avnika.com:14
myOrgServerLocation: 4445-12-5
myOrgFacility: MIDWEST
myOrgCustomer: Compiler Development Group (CDG01)
myOrgCustomer: Compiler QA Group (CDG05)
myOrgCustomer: Architecture and New Projects Group (CDG67)
myOrgSLA: Level A
myOrgAssetNumber: sun-6CEDFGHY
myOrgServerType: Sun E 450
myOrgServerMemory: 2GB
myOrgServerNumCPU: 4x450Mhz

In this example, I have taken the standard entry and added other attributes to capture information about the host that I want to track. Some attributes are multi-valued: servers that are either multi-homed or have trunked connections will be "punched down" to multiple locations as shown by the myOrgTriJack attribute.

When troubleshooting this host, I can immediately discern a lot of information -- its location (myOrgServerLocation), who's going to be primarily impacted by an outage on this server (myOrgCustomer), and who I should contact as the primary administrator for this server (myOrgRootMail). The SLA level (myOrgSLA) actually allows me see the level of service associated with this host and helps determine the escalation process to be followed in resolving issues with this host.

The schema can be extended to house other host-specific information that may usually be tracked in Asset Tracking software as well. Although LDAP directories don't offer a syntax as rich as SQL's for queries that are used in generating audit reports, I have found that scripts using PerLDAP (http://www.perldap.org) are quite adequate for generating snapshots for audit reports.

Using PerLDAP, it was also extremely simple to build CGIs that updated the data in the LDAP database. A host configuration Web page allows administrators to update the information associated with a given host. The CGI also performs some error checking to ensure that the information is not duplicated or in error. For example, it ensures that a host will not be assigned an IP that is already taken by another server, and that the default router listed actually exists and is appropriate.

Server Configuration Tracking Using the Hosts Map

The second type of information that can be tracked pertains to the host's operating system configuration. Here is what I added for tracking operating system configuration:

dn: cn=sol9login1+ipHostNumber=192.168.0.201, ou=Hosts, dc=avnika, dc=com
...
myOrgETCRelease: Solaris 8 10/00 s28s_u2wos_11b SPARC
myOrgPatchLevel: Solaris 8 SPARC May 2003 (applied 2003-07-03)
myOrgResolvConf: SearchList: avnika.com
myOrgResolvConf: SearchList: engineering.avnika.com
myOrgResolvConf: NameServer: 192.168.0.10
myOrgResolvConf: NameServer: 192.168.0.11
myOrgResolvConf: NameServer: 192.168.0.12
myOrgResolvConf: Domain: avnika.com
myOrgNSSWITCH: hosts: files dns ldap
myOrgNSSWITCH: passwd: files ldap
myOrgNSSWITCH: shadow: files
myOrgNSSWITCH: group: files ldap
myOrgNSSWITCH: bootparams: files
myOrgNSSWITCH: ethers: files
myOrgNSSWITCH: netmasks: files
myOrgNSSWITCH: networks: files
myOrgNSSWITCH: protocols: files
myOrgNSSWITCH: rpc: files
myOrgNSSWITCH: services: files
myOrgNSSWITCH: netgroup: files
myOrgNSSWITCH: publickey: files
myOrgNSSWITCH: automount: files
myOrgNSSWITCH: aliases: files

I started off storing the contents of /etc/defaultrouter in the host map. Then I added the contents of the /etc/release file as well as the contents of the resolver configuration (/etc/resolv.conf) and the name service configuration (/etc/nsswitch.conf) files. I also added an attribute (myOrgPatchLevel) that captures the latest patch level applied to the host as well as the date that it was applied.

On some machines on which I had Veritas installed, I included the Veritas software information and the license keys being used. This can be extended to include the software information for other tools that are installed on the host.

For Solaris hosts, one option I considered was to include the packages that have been installed on the host as well as all the patches or patch clusters that have been applied. I do not believe that putting all of this information in the Directory directly is appropriate -- mainly because I see 580 system packages, 66 application and user packages, and 18 other packages installed on my Solaris 8 servers. This much data can only be tracked by adhering to a process for keeping this information current, because the information is dynamic.

For a look into what else you could stuff into the directory, consider the output of the "explorer" tool on a Solaris host. There is far too much information there to be stored en masse in the directory. But you can pick out the information you consider vital and store that in the directory. For example, you may want to store the contents of the /etc/vfstab (or /etc/fstab) file. Or, you might choose the active lines of /etc/inetd.conf. In essence, anything that is relatively static and is useful to have available when troubleshooting a problem.

My choices so far allow me to use the LDAP directory as a distributed database containing the configuration information necessary to bring any host back online in case of a failure. This greatly reduces the cycle time for bringing a server back online after a crash. I can also, at a glance, see what configuration a host is supposed to use and whether it deviates from that. Remember that the LDAP directory is being used as an NIS replacement, so this information is replicated onto every machine that would function as an "NIS slave" in your environment.

Extending the Host Schema for Account Provisioning

Our business application servers are deployed to use a sparse Passwd file -- this contains the system accounts, the application account, and the accounts of the application administrators taken from the standard Passwd map. All their data is kept in flat files; they do not use NIS or YP. For these hosts, I keep the flat files synchronized with the NIS domain by using scripts to grab the information from LDAP and update the various maps that are needed.

As an example, let's consider an Oracle database server. A Unix user, "oracle", is created and it's the only Unix account that resides on the box. The user password is known to the Oracle DBAs and is only used in disaster recovery scenarios. Most of the time, the DBAs log into the server with their own logins and then use a tool like PowerBroker to switch user to "oracle" to perform database maintenance activities.

It is desirable for us to keep the username, userids, and passwords consistent between the application hosts and the rest of the NIS maps. The alternative is to create point solutions, which increases the possibility of confusion and thus the support costs of the environment.

We can do this by further extending the Hosts map to include the user information for the host. The second step is to classify hosts into functional groups and then use LDAP groups to manage account provisioning.

I will illustrate this using the case of the Oracle server. We have been told that, in addition to the standard system accounts in /etc/passwd, we need to provide access to the following users: oracle, divya, samson, rodney, monitor. We can simply indicate this information by using an attribute that is added to the hosts entry:

$ /opt/myOrg/bin/ldapsearch -h ldap0.avnika.com -b \
"ou=Hosts, dc=avnika, dc=com" cn=sol8oradb1
dn: cn=sol8oradb1+ipHostNumber=192.168.0.191, ou=Hosts, dc=avnika, dc=com
objectClass: ipHost
objectClass: device
objectClass: top
cn: sol8oradb1
ipHostNumber: 192.168.0.191
myOrgServiceUserID: oracle|x|198|88|Oracle User|/export/oracle|/bin/ksh
myOrgServiceUserID: divya
myOrgServiceUserID: samson
myOrgServiceUserID: rodney
myOrgServiceUserID: monitor

In this case, the user "oracle" will have only its password plucked out of the "passwd" map stored in the directory. The rest of the information is contained in the attribute as defined above and serves to override the information in the LDAP-based "passwd" map stored in the "ou=people, dc=avnika, dc=com" container.

Meanwhile, to implement the configuration that has been applied, on the server sol8oradb1, we use a PerLDAP script to grab the information from the LDAP server and write it to /etc/passwd.new. Then a cron job swaps out /etc/passwd for /etc/passwd.new after performing a sanity check.

This approach is much like what is done using the old "Yellow Pages" NIS (i.e., yp) where a cron job would be used to download the maps onto the NIS slaves. The advantage here is that the script can be enabled to permit some sanity checking functions to ensure that a corrupt file is not being generated.

A Web interface can be built to allow administrators to manage the information for these sparse passwd maps. However, this information must be tracked accurately for every host. Whenever a new person joins the support team, the administrators must manually add this user's login ID to the entry for each host that is affected. Because this is a manual process, it is prone to error. This approach is best for a few systems, but doesn't scale well when you have a large number of servers.

In our data center, business applications servers may be deployed as part of a project, or based upon functionality. For example, we have one Oracle Database Administrator Team but more than a hundred Oracle database servers. In this scenario, all the members of the DBA team would need to have access to all the servers classified as Oracle servers. To solve this problem, I created an " LDAP group" in a new branch of the LDAP directory and associated hosts and accounts with it. The logical way to do this is as follows:

dn: cn=Oracle DBA, ou=application groups, dc=avnika, dc=com
objectClass: top
objectClass: myOrgApplicationGroup
myOrgHostCN: cn=sol8oradb1+ipHostNumber=192.168.0.191
myOrgHostCN: cn=sol8oradb2+ipHostNumber=192.168.0.195
myOrgHostCN: cn=sol7oradb3+ipHostNumber=192.168.0.198
myOrgHostCN: cn=sol8oradb4+ipHostNumber=192.168.0.212
myOrgHostCN: cn=sol9oradb5+ipHostNumber=192.168.0.211
myOrgServiceUserID: uid=oracle|x|198|88|Oracle User|/export/oracle|/bin/ksh
myOrgServiceUserID: uid=divya
myOrgServiceUserID: uid=samson
myOrgServiceUserID: uid=rodney
myOrgServiceUserID: uid=monitor

This entry in the LDAP database indicates both the hosts that are classified as belonging to the Oracle DBA team as well as the non-system accounts that need to be included in its /etc/passwd file. The details of the Oracle application account are listed in full, but the rest of the user accounts are read from the user's LDAP entry.

The script that retrieves the information from the LDAP Server and parses it to build the new /etc/passwd file would be installed on all the servers and will allow the servers to automatically generate the /etc/passwd and /etc/shadow files. It could be run periodically from within cron. This approach requires some planning, but the advantage is that it reduces the possibility of error because updates made once get propagated everywhere.

Conclusion

Over the next few years, Unix administrators will become extremely familiar with LDAP. Sun's decision to include their Directory Server as part of the Solaris operating system will allow systems administrators to leverage the directory for a variety of tasks. The ability to create new fields in the LDAP database provides a great deal of flexibility to solve common systems administration problems.

Divya Sundaram is currently employed as the Manager of Directory Services at Motorola. Divya has been a Unix systems administrator since 1994 in various roles at Motorola and been involved with LDAP directories since 1997.