Recently a colleague contacted me with a question relating to hostname resolution and DNS on AIX 6.1. I thought it was an interesting discussion so I thought I’d share it with you here.

 

His question was basically this:

 

“In AIX 6.1, as you know, the resolv.conf has some additional options. Do you know what would happen if I have two nameservers in my file and the target hostname isn't found, will the second nameserver necessarily be looked up? The man page says:

 

If more than one name server is listed, the resolver routines query each name server (in the order listed) until either the query succeeds or the maximum number of attempts have been made.

 

but the rotate option seems to be set for that purpose:

 

Enables the resolver to use all the nameservers in the resolv.conf file, not just the first one.

 

If I have multiple name servers in /etc/resolv.conf, and the first one is available but the query fails, will the name resolution inevitably go to the second nameserver?

 

(My /etc/netsvc.conf entry says:

hosts = local4, bind4

and I don't have NSORDER set).

 

Any thoughts on how I could test this?”

 

A while back I mentioned that there were some new options available in the resolv.conf file for AIX 6.1, including the rotate feature. So, my response to this question was as follows.

 

By default, if the first nameserver is able to answer the query, either by returning the IP address for the target hostname OR a 'host does not exist', then this equates to a successful lookup. Only if the first nameserver does not respond and/or times out will the resolver routine send the query to the next nameserver in the list.

 

To debug this you could use the RES_OPTIONS environment variable and examine the output to see what nameservers are being called and when and in what order. For example:

 

- In the following test, my resolv.conf file has what you would typically configure i.e. a couple of nameservers and a domain entry. Note that I have two nameservers listed in this file.

 

# cat /etc/resolv.conf

nameserver 10.1.50.201

nameserver 10.1.50.202

domain cg.com

 

- I then perform a lookup of a host that is known to DNS and returns an IP address. The output indicates that only one nameserver is queried not both. As expected.

 

# RES_OPTIONS=debug host mygoodhostname | grep Query

;; Querying server (# 1) address = 10.1.50.201

 

- Likewise if I perform a lookup on a hostname that is not known to DNS, I receive a reply from the first nameserver in the list only. Again, as expected.

 

# RES_OPTIONS=debug host mybadhostname | grep Query

host: 0827-801 Host name mybadhostname does not exist.

;; Querying server (# 1) address = 10.1.50.201

 

- Now, if I add the new rotate option to my resolv.conf file, I observe different behaviour. Both nameservers are queried, regardless.

 

# cat /etc/resolv.conf

nameserver 10.1.50.201

nameserver 10.1.50.202

domain cg.com

options rotate

 

-    Both nameservers are queried to lookup the hostname of a host known to DNS.

 

# RES_OPTIONS=debug host mygoodhostname  | grep Query 

;; Querying server (# 1) address = 10.1.50.202

;; Querying server (# 2) address = 10.1.50.201

 

- Again, both nameservers are queried to lookup the hostname of a host not known to DNS. In this case, the second nameserver (10.1.50.202) is bogus and it is actually the first nameserver, 10.1.50.201, that replies i.e. Query #1.

 

# RES_OPTIONS=debug host mybadhostname  | grep Query

host: 0827-801 Host name mybadhostname does not exist.

;; Querying server (# 1) address = 10.1.50.202

;; Querying server (# 2) address = 10.1.50.201

;; Querying server (# 1) address = 10.1.50.201

 

I finished off my response by stating that this approach was probably good practice, but might have the potential to slow down hostname lookups if there are several (max. of 3) nameservers to query. I expect the performance impact would be minimal. If he was concerned with the performance hit, he could always enable the netcd daemon to cache DNS lookups locally, which might speed things up for hosts that were referenced frequently.

 

Which brings me to the netcd daemon. This was first introduced with AIX 6.1 and is included in the bos.net.tcp.client fileset.

 

# lslpp -f bos.net.tcp.client | grep netcd

                        /usr/sbin/netcdctrl

                        /usr/sbin/netcd

                        /usr/samples/tcpip/netcd.conf

 

This new subsystem can be enabled to help improve network performance and reduce network traffic. You can configure this daemon to cache answers from DNS, NIS and other server queries. This daemon is not activated by default in AIX 6.1.

 

The netcd daemon can cache resolver lookups to a network resource such as a DNS server. It will populate its cache with the result of each query. Negative answers are cached as well. When an entry is inserted to the cache, a TTL is associated to it. For DNS queries, the TTL value returned by the DNS server is used (with the default settings). The daemon will also check periodically for expired entries and remove them.

 

There are a number of configurable options for netcd. However, on my test LPAR, I simply ran the following command to start the daemon and test it. I used the lssrc command to get an overview of the active configuration.

 

# startsrc –s netcd

# lssrc -ls netcd

Subsystem         Group            PID          Status

 netcd            netcd            569432       active

Debug                     Inactive

Configuration File        /etc/netcd.conf

Configured Cache         local services

Configured Cache         local protocols

Configured Cache         local hosts

Configured Cache         local networks

Configured Cache         local netgroup

Configured Cache         dns services

Configured Cache         dns protocols

Configured Cache         dns hosts

Configured Cache         dns networks

Configured Cache         dns netgroup

Configured Cache         nisplus services

Configured Cache         nisplus protocols

Configured Cache         nisplus hosts

Configured Cache         nisplus networks

Configured Cache         nisplus netgroup

Configured Cache         nis services

Configured Cache         nis protocols

Configured Cache         nis hosts

Configured Cache         nis networks

Configured Cache         nis netgroup

 yp passwd.byname

 yp passwd.byuid

 yp group.byname

 yp group.bygid

 yp netid.byname

 yp passwd.adjunct.byname

Configured Cache         ulm services

Configured Cache         ulm protocols

Configured Cache         ulm hosts

Configured Cache         ulm networks

Configured Cache         ulm netgroup

 

If you would like the daemon to start automatically on a system restart, uncomment the following entry from the /etc/rc.tcpip file.

 

#start /usr/sbin/netcd "$src_running"

 

By default, if you start the daemon without configuring it’s associated configuration file (/etc/netcd.conf), then it will start with its default values. So just about everything is cached. If you want to trim down the configuration you can create your own /etc/netcd.conf file. There is a sample file located in /usr/samples/tcpip/netcd.conf. You can copy the file to the /etc/ directory and use it as a template for your configuration.

 

The netcdctrl command can be used to control and manage the netcd cache(s). You can dump the current contents of a cache, flush a cache, change the logging level and view statistics. To verify that netcd was caching DNS lookups on my test system, I performed the following.

 

- First I dumped the DNS cache to a file. The contents did not contain any cached DNS lookups at this point in time.

 

# netcdctrl -t dns -e hosts -a /tmp/dns.out

# cat /tmp/out1

CACHE dns, hosts, name

END CACHE dns, hosts, name

CACHE dns, hosts, address

END CACHE dns, hosts, address

 

- Next performed a DNS lookup of an internet host, ibm.com.

 

# host ibm.com

ibm.com is 129.42.17.103

 

- Again, I dumped the contents of the cache. Now I could see a cached entry for ibm.com.

 

# netcdctrl -t dns -e hosts -a /tmp/dns.out

# cat /tmp/dns.out

CACHE dns, hosts, name

>>>>>>>>>>>>>>>>>>>>>>>>>>>> ELEM #1

Expiration date : Wed Jan 27 07:50:24 2010

Ulm or resolver name : dns

Query type : 10100002

Query length : 7

Answer (0: positive; otherwise : negative) : 0

Query key : 1264134311

String used in query : ibm.com

Additional parameters in query:

        query param1 : 2

        query param2 : 0

Length of cached element : 37

################### hostent

Number of aliases = 0

Number of addresses = 3

Type = 2

Length = 4

Host name = ibm.com

Alias = 

Address = 129.42.17.103

Address = 129.42.18.103

Address = 129.42.16.103

#################### end of hostent

>>>>>>>>>>>>>>>>>>>>>>>>>>>> END ELEM #1

 

END CACHE dns, hosts, name

CACHE dns, hosts, address

END CACHE dns, hosts, address

 

It is also possible to flush the cache if something is stale and needs to be refreshed manually.

 

# netcdctrl -t dns -e hosts -f

 

The netcd daemon can cache lookups for all sorts of resolver queries (not just DNS). Some of these include local (/etc/hosts), NIS, NIS+ and YP.

 

http://www-01.ibm.com/support/docview.wss?uid=isg3T1010925

 

http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.cmds/doc/aixcmds4/netcd.htm

 

http://www.redbooks.ibm.com/abstracts/sg247559.html

 

 

Follow me on twitter: http://twitter.com/cgibbo