NIM Master, NIMSH and SSL on AIX 7.1 TL4 SP3Whilst working with one of my AIX customers recently I discovered a problem with NIMSH and SSL. The customer had updated their NIM master from AIX 7.1 TL4 SP1 to AIX 7.1 TL4 SP3. After the SP update, any attempt to connect to a NIM client (over NIMSH+SSL), from the NIM master, would simply hang. For example, we tried to list the filesets on the NIM client with this command, which never returned any output.
[root@750lpar4]/ # nim -o lslpp 750lpar9
The /var
… Thu Jan 12 14:31:49 2017 Loading certificates.. Thu Jan 12 14:31:49 2017 Loading private key file.. Thu Jan 12 14:31:49 2017 create BIO …
NIM master: 750lpar4 7100-04-03-1543
NIM client: 750lpar9 7100-04-02-1614
[root@750lpar4]/ # lsnim -l 750lpar9 750lpar9: class = machines type = standalone connect = nimsh (secure) platform = chrp netboot_kernel = 64 if1 = 10_1_50 750lpar9 0 cable_type1 = N/A Cstate = ready for a NIM operation prev_state = not running Mstate = currently running cpuid = 00F603CD4C00 Cstate_result = success
The root cause of the problem become apparent when we ran truss against the nim –o command.
[root@750lpar4]/ # truss -adef -o truss.lsnim.out -w all nim -o lslpp 750lpar9
[root@750lpar4]/ # cat truss.lsnim.out … 13959372: C o u l d n o t l o a d m o d u l e / u s r / l i b / l 13959372: i b s s l . s o .\n S y s t e m e r r o r : N o s u c h 13959372: f i l e o r d i r e c t o r y … 19267612: C o u l d n o t l o a d m o d u l e / u s r / l i b / l 19267612: i b c r y p t o . s o .\n S y s t e m e r r o r : N o s u 19267612: c h f i l e o r d i r e c t o r y
The required shared library object files were missing on the NIM master.
[roo libssl.so not found libcrypto.so not found
We fixed this issue by extracting the missing files from the (existing) /usr/lib/libssl.a and /usr
[roo
[roo x - /usr/lib/libssl.so
[roo x - /usr
[roo -rwxr-xr-x 1 root system 724913 Jan 18 09:08 libssl.so -rwxr-xr-x 1 root system 3031337 Jan 18 09:08 libcrypto.so
After that, the nim –o commands started working again.
[roo HELLO
So, the question is why did this happen? Well, in the past, the libssl.so.0.9.8 shared object was extracted by NIM, but more recent updates by the OpenSSL version has forced IBM to move to libssl.so. Usually, the extracted shared library object is added (if not currently present) when nimconfig -c is run. But given that this is an existing NIM master, we did not want to run this again (as we would lose all of the current SSL key access). So extracting the objects is preferred. The problem is due to the fact that the libssl.so and libcrypto.so files are not populated when the AIX 7100-04-03 update is applied. This is a bug and will be officially addressed, soon, under APAR IV93152 NIM push operation to client hang on nimsh over SSL.
I believe this issue may also occur when you migrate your NIM master from AIX 7.1 to 7.2 (with nimadm for example). But I need to do more testing to reproduce and confirm the issue.
Here’s one good reason to setup NIMSH over SSL.
NIMSH, SSL and LPM http
The following link is a great reference guide for configuring NIMSH over SSL.
NIMSH over SSL http
|