A little over a year ago, I wrote about the new AIX Live Update feature, available with the AIX 7.2. I discussed how to use this new feature to install an interim fix (ifix) on a running AIX system without a reboot. On October 11th, 2016, IBM announced the next exciting step in AIX Live Updates.

Introducing IBM AIX 7.2.1 Live Update

http://www.ibm.com/developerworks/aix/library/au-aix7.2.1-liveupdate-trs/index.html?ca=drs-&ce=ism0070&ct=is&cmp=ibmsocial&cm=h&cr=crossbrand&ccy=us

This is a significant enhancement, as it will allow AIX administrators to install TLs and SPs (and ifixes) without restarting their AIX systems. From the announcement:

"AIX Live Update for Technology Levels, Service Packs, and Interim Fixes

Introduced in AIX 7.2, AIX Live Update is extended in Technology Level 1 to support any future update without a reboot, with either the geninstall command or NIM.
The genld command is enhanced to list processes that have an old version of a library loaded so that processes can be restarted when needed in order to load the updated libraries."

In this post I'll show you how to install updates without rebooting your AIX server. I recommend you first review my original article on Live Updates (from Oct 2015) in order to better understand the Live Update process, how it works and the requirements.

TL and SP Live Update support is delivered in AIX 7.2 TL1 (available November 11th 2016).

One of the biggest differences between the Live Update process for TL/SPs versus ifixes, is that you must backup your system prior to the update. This will be used in case you need to back out your system. The easiest way to do this is, is to create an alternate rootvg (alt disk clone).

On my system, I first applied TL1 for AIX 7.2 and verified the correct level was installed.

root@AIXmig / # oslevel -s

7200-01-00-0000

root@AIXmig / # cat /proc/version

Oct 10 2016

11:53:17

1640C_72D

@(#) _kdb_buildinfo unix_64 Oct 10 2016 11:53:17 1640C_72D

I had several "free" disks that I could use for the Live Update process. I'd need at least 3 disks, one for my alternate rootvg (back out), one for the mirror disk and one for the new rootvg. In this case I used hdisk6 (alt rootvg), hdisk3 (mdisk) and hdisk2 (ndisk). This was specified in the lvupdate.data configuration file. All three disks were large enough to hold a complete copy of my existing rootvg.

root@AIXmig / # lspv

hdisk0 00f94f58cecabed6 rootvg active

hdisk1 00f94f58697b768f datavg active

hdisk2 00f94f58697b7655 None

hdisk3 00f94f58ce74a739 None

hdisk4 00f94f58a3b2f963 None

hdisk5 00f94f58a3b2f9d4 None

hdisk6 00f94f58def77f2c None

root@AIXmig / # cat /var/adm/ras/liveupdate/lvupdate.data

...

disks:

nhdisk = hdisk2

mhdisk = hdisk3

Note, with 7.2 TL1, only two disks need to be specified in the lvupdate.data file. The tohdisk and tshdisk are not needed with TL1 unless you have a paging or dump device outside of rootvg.

I cloned my rootvg to a spare disk (hdisk6) first. I specified the -B flag to ensure the boot list was not changed.

root@AIXmig / # alt_disk_copy -Bd hdisk6

The updates I needed to install were located in the /tmp directory.

root@AIXmig /tmp/cg/lku_ptfs # ls -ltr

total 50328

-rw-r--r-- 1 root system 454656 Oct 20 00:49 bos.liveupdate.rte.7.2.1.1.U

-rw-r--r-- 1 root system 41306112 Oct 20 00:50 bos.mp64.7.2.1.1.U

-rw-r--r-- 1 root system 253952 Oct 20 00:50 bos.net.tcp.ftp.7.2.1.1.U

-rw-r--r-- 1 root system 369664 Oct 20 00:50 bos.rte.install.7.2.1.1.U

-rw-r--r-- 1 root system 183296 Oct 20 00:50 bos.rte.libcfg.7.2.1.1.U

-rw-r--r-- 1 root system 1713152 Oct 20 00:50 bos.sysmgt.hmc.7.2.1.1.U

-rw-r--r-- 1 root system 155648 Oct 20 00:50 devices.vdevice.IBM.vfc-client.rte.7.2.1.1.U

-rw-r--r-- 1 root system 2264 Oct 20 10:31 .toc

All of the filesets were currently installed at the 7.2.1.0 level.

root@AIXmig / # lslpp -l bos.liveupdate.rte bos.mp64 bos.net.tcp.ftp bos.rte.install bos.rte.libcfg bos.sysmgt.hmc devices.vdevice.IBM.vfc-client.rte

Fileset Level State Description

----------------------------------------------------------------------------

Path: /usr/lib/objrepos

bos.liveupdate.rte 7.2.1.0 COMMITTED Live Update Runtime

bos.mp64 7.2.1.0 COMMITTED Base Operating System 64-bit

Multiprocessor Runtime

bos.net.tcp.ftp 7.2.1.0 COMMITTED TCP/IP ftp Client Application

bos.rte.install 7.2.1.0 COMMITTED LPP Install Commands

bos.rte.libcfg 7.2.1.0 COMMITTED libcfg Library

bos.sysmgt.hmc 7.2.1.0 COMMITTED HMC Client Runtime

devices.vdevice.IBM.vfc-client.rte

7.2.1.0 COMMITTED Virtual Fibre Channel Client

Support

Path: /etc/objrepos

bos.liveupdate.rte 7.2.1.0 COMMITTED Live Update Runtime

bos.mp64 7.2.1.0 COMMITTED Base Operating System 64-bit

Multiprocessor Runtime

bos.net.tcp.ftp 7.2.1.0 COMMITTED TCP/IP ftp Client Application

bos.rte.install 7.2.1.0 COMMITTED LPP Install Commands

bos.rte.libcfg 7.2.1.0 COMMITTED libcfg Library

bos.sysmgt.hmc 7.2.1.0 COMMITTED HMC Client Runtime

devices.vdevice.IBM.vfc-client.rte

7.2.0.0 COMMITTED Virtual Fibre Channel Client

Support

I ensured that my AIX partition could authenticate with my HMC (hsc02) and that I had a valid token.

root@AIXmig / # hmcauth -l

Could not retrieve <tokens>.

root@AIXmig / # hmcauth -a hsc02 -u hscroot -p abc1234

root@AIXmig / # hmcauth -l

Address : 10.1.50.30

User name: hscroot

port : 12443

Next I performed a preview, without specifying any updates. This was to verify the environment was ready to accept a Live Update and shake out any problems.

root@AIXmig / # geninstall -k -p

Validating live update input data.

Computing the estimated time for the live update operation:

-------------------------------------------------------

LPAR: AIXmig

Blackout_time(s): 18

Global_time(s): 708

Checking mirror vg device size:

------------------------------------------

Required device size: 7744 MB

Given device size: 10239 MB

PASSED: device size is sufficient.

Checking new root vg device size:

------------------------------------------

Required device size: 7744 MB

Given device size: 10239 MB

PASSED: device size is sufficient.

Checking temporary storage size for original LPAR:

------------------------------------------

Required device size: 0 or Undetermined MB

Given device size: 2047 MB

PASSED: device size is sufficient.

Checking temporary storage size for surrogate LPAR:

------------------------------------------

Required device size: 0 or Undetermined MB

Given device size: 2047 MB

PASSED: device size is sufficient.

Validating the adapters and their paths:

------------------------------------------

PASSED: adapters can be divided into two sets so that each has paths to all disks.

Checking lpar minimal memory size:

------------------------------------------

Required memory size: 2048 MB

Current memory size: 2048 MB

PASSED: memory size is sufficient.

Checking other requirements:

------------------------------------------

PASSED: sufficient space available in /var.

PASSED: sufficient space available in /.

PASSED: sufficient space available in /home.

PASSED: no existing altinst_rvgLvup.

PASSED: rootvg is not part of a snapshot.

PASSED: pkcs11 is not installed.

PASSED: DoD/DoDv2 profile is not applied.

PASSED: Advanced Accounting is not on.

PASSED: Virtual Trusted Platform Module is not on.

PASSED: multiple semid lists is not on.

PASSED: sufficient file system space for interim fix(es) is available.

PASSED: The trustchk Trusted Execution Policy is not on.

PASSED: The trustchk Trusted Library Policy is not on.

PASSED: The trustchk TSD_FILES_LOCK policy is not on.

PASSED: the boot disk is set to the current rootvg.

PASSED: the mirrorvg name is available.

PASSED: the rootvg is uniformly mirrored.

PASSED: the rootvg does not have the maximum number of mirror copies.

PASSED: the rootvg does not have stale logical volumes.

PASSED: all of the mounted file systems are of a supported type.

PASSED: this AIX instance is not diskless.

PASSED: no Kerberos configured for NFS mounts.

PASSED: multibos environment not present.

PASSED: Trusted Computing Base not defined.

PASSED: no local tape devices found.

PASSED: live update not executed from console.

PASSED: the execution environment is valid.

PASSED: enough available space for /var to dump Component Trace buffers.

PASSED: enough available space for /var to dump Light weight memory Trace buffers.

PASSED: all devices are virtual devices.

PASSED: No active workload partition found.

PASSED: nfs configuration supported.

PASSED: HMC token is present.

PASSED: HMC requests for managed system successful.

PASSED: HMC requests for lpar successful.

PASSED: HMC is at a supported level.

PASSED: HMC token is valid.

PASSED: A virtual slot is available.

PASSED: All VIOS are at supported level.

PASSED: All storage adapter mappings are visible on VIOS.

PASSED: All disk reserve policies set to NoReserve or PersistentStorage.

PASSED: RSCT daemons are active.

PASSED: no Kerberos configuration.

PASSED: lpar is not remote restart capable.

PASSED: the simplified remote restart configuration is supported.

PASSED: no virtual log device configured.

PASSED: lpar is using dedicated memory.

PASSED: the disk configuration is supported.

PASSED: no Generic Routing Encapsulation (GRE) tunnel configured.

PASSED: Firmware level is supported.

PASSED: vNIC resources available.

PASSED: Consolidated system trace buffers size is within the limit of 64 MB.

PASSED: SMT number is valid.

PASSED: No process attached to vty0.

PASSED: Audit is not enabled in stream mode.

PASSED: No exclusive rsets (sysxrset) found.

PASSED: Managed System state is operating.

INFO: Any system dumps present in the current dump logical volumes will not be available after live update is complete.

Then I performed another preview, this time specifying the location of the updates to be installed.

root@AIXmig / # geninstall -k -p -d /tmp/cg/lku_ptfs update_all

*******************************************************************************

installp PREVIEW: installation will not actually occur.

*******************************************************************************

+-----------------------------------------------------------------------------+

Pre-installation Verification...

+-----------------------------------------------------------------------------+

Verifying selections...done

Verifying requisites...done

Results...

SUCCESSES

---------

Filesets listed in this section passed pre-installation verification

and will be installed.

Selected Filesets

-----------------

bos.liveupdate.rte 7.2.1.1 # Live Update Runtime

bos.mp64 7.2.1.1 # Base Operating System 64-bit...

bos.net.tcp.ftp 7.2.1.1 # TCP/IP ftp Client Application

bos.rte.install 7.2.1.1 # LPP Install Commands

bos.rte.libcfg 7.2.1.1 # libcfg Library

bos.sysmgt.hmc 7.2.1.1 # HMC Client Runtime

devices.vdevice.IBM.vfc-client.rte 7.2.1.1 # Virtual Fibre Channel Client...

<< End of Success Section >>

+-----------------------------------------------------------------------------+

BUILDDATE Verification ...

+-----------------------------------------------------------------------------+

Verifying build dates...done

FILESET STATISTICS

------------------

7 Selected to be installed, of which:

7 Passed pre-installation verification

----

7 Total to be installed

RESOURCES

---------

Estimated system resource requirements for filesets being installed:

(All sizes are in 512-byte blocks)

Filesystem Needed Space Free Space

/usr 496 2352168

/tmp 43952 2158040

----- -------- ------

TOTAL: 44448 4510208

NOTE: "Needed Space" values are calculated from data available prior

to installation. These are the estimated resources required for the

entire operation. Further resource checks will be made during

installation to verify that these initial estimates are sufficient.

...

******************************************************************************

End of installp PREVIEW. No apply operation has actually occurred.

******************************************************************************

Validating live update input data.

Computing the estimated time for the live update operation:

-------------------------------------------------------

LPAR: AIXmig

Blackout_time(s): 18

Global_time(s): 701

Checking mirror vg device size:

------------------------------------------

Required device size: 7744 MB

Given device size: 10239 MB

PASSED: device size is sufficient.

...

PASSED: Firmware level is supported.

PASSED: vNIC resources available.

PASSED: Consolidated system trace buffers size is within the limit of 64 MB.

PASSED: SMT number is valid.

PASSED: No process attached to vty0.

PASSED: Audit is not enabled in stream mode.

PASSED: No exclusive rsets (sysxrset) found.

PASSED: Managed System state is operating.

INFO: Any system dumps present in the current dump logical volumes will not be available after live update is complete.

root@AIXmig / #

I took note of the current partition ID, as this would change after the Live Update was completed.

root@AIXmig / # uname -L

20 AIXmig

I started the Live Update process.

root@AIXmig / # geninstall -k -d /tmp/cg/lku_ptfs update_all

+-----------------------------------------------------------------------------+

Pre-installation Verification...

+-----------------------------------------------------------------------------+

Verifying selections...done

Verifying requisites...done

Results...

SUCCESSES

---------

Filesets listed in this section passed pre-installation verification

and will be installed.

Selected Filesets

-----------------

bos.liveupdate.rte 7.2.1.1 # Live Update Runtime

bos.mp64 7.2.1.1 # Base Operating System 64-bit...

bos.net.tcp.ftp 7.2.1.1 # TCP/IP ftp Client Application

bos.rte.install 7.2.1.1 # LPP Install Commands

bos.rte.libcfg 7.2.1.1 # libcfg Library

bos.sysmgt.hmc 7.2.1.1 # HMC Client Runtime

devices.vdevice.IBM.vfc-client.rte 7.2.1.1 # Virtual Fibre Channel Client...

<< End of Success Section >>

+-----------------------------------------------------------------------------+

BUILDDATE Verification ...

+-----------------------------------------------------------------------------+

Verifying build dates...done

FILESET STATISTICS

------------------

7 Selected to be installed, of which:

7 Passed pre-installation verification

----

7 Total to be installed

0503-409 installp: bosboot verification starting...

installp: bosboot verification completed.

+-----------------------------------------------------------------------------+

Installing Software...

+-----------------------------------------------------------------------------+

installp: APPLYING software for:

devices.vdevice.IBM.vfc-client.rte 7.2.1.1

. . . . . << Copyright notice for devices.vdevice.IBM.vfc-client >> . . . . . . .

Licensed Materials - Property of IBM

5765CD200

US Government Users Restricted Rights - Use, duplication or disclosure

restricted by GSA ADP Schedule Contract with IBM Corp.

. . . . . << End of copyright notice for devices.vdevice.IBM.vfc-client >>. . . .

Filesets processed: 1 of 7 (Total time: 4 secs).

installp: APPLYING software for:

bos.sysmgt.hmc 7.2.1.1

...

installp: bosboot process completed.

+-----------------------------------------------------------------------------+

Summaries:

+-----------------------------------------------------------------------------+

Installation Summary

--------------------

Name Level Part Event Result

-------------------------------------------------------------------------------

devices.vdevice.IBM.vfc-cli 7.2.1.1 USR APPLY SUCCESS

bos.sysmgt.hmc 7.2.1.1 USR APPLY SUCCESS

bos.sysmgt.hmc 7.2.1.1 ROOT APPLY SUCCESS

bos.rte.libcfg 7.2.1.1 USR APPLY SUCCESS

bos.rte.libcfg 7.2.1.1 ROOT APPLY SUCCESS

bos.rte.install 7.2.1.1 USR APPLY SUCCESS

bos.rte.install 7.2.1.1 ROOT APPLY SUCCESS

bos.net.tcp.ftp 7.2.1.1 USR APPLY SUCCESS

bos.net.tcp.ftp 7.2.1.1 ROOT APPLY SUCCESS

bos.mp64 7.2.1.1 USR APPLY SUCCESS

bos.mp64 7.2.1.1 ROOT APPLY SUCCESS

bos.liveupdate.rte 7.2.1.1 USR APPLY SUCCESS

bos.liveupdate.rte 7.2.1.1 ROOT APPLY SUCCESS

bos.rte.libcfg 7.2.1.1 USR COMMIT SUCCESS

bos.rte.libcfg 7.2.1.1 ROOT COMMIT SUCCESS

bos.rte.install 7.2.1.1 USR COMMIT SUCCESS

bos.rte.install 7.2.1.1 ROOT COMMIT SUCCESS

bos.sysmgt.hmc 7.2.1.1 USR COMMIT SUCCESS

bos.sysmgt.hmc 7.2.1.1 ROOT COMMIT SUCCESS

devices.vdevice.IBM.vfc-cli 7.2.1.1 USR COMMIT SUCCESS

bos.mp64 7.2.1.1 USR COMMIT SUCCESS

bos.mp64 7.2.1.1 ROOT COMMIT SUCCESS

bos.liveupdate.rte 7.2.1.1 USR COMMIT SUCCESS

bos.liveupdate.rte 7.2.1.1 ROOT COMMIT SUCCESS

bos.net.tcp.ftp 7.2.1.1 USR COMMIT SUCCESS

bos.net.tcp.ftp 7.2.1.1 ROOT COMMIT SUCCESS

Validating live update input data.

Computing the estimated time for the live update operation:

-------------------------------------------------------

LPAR: AIXmig

Blackout_time(s): 17

Global_time(s): 759

...

PASSED: Managed System state is operating.

INFO: Any system dumps present in the current dump logical volumes will not be available after live update is complete.

Non-interruptable live update operation begins in 10 seconds.

Broadcast message from root@AIXmig (pts/0) at 10:38:35 ...

Live AIX update in progress.

Initializing live update on original LPAR.

Validating original LPAR environment.

Beginning live update operation on original LPAR.

Requesting resources required for live update.

........................

Notifying applications of impending live update.

Creating rootvg for boot of surrogate.

....................................

Starting the surrogate LPAR.

....................................

Creating mirror of original LPAR's rootvg.

........................................................

Moving workload to surrogate LPAR.

................

Blackout Time started.

Blackout Time end.

Workload is running on surrogate LPAR.

................................

Shutting down the Original LPAR.

............................

The live update operation succeeded.

Broadcast message from root@AIXmig (pts/0) at 10:51:35 ...

Live AIX update completed.

The process completed successfully and all of the filesets were updated to 7.2.1.1.

root@AIXmig / # lslpp -l bos.liveupdate.rte bos.mp64 bos.net.tcp.ftp bos.rte.install bos.rte.libcfg bos.sysmgt.hmc devices.vdevice.IBM.vfc-client.rte

Fileset Level State Description

----------------------------------------------------------------------------

Path: /usr/lib/objrepos

bos.liveupdate.rte 7.2.1.1 COMMITTED Live Update Runtime

bos.mp64 7.2.1.1 COMMITTED Base Operating System 64-bit

Multiprocessor Runtime

bos.net.tcp.ftp 7.2.1.1 COMMITTED TCP/IP ftp Client Application

bos.rte.install 7.2.1.1 COMMITTED LPP Install Commands

bos.rte.libcfg 7.2.1.1 COMMITTED libcfg Library

bos.sysmgt.hmc 7.2.1.1 COMMITTED HMC Client Runtime

devices.vdevice.IBM.vfc-client.rte

7.2.1.1 COMMITTED Virtual Fibre Channel Client

I used the genld command to list the processes that had an old version of a library loaded.

root@AIXmig / # genld -lu

Proc_pid: 4915634 Proc_name: hostmibd

9000000004f5ce0 16c90 /usr/lib/libcfg.a[shr_64.o]

Proc_pid: 5177766 Proc_name: nimsh