The benefits of AIX 7.2, XIVE, XGZIP and POWER9 systems

 

With AIX 7.2 TL4 (or higher) and POWER9 firmware 940 (or later), administrators can make an explicit decision to run in POWER9 compatibility mode if they want to take advantage of new features like XIVE and NX GZip.

 

 

Specifically enabling POWER9 mode will allow LPARs to leverage the advanced capabilities of the POWER9 processors and take advantage of new AIX features. You can enable your LPAR for POWER9 mode by changing the “Processor compatibility mode” in the partition profile, and restarting the LPAR.

 

A screenshot of a cell phone

Description automatically generated

 

Enabling AIX 7.2 for POWER9 GZIP Compression

Once your LPAR is booted in POWER9 mode (with FW940), you can take advantage of the P9 NX GZIP compression engine. AIX can then leverage POWER9 GZIP Compression with xgzip. AIX 7.2 TL4 adds a new zlibNX library and xgzip command that exploits the POWER9 HW GZIP compression engine for enhanced performance.

 

You need to install the zlibNX.rte and xgzip.rte filesets first. The zlibNX.rte fileset can be found on the AIX 7.2 TL4 Expansion Pack* and xgzip.rte can be downloaded from the AIX Web Download Pack (or here: https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=aixbp).

 

*UPDATE: Starting with IBM AIX 7.2 Technology Level 5 Service Pack 2, the zlibNX installation package is available on the AIX base media and not on the Expansion Pack.

 

# loopmount -i ESD-AIX-v7.2BaseInstFlash7200-05-02-2114.iso -m /mnt -o "-V cdrfs -o ro"

# cd /mnt

# find . -name zlibNX -ls

6065418240  895 -rw-r--r--  2 4000      4000        916480 Apr  1 05:14 ./installp/ppc/zlibNX

6065418240  895 -rw-r--r--  2 4000      4000        916480 Apr  1 05:14 ./usr/sys/inst.images/zlibNX

 

# lslpp -l zlibNX.rte

  Fileset                      Level  State      Description

  ----------------------------------------------------------------------------

Path: /usr/lib/objrepos

  zlibNX.rte                 7.2.4.0  COMMITTED  NX accelerated zlib

                                                 compression library

# lslpp -l xgzip.rte

  Fileset                      Level  State      Description

  ----------------------------------------------------------------------------

Path: /usr/lib/objrepos

  xgzip.rte                 4.0.20.0  COMMITTED  A command utility to exploit

                                                 NX accelerated zlib

                                                 compression library

 

You can use kdb to verify your LPAR is running in the correct mode, such that the NX GZip engine is available to AIX.

 

# echo "dw nx_available" | kdb -script

read vscsi_scsi_ptrs OK, ptr = 0xF100091590128E90

(0)> dw nx_available

Executing dw command

nx_available+000000: 00000001 00000000 00000000 00000007  ................

 

My tests with xgzip showed a vast improvement in compression times, when compared with gzip.

 

# lsconf

System Model: IBM,9040-MR9

Processor Type: PowerPC_POWER9

Processor Implementation Mode: POWER 9

Processor Version: PV_9_Compat

Number Of Processors: 2

Processor Clock Speed: 3300 MHz

CPU Type: 64-bit

Kernel Type: 64-bit

Memory Size: 172032 MB

Good Memory Size: 172032 MB

Platform Firmware level: VM940_037

Firmware Version: IBM,FW940.10 (VM940_037)

 

# oslevel -s

7200-04-01-1939

 

# time gzip myfile

 

real    0m8.47s

user    0m2.57s

sys     0m0.18s

 

# time xgzip myfile

 

real    0m0.98s

user    0m0.07s

sys     0m0.22s

 

So, why xgzip not gzip? Why didn’t IBM just try accelerating gzip instead? Well, the implementation of gzip is not modular such that it can use an alternate zlib. It would require substantial refactoring of gzip. It would be great if the community around gzip revised gzip to use zlib.

 

PLEASE NOTE:  So, why not just change the Default mode to POWER9 (instead of POWER9_Base)? IBM made a conscious decision not to change the underlying mapping for two reasons. First, IBM didn't want to automatically change the behaviour of LPARs just because you installed a new firmware level and second, keeping the meaning of Default (mapped to POWER9_Base) ensures that existing partitions will successfully migrate across all vintages of P9 firmware levels. Meaning, that if you explicitly select POWER9 mode, the partition will no longer be capable of LPM'ing to 910, 920 or 930 firmware levels.

 

Data compression by using the zlibNX library

https://www.ibm.com/support/knowledgecenter/ssw_aix_72/performance/zlibNX.html

 

NOTE: Power “Guru” Gareth Coates (from IBM UK) has also written a good article about the POWER9 NX gzip accelerator: “Using the POWER9™ NX (gzip) accelerator in AIX” https://www.ibm.com/support/pages/node/6243342

 

Enabling AIX 7.2 for POWER9 XIVE

IBM POWER9 systems with FW940 or later support user mode enablement of the External Interrupt Virtualization Engine (XIVE).  Systems running FW940 or later support XIVE. AIX 7.2 TL4 and VIOS 3.1.1 both support XIVE. This user mode enables the management of interrupts to move from the hypervisor to the operating system for improved efficiency. Prior processors distributed interrupts across the entire system. This generated a significant software overhead to route interrupts. The new XIVE hardware routes correctly first time. It eliminates host processor overhead. It directly targets guest Operating Systems by enabling a User level Interrupt.

 

A screenshot of a cell phone

Description automatically generated

 

Operating systems may need to be updated to enable this support. The LPAR must be running in POWER9 compatibility mode to use this feature.  For more information on compatibility modes, see the following two articles in the IBM Knowledge Centre:

 

Processor compatibility mode overview:    https://www.ibm.com/support/knowledgecenter/POWER9/p9hc3/p9hc3_pcm.htm

 

Processor compatibility mode definitions:  https://www.ibm.com/support/knowledgecenter/POWER9/p9hc3/p9hc3_pcmdefs.htm

 

When considering if you should take advantage of XIVE (or not), please keep in mind that, a) you need to be driving a huge I/O load where lots of interrupts are being generated to see much in respect to performance improvement and b) there has been lots of field testing of the previous interrupt model and the XIVE support is all new for FW940 & later.

 

Results with XIVE have been promising. One customer reported that after enabling full P9 mode, SMT8 and XIVE, their SAP BW nightly batch jobs now completed up to 41% faster.

 

Once your systems have been updated to FW940, start your AIX 7.2 TL4 and VIOS 3.1.1 partitions in POWER9 mode. And, don’t forget to enable at the OS level:

 

# oslevel -s

7200-04-02-2016

 

# chdev -l sys0 -a ex_intr_virt=true

# bosboot -aD -l hd5

# sync;sync;reboot -q

# lsattr -El sys0 -a ex_intr_virt

ex_intr_virt true Enable external interrupt virtualization True

 

Use kdb to verify that XIVE is, indeed, active on your system.

 

# echo d opic_type | kdb | grep opic | tail -1 | awk {'print $2'}

0003

 

0002 -> Legacy PowerPC interrupts XIRR

0003 -> XIVE interrupts

 

XIVE: External interrupt virtualization for the cloud infrastructure

Publisher: IBM

https://ieeexplore.ieee.org/document/8383690

 

You can find more information on these new capabilities with AIX 7.2 and POWER9 in the following best practices document published by IBM support:

 

Hints and Tips for Migrating Workload to IBM POWER9 Processor-Based Systems

https://www.ibm.com/downloads/cas/39XWR7YM