AIX 7.3 TL2, Cisco Discovery Protocol (CDP) and SR-IOV

Starting with AIX 7.3 TL2, AIX systems that are using native SR-IOV adapters (logical ports) can now display information for Cisco network devices (switches) that they are directly connected to.

CDP is defined as “…a network discovery tool, which assists network administrators and engineers in identifying neighboring Cisco devices, particularly those running lower-layer, transparent protocols. All Cisco devices transmit CDP packets periodically (default time interval value is 60 seconds though this is adjustable)”

So, if your AIX LPAR is directly connected to a Cisco network switch, through a physical Ethernet adapter or (now) through a native SR-IOV adapter (a VF adapter), then you can display information about the Cisco switch that the adapter is connected to.

This is handy for troubleshooting network issues, as it allows an AIX admin to identify the network switch that an AIX server is connected to, on their network.

Note: The CDP support has been around, on AIX and VIOS, for several years but it has never supported SR-IOV VF adapters. AIX 7.3 TL2 allows the VF adapter to work with CDP. Also note that this approach is not supported with Virtual Ethernet Adapters (VEAs).

Here's what I found in my testing with an AIX LPAR running 7.3 TL2 SP1.

# oslevel -s

7300-02-01-2346

An SR-IOV logical port is assigned to this LPAR and is connected to my network.

# lsdev -Cc adapter | grep VF

ent1 Available 07-00 PCIe3 100/1000 Base-TX 4-port Converged Network Adapter VF (df1028e214103c04)

# ifconfig en1

en1: flags=1e084863,18c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN>

inet 10.8.12.11 netmask 0xffffff00 broadcast 10.8.12.255

tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0

We start the cdpd daemon on AIX.

# startsrc -s cdpd

0513-059 The cdpd Subsystem has been started. Subsystem PID is 20054430.

We add the SR-IOV VF network interface (en1) to the cdpd monitoring framework so that it can receive data packets and messages from the physically connected Cisco switch, using CDP.

# cdpctl add en1

cdpctl: 0812-005 successfully added port en1

We then display the list of network interfaces that are configured to receive incoming data using CDP on this AIX LPAR.

# cdpctl show portlist

1.en1

Finally, we display the CDP information (such as the network switch and router capabilities) that are being received on the configured CDP-capable network interfaces, i.e. en1.

# cdpctl show port en1

Waiting for CDP advertise (default 60 seconds)......

Device ID : route1-n1.local(AGE19190TKY)

Address : 10.10.10.2

Port ID : Ethernet3/43

Capabilities

: Router Level 3

: Level 2 Switch

Cisco switch OS Version : Cisco Nexus Operating System (NX-OS) Software, Version 9.3(9)

Platform : N9K-C9508

Native VLAN ID : 32

Trusted Bitmap : N/A

AVVID untrusted ports : N/A

Duplex : Full

MTU : 576

System Name : route1-n1

System Object ID : N/A

Management Addresses : 10.1.2.10

CDP record received on dev en1 also stored at /tmp/cdp_record_en1

The information is also saved to a file in /tmp called cdp_record_enX, where X is the interface number.

# cat /tmp/cdp_record_en1

CDP Advertise captured on Tue Oct 24 21:09:20 2023

Device ID : route1-n1.local(AGE19190TKY)

Address : 10.10.10.2

Port ID : Ethernet3/43

Capabilities

: Router Level 3

: Level 2 Switch

Cisco switch OS Version : Cisco Nexus Operating System (NX-OS) Software, Version 9.3(9)

Platform : N9K-C9508

Native VLAN ID : 32

Trusted Bitmap : N/A

AVVID untrusted ports : N/A

Duplex : Full

MTU : 576

System Name : route1-n1

System Object ID : N/A

Management Addresses : 10.1.2.10

Please refer to the AIX 7.3 command reference pages for more information:

cdpd Daemon

https://www.ibm.com/docs/en/aix/7.3?topic=c-cdpd-daemon

cdpctl Command

https://www.ibm.com/docs/en/aix/7.3?topic=c-cdpctl-command

Also, please check out this awesome post by IBM Champion, Andrey Klyachkin, on AIX and CDP:

Cisco Discovery Protocol and IBM AIX

https://www.linkedin.com/pulse/cisco-discovery-protocol-ibm-aix-andrey-klyachkin/

The cdpd daemon and the cdpctl command were first introduced in AIX 7.2 TL5, in 2020.

From the AIX 7.2 What’s New page: https://www.ibm.com/docs/en/aix/7.2?topic=whats-new

“Added information about the cdpd daemon and cdpctl command that control the Cisco Discovery Protocol (CDP) incoming data packets or messages from physically connected Cisco devices”

From the AIX 7.2 TL5 release notes: https://www.ibm.com/docs/en/aix/7.2?topic=notes-aix-725-release

“bos.net.tcp.cdp

This fileset contains new commands cdpd and cdpctl that are used to receive Cisco Discovery Protocol (CDP) messages for discovering other Cisco devices that are directly connected to the AIX logical partition. The cdpd command runs as a daemon to receive CDP packets that are sent by the resource controller. The cdpctl command controls the CDP daemon by configuring the network interface that is physically connected to the Cisco device”

Note that AIX 7.3 TL2 introduced new Link Layer Discovery Protocol (LLDP) enhancements to report additional details for EtherChannel and underlying adapters when the AIX LLDP daemon is running. Refer to this link for the announcement: https://www.ibm.com/docs/en/announcements/aix-enhancements-aix-73-std-ed-tl-2-aix-7-enterprise-ed-110-private-cloud-edition-110-private-cloud-edition-aix-110-powersc-22?region=US

Initially I considered using lldpd to obtain similar information to cdpd, but there’s no “lldpsvc” attribute for a SR-IOV VF adapter. Here are some great links about LLDP:

https://powercampus.de/en/article/aix/virtual-i-o-server-configuring-lldp/

https://www.linkedin.com/pulse/cisco-discovery-protocol-ibm-aix-andrey-klyachkin

The cdpd changes were also made available with AIX 7.2 TL5 SP7, which also became available in November 2023. The related AIX 7.2 APAR is IJ48479, “cdpd does not add multicast address during port add operation”, “Port information was not available in case of SRIOV VF interfaces. Interface added/enabled for multicast address to handle SRIOV VF interface”. The related AIX 7.3 APAR is IJ48390.

The CDP method is a great alternative to using the classic tcpdump approach (which, by the way, doesn’t work with SR-IOV logical ports). For example:

# tcpdump -nn -v -i en0 -s 1500 -c 1 'ether[20:2] == 0x2000'

The tcpdump technique still works well with physical adapters on AIX and is discussed in detail, here:

Using tcpdump to discover network information

https://unixhealthcheck.com/blog?id=358

P.S. I tested this (cdpd/cdpctl) against one of the Shared Ethernet Adapters (SEA) on my VIOS and it also worked as advertised. However, I note that this was always working with an SEA consisting of an underlying physical Ethernet adapter; even on previous releases of VIOS code.

$ ioslevel

3.1.3.21

$ oem_setup_env

# lsdev -Cc adapter | grep ent5

ent5 Available Shared Ethernet Adapter

# ifconfig en5

en5: flags=1e084863,2000c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN>

inet 10.4.7.201 netmask 0xfffff000 broadcast 10.4.7.255

# startsrc -s cdpd

0513-059 The cdpd Subsystem has been started. Subsystem PID is 30146884.

# cdpctl add en5

cdpctl: 0812-005 successfully added port en5

# cdpctl show port en5

Waiting for CDP advertise (default 60 seconds)......

Device ID : labrouter.local(AHA09360044)

Address : 10.1.7.100

Port ID : mgmt0

Capabilities

: Router Level 3

: Level 2 Switch

Cisco switch OS Version : Cisco Nexus Operating System (NX-OS) Software, Version 6.2(13a)

Platform : DS-C9513

Duplex : Full

System Name : labrouter

System Object ID : N/A

Management Addresses : 10.1.7.100

CDP record received on dev en5 also stored at /tmp/cdp_record_en5