Bug#1121032: net: thunderbolt: missing ndo_set_mac_address breaks 802.3ad bonding

[Ian MacDonald <ian@netstatz.com>]

Hi,

Using two Thunderbolt network interfaces as slaves in a bonding device
in mode 802.3ad (LACP) fails because the bonding driver cannot set the
MAC address on the thunderbolt_net interfaces. The same setup works in
mode active-backup.

Hardware: AMD Strix Halo (Framework connect to Sixunited AXB35 USB4 ports)
Kernel:  6.12.57 (also reproduced on 6.16.12 and 6.18~rc6)

Steps to reproduce:
1. Create a bond with mode 802.3ad and add thunderbolt0 and thunderbolt1
   as slaves.
2. Bring up the bond and slaves.
3. Observe that bonding fails to set the slave MAC addresses and logs:

   [   25.922317] bond0: (slave thunderbolt0): The slave device
   specified does not support setting the MAC address
   [   25.922328] bond0: (slave thunderbolt0): Error -95 calling
   set_mac_address
   [   25.980235] bond0: (slave thunderbolt1): The slave device specified
   does not support setting the MAC address
   [   25.980242] bond0: (slave thunderbolt1): Error -95 calling
   set_mac_address

Expected result:
- bond0 and both Thunderbolt interfaces share bond0's MAC address.
- 802.3ad operates normally and the link comes up.

Actual result:
- dev_set_mac_address(thunderboltX, bond0_mac) fails with -EOPNOTSUPP.
- bonding reports that the slave does not support setting the MAC address
  and cannot use the interfaces in 802.3ad mode.

>From reading drivers/net/thunderbolt/main.c:

- thunderbolt_net generates a locally administered MAC from the
  Thunderbolt UUID and sets it with eth_hw_addr_set().
- The net_device_ops for thunderbolt_net currently define:
    .ndo_open
    .ndo_stop
    .ndo_start_xmit
    .ndo_get_stats64
  but do not implement .ndo_set_mac_address.

As a result, dev_set_mac_address() returns -EOPNOTSUPP and bonding treats
the device as not supporting MAC address changes.

A bit of research suggests it should be possible to implement
ndo_set_mac_address using
eth_mac_addr(), and, if appropriate, mark the device with
IFF_LIVE_ADDR_CHANGE so MAC changes while the interface is up are
allowed.   I have a feeling there is a lot more to it;

There is a corresponding downstream Debian bug with additional
hardware details https://bugs.debian.org/1121032

Thanks,
Ian