Skip to content

Commit

Permalink
Eliminate use of term "slave" in bond, LACP, and bundle contexts.
Browse files Browse the repository at this point in the history 
The new term is "member".

Most of these changes should not change user-visible behavior.  One
place where they do is in "ovs-ofctl dump-flows", which will now output
"members:..." inside "bundle" actions instead of "slaves:...".  I don't
expect this to cause real problems in most systems.  The old syntax
is still supported on input for backward compatibility.

Signed-off-by: Ben Pfaff <[email protected]>
Acked-by: Alin Gabriel Serdean <[email protected]>
  • Loading branch information
@blp
blp committed Oct 21, 2020
1 parent f51cf36 commit 91fc374
Show file tree
Hide file tree
Showing 45 changed files with 1,393 additions and 1,356 deletions.
167 changes: 84 additions & 83 deletions Documentation/topics/bonding.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,22 +25,22 @@
Bonding
=======

Bonding allows two or more interfaces (the "slaves") to share network traffic.
Bonding allows two or more interfaces, its "members", to share network traffic.
From a high-level point of view, bonded interfaces act like a single port, but
they have the bandwidth of multiple network devices, e.g. two 1 GB physical
interfaces act like a single 2 GB interface. Bonds also increase robustness:
the bonded port does not go down as long as at least one of its slaves is up.
the bonded port does not go down as long as at least one of its members is up.

In vswitchd, a bond always has at least two slaves (and may have more). If a
configuration error, etc. would cause a bond to have only one slave, the port
In vswitchd, a bond always has at least two members (and may have more). If a
configuration error, etc. would cause a bond to have only one member, the port
becomes an ordinary port, not a bonded port, and none of the special features
of bonded ports described in this section apply.

There are many forms of bonding of which ovs-vswitchd implements only a few.
The most complex bond ovs-vswitchd implements is called "source load balancing"
or SLB bonding. SLB bonding divides traffic among the slaves based on the
Ethernet source address. This is useful only if the traffic over the bond has
multiple Ethernet source addresses, for example if network traffic from
or SLB bonding. SLB bonding divides traffic among the members based on
the Ethernet source address. This is useful only if the traffic over the bond
has multiple Ethernet source addresses, for example if network traffic from
multiple VMs are multiplexed over the bond.

.. note::
Expand All @@ -50,89 +50,90 @@ multiple VMs are multiplexed over the bond.
specified.


Enabling and Disabling Slaves
-----------------------------
Enabling and Disabling Members
------------------------------

When a bond is created, a slave is initially enabled or disabled based on
whether carrier is detected on the NIC (see ``iface_create()``). After that, a
slave is disabled if its carrier goes down for a period of time longer than the
downdelay, and it is enabled if carrier comes up for longer than the updelay
(see ``bond_link_status_update()``). There is one exception where the updelay
is skipped: if no slaves at all are currently enabled, then the first slave on
which carrier comes up is enabled immediately.
When a bond is created, a member is initially enabled or disabled based
on whether carrier is detected on the NIC (see ``iface_create()``). After
that, a member is disabled if its carrier goes down for a period of time
longer than the downdelay, and it is enabled if carrier comes up for longer
than the updelay (see ``bond_link_status_update()``). There is one exception
where the updelay is skipped: if no members at all are currently
enabled, then the first member on which carrier comes up is enabled
immediately.

The updelay should be set to a time longer than the STP forwarding delay of the
physical switch to which the bond port is connected (if STP is enabled on that
switch). Otherwise, the slave will be enabled, and load may be shifted to it,
before the physical switch starts forwarding packets on that port, which can
cause some data to be "blackholed" for a time. The exception for a single
enabled slave does not cause any problem in this regard because when no slaves
are enabled all output packets are blackholed anyway.

When a slave becomes disabled, the vswitch immediately chooses a new output
port for traffic that was destined for that slave (see
``bond_enable_slave()``). It also sends a "gratuitous learning packet",
specifically a RARP, on the bond port (on the newly chosen slave) for each MAC
address that the vswitch has learned on a port other than the bond (see
``bundle_send_learning_packets()``), to teach the physical switch that the new
slave should be used in place of the one that is now disabled. (This behavior
probably makes sense only for a vswitch that has only one port (the bond)
connected to a physical switch; vswitchd should probably provide a way to
disable or configure it in other scenarios.)
switch). Otherwise, the member will be enabled, and load may be shifted
to it, before the physical switch starts forwarding packets on that port, which
can cause some data to be dropped for a time. The exception for a single
enabled member does not cause any problem in this regard because when no
members are enabled all output packets are dropped anyway.

When a member becomes disabled, the vswitch immediately chooses a new
output port for traffic that was destined for that member (see
``bond_enable_member()``). It also sends a "gratuitous learning packet",
specifically a RARP, on the bond port (on the newly chosen member) for
each MAC address that the vswitch has learned on a port other than the bond
(see ``bundle_send_learning_packets()``), to teach the physical switch that the
new member should be used in place of the one that is now disabled.
(This behavior probably makes sense only for a vswitch that has only one port
(the bond) connected to a physical switch; vswitchd should probably provide a
way to disable or configure it in other scenarios.)

Bond Packet Input
-----------------

Bonding accepts unicast packets on any bond slave. This can occasionally cause
packet duplication for the first few packets sent to a given MAC, if the
Bonding accepts unicast packets on any member. This can occasionally
cause packet duplication for the first few packets sent to a given MAC, if the
physical switch attached to the bond is flooding packets to that MAC because it
has not yet learned the correct slave for that MAC.
has not yet learned the correct member for that MAC.

Bonding only accepts multicast (and broadcast) packets on a single bond slave
(the "active slave") at any given time. Multicast packets received on other
slaves are dropped. Otherwise, every multicast packet would be duplicated,
once for every bond slave, because the physical switch attached to the bond
will flood those packets.
Bonding only accepts multicast (and broadcast) packets on a single bond
member (the "active member") at any given time. Multicast
packets received on other members are dropped. Otherwise, every
multicast packet would be duplicated, once for every bond member,
because the physical switch attached to the bond will flood those packets.

Bonding also drops received packets when the vswitch has learned that the
packet's MAC is on a port other than the bond port itself. This is because it
is likely that the vswitch itself sent the packet out the bond port on a
different slave and is now receiving the packet back. This occurs when the
packet is multicast or the physical switch has not yet learned the MAC and is
flooding it. However, the vswitch makes an exception to this rule for
different member and is now receiving the packet back. This occurs when
the packet is multicast or the physical switch has not yet learned the MAC and
is flooding it. However, the vswitch makes an exception to this rule for
broadcast ARP replies, which indicate that the MAC has moved to another switch,
probably due to VM migration. (ARP replies are normally unicast, so this
exception does not match normal ARP replies. It will match the learning
packets sent on bond fail-over.)

The active slave is simply the first slave to be enabled after the bond is
created (see ``bond_choose_active_slave()``). If the active slave is disabled,
then a new active slave is chosen among the slaves that remain active.
Currently due to the way that configuration works, this tends to be the
remaining slave whose interface name is first alphabetically, but this is by no
means guaranteed.
The active member is simply the first member to be enabled after
the bond is created (see ``bond_choose_active_member()``). If the active
member is disabled, then a new active member is chosen among the
members that remain active. Currently due to the way that configuration
works, this tends to be the remaining member whose interface name is
first alphabetically, but this is by no means guaranteed.

Bond Packet Output
------------------

When a packet is sent out a bond port, the bond slave actually used is selected
based on the packet's source MAC and VLAN tag (see ``bond_choose_output_slave()``).
In particular, the source MAC and VLAN tag are hashed into one of 256 values,
and that value is looked up in a hash table (the "bond hash") kept in the
``bond_hash`` member of struct port. The hash table entry identifies a bond
slave. If no bond slave has yet been chosen for that hash table entry,
vswitchd chooses one arbitrarily.

Every 10 seconds, vswitchd rebalances the bond slaves (see
``bond_rebalance()``). To rebalance, vswitchd examines the statistics for
the number of bytes transmitted by each slave over approximately the past
When a packet is sent out a bond port, the bond member actually used is
selected based on the packet's source MAC and VLAN tag (see
``bond_choose_output_member()``). In particular, the source MAC and VLAN tag
are hashed into one of 256 values, and that value is looked up in a hash table
(the "bond hash") kept in the ``bond_hash`` member of struct port. The hash
table entry identifies a bond member. If no bond member has yet been chosen
for that hash table entry, vswitchd chooses one arbitrarily.

Every 10 seconds, vswitchd rebalances the bond members (see
``bond_rebalance()``). To rebalance, vswitchd examines the statistics for the
number of bytes transmitted by each member over approximately the past
minute, with data sent more recently weighted more heavily than data sent less
recently. It considers each of the slaves in order from most-loaded to
least-loaded. If highly loaded slave H is significantly more heavily loaded
than the least-loaded slave L, and slave H carries at least two hashes, then
vswitchd shifts one of H's hashes to L. However, vswitchd will only shift a
hash from H to L if it will decrease the ratio of the load between H and L by
at least 0.1.
recently. It considers each of the members in order from most-loaded to
least-loaded. If highly loaded member H is significantly more heavily
loaded than the least-loaded member L, and member H carries at
least two hashes, then vswitchd shifts one of H's hashes to L. However,
vswitchd will only shift a hash from H to L if it will decrease the ratio of
the load between H and L by at least 0.1.

Currently, "significantly more loaded" means that H must carry at least 1 Mbps
more traffic, and that traffic must be at least 3% greater than L's.
Expand Down Expand Up @@ -166,11 +167,11 @@ behavior on Open vSwitch.
Active Backup Bonding
~~~~~~~~~~~~~~~~~~~~~

Active Backup bonds send all traffic out one "active" slave until that slave
becomes unavailable. Since they are significantly less complicated than SLB
bonds, they are preferred when LACP is not an option. Additionally, they are
the only bond mode which supports attaching each slave to a different upstream
switch.
Active Backup bonds send all traffic out one "active" member until that
member becomes unavailable. Since they are significantly less
complicated than SLB bonds, they are preferred when LACP is not an option.
Additionally, they are the only bond mode which supports attaching each
member to a different upstream switch.

SLB Bonding
~~~~~~~~~~~
Expand All @@ -195,15 +196,15 @@ SLB bonding has the following complications:
This would cause packet duplication if not handled specially.

Open vSwitch avoids packet duplication by accepting multicast and broadcast
packets on only the active slave, and dropping multicast and broadcast
packets on all other slaves.
packets on only the active member, and dropping multicast and
broadcast packets on all other members.

2. When Open vSwitch forwards a multicast or broadcast packet to a link in the
SLB bond other than the active slave, the remote switch will forward it to
all of the other links in the SLB bond, including the active slave. Without
special handling, this would mean that Open vSwitch would forward a second
copy of the packet to each switch port (other than the bond), including the
port that originated the packet.
SLB bond other than the active member, the remote switch will forward
it to all of the other links in the SLB bond, including the active
member. Without special handling, this would mean that Open vSwitch
would forward a second copy of the packet to each switch port (other than
the bond), including the port that originated the packet.

Open vSwitch deals with this case by dropping packets received on any SLB
bonded link that have a source MAC+VLAN that has been learned on any other
Expand All @@ -226,11 +227,11 @@ SLB bonding has the following complications:
4. Suppose that a MAC+VLAN moves from an SLB bond to another port (e.g. when a
VM is migrated from a different hypervisor to this one), that the MAC+VLAN
emits a gratuitous ARP, and that Open vSwitch forwards that gratuitous ARP
to a link in the SLB bond other than the active slave. The remote switch
will forward the gratuitous ARP to all of the other links in the SLB bond,
including the active slave. Without additional special handling, this would
mean that Open vSwitch would learn that the MAC+VLAN was located on the SLB
bond, as a consequence of rule #3.
to a link in the SLB bond other than the active member. The remote
switch will forward the gratuitous ARP to all of the other links in the SLB
bond, including the active member. Without additional special
handling, this would mean that Open vSwitch would learn that the MAC+VLAN
was located on the SLB bond, as a consequence of rule #3.

Open vSwitch avoids this problem by "locking" the MAC learning table entry
for a MAC+VLAN from which a gratuitous ARP was received from a non-SLB bond
Expand Down
4 changes: 2 additions & 2 deletions Documentation/topics/porting.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ concordance, indexed by the area of the source tree:
datapath/ vport ---
vswitchd/ iface port
ofproto/ port bundle
ofproto/bond.c slave bond
lib/lacp.c slave lacp
ofproto/bond.c member bond
lib/lacp.c member lacp
lib/netdev.c netdev ---
database Interface Port

Expand Down
4 changes: 3 additions & 1 deletion NEWS
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ Post-v2.14.0
- Terminology:
* The terms "master" and "slave" have been replaced by "primary" and
"secondary", respectively, for OpenFlow connection roles.
* The term "slave" has been replaced by "member", for bonds, LACP, and
OpenFlow bundle actions.


v2.14.0 - 17 Aug 2020
Expand Down Expand Up @@ -833,7 +835,7 @@ v2.4.0 - 20 Aug 2015
The implementation has been tested successfully against the Ixia Automated
Network Validation Library (ANVL).
- Stats are no longer updated on fake bond interface.
- Keep active bond slave selection across OVS restart.
- Keep active bond interface selection across OVS restart.
- A simple wrapper script, 'ovs-docker', to integrate OVS with Docker
containers. If and when there is a native integration of Open vSwitch
with Docker, the wrapper script will be retired.
Expand Down
2 changes: 1 addition & 1 deletion debian/changelog
View file
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,7 @@ openvswitch (2.4.0-1) unstable; urgency=low
The implementation has been tested successfully against the Ixia Automated
Network Validation Library (ANVL).
- Stats are no longer updated on fake bond interface.
- Keep active bond slave selection across OVS restart.
- Keep active bond member selection across OVS restart.
- A simple wrapper script, 'ovs-docker', to integrate OVS with Docker
containers. If and when there is a native integration of Open vSwitch
with Docker, the wrapper script will be retired.
Expand Down
4 changes: 2 additions & 2 deletions debian/ifupdown.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,9 @@ if [ "${MODE}" = "start" ]; then
${OVS_EXTRA+-- $OVS_EXTRA}

ip link set "${IFACE}" up
for slave in ${IF_OVS_BONDS}
for member in ${IF_OVS_BONDS}
do
ip link set "${slave}" up
ip link set "${member}" up
done
;;
OVSPatchPort)
Expand Down
28 changes: 15 additions & 13 deletions include/openvswitch/ofp-actions.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ struct vl_mff_map;
OFPACT(CONTROLLER, ofpact_controller, userdata, "controller") \
OFPACT(ENQUEUE, ofpact_enqueue, ofpact, "enqueue") \
OFPACT(OUTPUT_REG, ofpact_output_reg, ofpact, "output_reg") \
OFPACT(BUNDLE, ofpact_bundle, slaves, "bundle") \
OFPACT(BUNDLE, ofpact_bundle, members, "bundle") \
\
/* Header changes. */ \
OFPACT(SET_FIELD, ofpact_set_field, ofpact, "set_field") \
Expand Down Expand Up @@ -364,24 +364,24 @@ struct ofpact_output_trunc {
);
};

/* Bundle slave choice algorithm to apply.
/* Bundle member choice algorithm to apply.
*
* In the descriptions below, 'slaves' is the list of possible slaves in the
* In the descriptions below, 'members' is the list of possible members in the
* order they appear in the OpenFlow action. */
enum nx_bd_algorithm {
/* Chooses the first live slave listed in the bundle.
/* Chooses the first live member listed in the bundle.
*
* O(n_slaves) performance. */
* O(n_members) performance. */
NX_BD_ALG_ACTIVE_BACKUP = 0,

/* Highest Random Weight.
*
* for i in [0,n_slaves):
* for i in [0,n_members):
* weights[i] = hash(flow, i)
* slave = { slaves[i] such that weights[i] >= weights[j] for all j != i }
* member = { members[i] such that weights[i] >= weights[j] for all j != i }
*
* Redistributes 1/n_slaves of traffic when a slave's liveness changes.
* O(n_slaves) performance.
* Redistributes 1/n_members of traffic when a member's liveness changes.
* O(n_members) performance.
*
* Uses the 'fields' and 'basis' parameters. */
NX_BD_ALG_HRW = 1
Expand All @@ -394,7 +394,7 @@ struct ofpact_bundle {
OFPACT_PADDED_MEMBERS(
struct ofpact ofpact;

/* Slave choice algorithm to apply to hash value. */
/* Member choice algorithm to apply to hash value. */
enum nx_bd_algorithm algorithm;

/* What fields to hash and how. */
Expand All @@ -403,10 +403,12 @@ struct ofpact_bundle {

struct mf_subfield dst;

/* Slaves for output. */
unsigned int n_slaves;
bool compat_syntax;

/* Members for output. */
unsigned int n_members;
);
ofp_port_t slaves[];
ofp_port_t members[];
};

/* OFPACT_SET_VLAN_VID.
Expand Down
Loading

0 comments on commit 91fc374

Please sign in to comment.