Merge pull request canonical#10139 from ru-fu/LXD-100-networking-rest…

…ructure

Networking docs restructure

.sphinx/conf.py

@@ -128,4 +128,8 @@
 # Setup redirects (https://documatt.gitlab.io/sphinx-reredirects/usage.html)
 redirects = {
     "index/index": "../index.html",
+    "network-peers/index": "../howto/network_ovn_peers/index.html",
+    "network-acls/index": "../howto/network_acls/index.html",
+    "network-forwards/index": "../howto/network_forwards/index.html",
+    "network-zones/index": "../howto/network_zones/index.html"
 }

doc/configuration.md

@@ -6,11 +6,6 @@ LXD stores the configuration for the following components:
 
 containers
 Instances <instances>
-Networks <networks>
-Network ACLs <network-acls>
-Network Forwards <network-forwards>
-Network Peers <network-peers>
-Network Zones <network-zones>
 Preseed files <preseed>
 profiles
 Projects <projects>

doc/explanation/networks.md

@@ -0,0 +1,117 @@
+# About networking
+
+There are different ways to connect your instances to the Internet. The easiest method is to have LXD create a network bridge during initialization and use this bridge for all instances, but LXD supports many different and advanced setups for networking.
+
+## Network devices
+
+To grant direct network access to an instance, you must assign it at least one network device, also called {abbr}`NIC (Network Interface Controller)`.
+You can configure the network device in one of the following ways:
+
+- Use the default network bridge that you set up during the LXD initialization.
+  Check the default profile to see the default configuration:
+
+        lxc profile show default
+
+  This method is used if you do not specify a network device for your instance.
+- Use an existing network interface by adding it as a network device to your instance.
+  This network interface is outside of LXD control.
+  Therefore, you must specify all information that LXD needs to use the network interface.
+
+  Use a command similar to the following:
+
+        lxc config device add <instance_name> <device_name> nic nictype=<nic_type> ...
+
+  See {ref}`instance_device_type_nic` for a list of available NIC types and their configuration properties.
+
+  For example, you could add a pre-existing Linux bridge (`br0`) with the following command:
+
+        lxc config device add <instance_name> eth0 nic nictype=bridged parent=br0
+- {doc}`Create a managed network </howto/network_create>` and add it as a network device to your instance.
+  With this method, LXD has all required information about the configured network, and you only need to specify the network name when adding the device:
+
+        lxc config device add <instance_name> <device_name> nic network=<network_name>
+
+  If needed, you can add further properties to the command to override the default settings for the network.
+
+## Managed networks
+
+Managed networks in LXD are created and configured with the `lxc network [create|edit|set]` command.
+
+Depending on the network type, LXD either fully controls the network or just manages an external network interface.
+
+Note that not all {ref}`NIC types <instance_device_type_nic>` are supported as network types.
+LXD can only set up some of the types as managed networks.
+
+### Fully controlled networks
+
+Fully controlled networks create network interfaces and provide most functionality, including, for example, the ability to do IP management.
+
+LXD supports the following network types:
+
+{ref}`network-bridge`
+: A network bridge creates a virtual L2 ethernet switch that instance NICs can connect to, making it possible for them to communicate with each other and the host.
+
+  In LXD context, the `bridge` network type creates an L2 bridge that connects the instances that use it together into a single network L2 segment.
+  This makes it possible to pass traffic between the instances.
+  The bridge can also provide local DHCP and DNS.
+
+  This is the default network type.
+
+{ref}`network-ovn`
+: {abbr}`OVN (Open Virtual Network)` is a software-defined networking system that supports virtual network abstraction.
+  You can use it to build your own private cloud.
+  See https://www.ovn.org/ for more information.
+
+  In LXD context, the `ovn` network type creates a logical network.
+  To set it up, you must install and configure the OVN tools.
+  In addition, you must create an uplink network that provides the network connection for OVN.
+  As the uplink network, you should use one of the external network types or a managed LXD bridge.
+
+  ```{tip}
+  Unlike the other network types, you can create and manage an OVN network inside a {ref}`project <projects>`.
+  This means that you can create your own OVN network as a non-admin user, even in a restricted project.
+  ```
+
+### External networks
+
+External networks use network interfaces that already exist.
+Therefore, LXD has limited possibility to control them.
+
+The main purpose for using external networks is to provide an uplink network through a parent interface.
+
+LXD supports the following network types:
+
+{ref}`network-macvlan`
+: Macvlan is a virtual {abbr}`LAN (Local Area Network)` that you can use if you want to assign several IP addresses to the same network interface, basically splitting up the network interface into several sub-interfaces with their own IP addresses.
+  You can then assign IP addresses based on the randomly generated MAC addresses.
+
+  In LXD context, the `macvlan` network type provides a preset configuration to use when connecting instances to a parent Macvlan interface.
+
+{ref}`network-sriov`
+: {abbr}`SR-IOV (Single root I/O virtualization)` is a hardware standard that allows a single network interface to appear as several virtual network interfaces in a virtualized environment.
+
+  In LXD context, the `sriov` network type provides a preset configuration to use when connecting instances to a parent SR-IOV interface.
+
+{ref}`network-physical`
+: The `physical` network type connects to an existing physical network, which can be a network card or a bridge, and serves as an uplink network for OVN.
+
+  It provides a preset configuration to use when connecting OVN networks to a parent interface.
+
+## Recommendations
+
+In general, if you can use a managed network, you should do so because networks are easy to configure and you can reuse the same network for several instances without repeating the configuration.
+
+Which network type to choose depends on your specific use case.
+If you choose a fully controlled network, it provides more functionality than using a network device.
+
+As a general recommendation:
+
+- If you are running LXD on a single system or in a public cloud, use a {ref}`network-bridge`, possibly in connection with the [Ubuntu Fan](https://www.youtube.com/watch?v=5cwd0vZJ5bw).
+- If you are running LXD in your own private cloud, use an {ref}`network-ovn`.
+
+  ```{note}
+  OVN requires a shared L2 uplink network for proper operation.
+  Therefore, using OVN is usually not possible if you run LXD in a public cloud.
+  ```
+- To connect an instance NIC to a managed network, use the `network` property rather than the `parent` property, if possible.
+  This way, the NIC can inherit the settings from the network and you don't need to specify the `nictype`.

doc/network-acls.md → doc/howto/network_acls.md

@@ -5,7 +5,7 @@ discourse: 13223
 # How to configure network ACLs
 
 ```{note}
-Network ACLs are available for the {ref}`OVN NIC type <instance_device_type_nic_ovn>`, the {ref}`OVN network <network-ovn>` and the {ref}`Bridge network <network-bridge>` (with some exceptions, see {ref}`network-acls-bridge-limitations`).
+Network ACLs are available for the {ref}`OVN NIC type <instance_device_type_nic_ovn>`, the {ref}`network-ovn` and the {ref}`network-bridge` (with some exceptions, see {ref}`network-acls-bridge-limitations`).
 ```
 
 Network {abbr}`ACLs (Access Control Lists)` define traffic rules that allow controlling network access between different instances connected to the same network, and access to and from other networks.
@@ -108,7 +108,7 @@ icmp\_code        | string     | no       | If protocol is `icmp4` or `icmp6`, t
 ### Use selectors in rules
 
 ```{note}
-This feature is supported only for the {ref}`OVN NIC type <instance_device_type_nic_ovn>` and the {ref}`OVN network <network-ovn>`.
+This feature is supported only for the {ref}`OVN NIC type <instance_device_type_nic_ovn>` and the {ref}`network-ovn`.
 ```
 
 The `source` field (for ingress rules) and the `destination` field (for egress rules) support using selectors instead of CIDR or IP ranges.
@@ -136,7 +136,7 @@ For example:
 source=@internal
 ```
 
-If your network supports [network peers](network-peers.md), you can reference traffic to or from the peer connection by using a network subject selector in the format `@<network_name>/<peer_name>`.
+If your network supports [network peers](network_ovn_peers.md), you can reference traffic to or from the peer connection by using a network subject selector in the format `@<network_name>/<peer_name>`.
 For example:
 
 ```bash

doc/howto/network_bgp.md

@@ -0,0 +1,23 @@
+---
+discourse: 11567
+---
+
+# How to configure LXD as a BGP server
+
+LXD can act as a BGP server, effectively allowing to establish sessions with upstream BGP routers and announce the addresses and subnets that it's using.
+
+This can be used to allow a LXD server or cluster to directly use internal/external address space, getting the specific subnets or addresses routed to the correct host for it to forward onto the target instance.
+
+For this to work, `core.bgp_address`, `core.bgp_asn` and `core.bgp_routerid` must be set.
+Once those are set, LXD will start listening for BGP sessions.
+
+Peers can be defined on both `bridged` and `physical` managed networks. Additionally in the `bridged` case, a set of per-server configuration keys are also available to override the next-hop. When those aren't specified, the next-hop defaults to the address used for the BGP session.
+
+The `physical` network case is used for `ovn` networks where the uplink network is the one holding the list of allowed subnets and the BGP configuration. Once that parent network is configured, children OVN networks will get their external subnets and addresses announced over BGP with the next-hop set to the OVN router address for the network in question.
+
+The addresses and networks currently being advertised are:
+ - Network `ipv4.address` or `ipv6.address` subnets when the matching `nat` property isn't set to `true`
+ - Network `ipv4.nat.address` and `ipv6.nat.address` when those are set
+ - Instance NIC routes defined through `ipv4.routes.external` or `ipv6.routes.external`
+
+At this time, there isn't a way to only announce some specific routes/addresses to particular peers. Instead it's currently recommended to filter prefixes on the upstream routers.

doc/howto/network_bridge_firewalld.md

@@ -0,0 +1,51 @@
+# How to configure Firewalld
+
+## Allow DHCP, DNS with Firewalld
+
+In order to allow instances to access the DHCP and DNS server that LXD runs on the host when using firewalld
+you need to add the host's bridge interface to the `trusted` zone in firewalld.
+
+To do this permanently (so that it persists after a reboot) run the following command:
+
+```
+firewall-cmd --zone=trusted --change-interface=<LXD network name> --permanent
+```
+
+E.g. for a bridged network called `lxdbr0` run the command:
+
+```
+firewall-cmd --zone=trusted --change-interface=lxdbr0 --permanent
+```
+
+This will then allow LXD's own firewall rules to take effect.
+
+
+## How to let Firewalld control the LXD's iptables rules
+
+When using firewalld and LXD together, iptables rules can overlaps. For example, firewalld could erase LXD iptables rules if it is started after LXD daemon, then LXD container will not be able to do any oubound internet access.
+One way to fix it is to delegate to firewalld the LXD's iptables rules and to disable the LXD ones.
+
+First step is to [allow DNS and DHCP](#allow-dhcp-dns-with-firewalld).
+
+Then to tell to LXD totally stop to set iptables rules (because firewalld will do it):
+```
+lxc network set lxdbr0 ipv4.nat false
+lxc network set lxdbr0 ipv6.nat false
+lxc network set lxdbr0 ipv6.firewall false
+lxc network set lxdbr0 ipv4.firewall false
+```
+
+Finally, to enable iptables firewalld's rules for LXD usecase (in this example, we suppose the bridge interface is `lxdbr0` and the associated IP range is `10.0.0.0/24`:
+```
+firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 0 -i lxdbr0 -s 10.0.0.0/24 -m comment --comment "generated by firewalld for LXD" -j ACCEPT
+firewall-cmd --permanent --direct --add-rule ipv4 filter OUTPUT 0 -o lxdbr0 -d 10.0.0.0/24 -m comment --comment "generated by firewalld for LXD" -j ACCEPT
+firewall-cmd --permanent --direct --add-rule ipv4 filter FORWARD 0 -i lxdbr0 -s 10.0.0.0/24 -m comment --comment "generated by firewalld for LXD" -j ACCEPT
+firewall-cmd --permanent --direct --add-rule ipv4 nat POSTROUTING 0 -s 10.0.0.0/24 ! -d 10.0.0.0/24 -m comment --comment "generated by firewalld for LXD" -j MASQUERADE
+firewall-cmd --reload
+```
+To check the rules are taken into account by firewalld:
+```
+firewall-cmd --direct --get-all-rules
+```
+
+Warning: what is exposed above is not a fool-proof approach and may end up inadvertently introducing a security risk.

doc/howto/network_bridge_resolved.md

@@ -0,0 +1,102 @@
+# How to integrate with systemd-resolved
+
+If the system running LXD uses systemd-resolved to perform DNS
+lookups, it's possible to notify resolved of the domain(s) that
+LXD is able to resolve.  This requires telling resolved the
+specific bridge(s), nameserver address(es), and dns domain(s).
+
+For example, if LXD is using the `lxdbr0` interface, get the
+ipv4 address with `lxc network get lxdbr0 ipv4.address` command
+(the ipv6 can be used instead or in addition), and the domain
+with `lxc network get lxdbr0 dns.domain` (if unset, the domain
+is `lxd` as shown in the table above).  Then notify resolved:
+
+```
+systemd-resolve --interface lxdbr0 --set-domain '~lxd' --set-dns n.n.n.n
+```
+
+Replace `lxdbr0` with the actual bridge name, and `n.n.n.n` with
+the actual address of the nameserver (without the subnet netmask).
+
+Also replace `lxd` with the domain name.  Note the `~` before the
+domain name is important; it tells resolved to use this
+nameserver to look up only this domain; no matter what your
+actual domain name is, you should prefix it with `~`.  Also,
+since the shell may expand the `~` character, you may need to
+include it in quotes.
+
+In newer releases of systemd, the `systemd-resolve` command has been
+deprecated, however it is still provided for backwards compatibility
+(as of this writing).  The newer method to notify resolved is using
+the `resolvectl` command, which would be done in two steps:
+
+```
+resolvectl dns lxdbr0 n.n.n.n
+resolvectl domain lxdbr0 '~lxd'
+```
+
+This resolved configuration will persist as long as the bridge
+exists, so you must repeat this command each reboot and after
+LXD is restarted (see below on how to automate this).
+
+Also note this only works if the bridge `dns.mode` is not `none`.
+
+Note that depending on the `dns.domain` used, you may need to disable
+DNSSEC in resolved to allow for DNS resolution. This can be done through
+the `DNSSEC` option in `resolved.conf`.
+
+To automate the `systemd-resolved` DNS configuration when LXD creates the `lxdbr0` interface so that it is applied
+on system start you need to create a systemd unit file `/etc/systemd/system/lxd-dns-lxdbr0.service` containing:
+
+```
+[Unit]
+Description=LXD per-link DNS configuration for lxdbr0
+BindsTo=sys-subsystem-net-devices-lxdbr0.device
+After=sys-subsystem-net-devices-lxdbr0.device
+
+[Service]
+Type=oneshot
+ExecStart=/usr/bin/resolvectl dns lxdbr0 n.n.n.n
+ExecStart=/usr/bin/resolvectl domain lxdbr0 '~lxd'
+
+[Install]
+WantedBy=sys-subsystem-net-devices-lxdbr0.device
+```
+
+Be sure to replace `n.n.n.n` in that file with the IP of the `lxdbr0` bridge.
+
+Then enable and start it using:
+
+```
+sudo systemctl daemon-reload
+sudo systemctl enable --now lxd-dns-lxdbr0
+```
+
+If the `lxdbr0` interface already exists (i.e LXD is running), then you can check that the new service has started:
+
+```
+sudo systemctl status lxd-dns-lxdbr0.service
+● lxd-dns-lxdbr0.service - LXD per-link DNS configuration for lxdbr0
+     Loaded: loaded (/etc/systemd/system/lxd-dns-lxdbr0.service; enabled; vendor preset: enabled)
+     Active: inactive (dead) since Mon 2021-06-14 17:03:12 BST; 1min 2s ago
+    Process: 9433 ExecStart=/usr/bin/resolvectl dns lxdbr0 n.n.n.n (code=exited, status=0/SUCCESS)
+    Process: 9434 ExecStart=/usr/bin/resolvectl domain lxdbr0 ~lxd (code=exited, status=0/SUCCESS)
+   Main PID: 9434 (code=exited, status=0/SUCCESS)
+```
+
+You can then check it has applied the settings using:
+
+```
+sudo resolvectl status lxdbr0
+Link 6 (lxdbr0)
+      Current Scopes: DNS
+DefaultRoute setting: no
+       LLMNR setting: yes
+MulticastDNS setting: no
+  DNSOverTLS setting: no
+      DNSSEC setting: no
+    DNSSEC supported: no
+  Current DNS Server: n.n.n.n
+         DNS Servers: n.n.n.n
+          DNS Domain: ~lxd
+```