Bridge Interface vs Macvtap Interface in TrueNAS

More clear information can be found in references.

Description

Note: This is based on my understanding, might be incorrect.

Bridge and macvtap both create a network interface on physical network, which is used by VMs.

One VM uses one dedicated Macvtap interface on host with same MAC address. Macvtap is a network interface on macvlan.

VMs on bridge share same bridge interface which has different MAC address in the host.

Bridge Mode

Virtual interfaces in VMs => Bridge interface => Physical Interface in Host

Macvlan

Macvtap interfaces => Physical Interface in Host

Pros

Macvtap

• Macvtap interfaces in host, can tap on different physical interface, changing to different physical interface is done in host.
• Passthru, VM uses same interface as host created
• If the VM is MAC address sensitive, then should use macvtap

Bridge

• VM and host can communicate to each other
• VM can use host services
• Bridge can be created without physical interface

Cons

Macvtap

• VM can not communicated with host

Bridge

• VMs' virtual interfaces use same bridge interface in host

Sample

Macvtap

truenas# ifconfig -a
bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 1500
        inet 192.168.1.19  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 06:b1:f7:6d:13:4c  txqueuelen 1000  (Ethernet)
        RX packets 1680348527  bytes 2208822464277 (2.0 TiB)
        RX errors 0  dropped 151  overruns 0  frame 0
        TX packets 1617739524  bytes 1698187389538 (1.5 TiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp17s0: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether 54:04:a6:4b:81:c8  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 5107071  bytes 2866624273 (2.6 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5107071  bytes 2866624273 (2.6 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

macvtap11: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::2a0:98ff:fe78:393  prefixlen 64  scopeid 0x20<link>
        ether 00:a0:98:78:03:93  txqueuelen 500  (Ethernet)
        RX packets 22627262  bytes 78234456341 (72.8 GiB)
        RX errors 2324  dropped 2324  overruns 0  frame 0
        TX packets 14142613  bytes 71245696317 (66.3 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

macvtap12: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::2a0:98ff:fe1f:d5c7  prefixlen 64  scopeid 0x20<link>
        ether 00:a0:98:1f:d5:c7  txqueuelen 500  (Ethernet)
        RX packets 480  bytes 943563 (921.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 301  bytes 36435 (35.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth7ae8af79: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::80de:6cff:fe6c:3ac2  prefixlen 64  scopeid 0x20<link>
        ether b2:16:c9:7a:5d:51  txqueuelen 0  (Ethernet)
        RX packets 1049172  bytes 819582891 (781.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1006473  bytes 532661621 (507.9 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethe3db1df9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::1001:84ff:feaf:f3c3  prefixlen 64  scopeid 0x20<link>
        ether 5e:93:0b:01:a8:b0  txqueuelen 0  (Ethernet)
        RX packets 818421  bytes 79271857 (75.5 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 858581  bytes 75468204 (71.9 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlp15s0: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether 00:08:ca:28:b8:d1  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Bridge

truenas# ifconfig -a
bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 1500
        ether 06:b1:f7:6d:13:4c  txqueuelen 1000  (Ethernet)
        RX packets 2273930881  bytes 3403517718704 (3.0 TiB)
        RX errors 0  dropped 4615  overruns 0  frame 0
        TX packets 417927732  bytes 27440289291 (25.5 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.19  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 06:62:23:59:d5:35  txqueuelen 1000  (Ethernet)
        RX packets 628064711  bytes 1811199459942 (1.6 TiB)
        RX errors 0  dropped 2  overruns 0  frame 0
        TX packets 331791210  bytes 1156251281548 (1.0 TiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp17s0: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether 54:04:a6:4b:81:c8  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 2676343  bytes 1435559819 (1.3 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2676343  bytes 1435559819 (1.3 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth8dfea17d: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::780d:4cff:fe3c:a108  prefixlen 64  scopeid 0x20<link>
        ether ae:fa:aa:94:e6:3f  txqueuelen 0  (Ethernet)
        RX packets 432268  bytes 41857026 (39.9 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 437864  bytes 38739338 (36.9 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethc44c60e0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::dc33:23ff:fef3:7f06  prefixlen 64  scopeid 0x20<link>
        ether 8a:af:5b:12:51:36  txqueuelen 0  (Ethernet)
        RX packets 549957  bytes 433379151 (413.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 480138  bytes 272756471 (260.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vnet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::fca0:98ff:fe78:393  prefixlen 64  scopeid 0x20<link>
        ether fe:a0:98:78:03:93  txqueuelen 1000  (Ethernet)
        RX packets 79558657  bytes 253665109225 (236.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 221506436  bytes 2888491048856 (2.6 TiB)
        TX errors 0  dropped 1220 overruns 0  carrier 0  collisions 0

wlp15s0: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether 00:08:ca:28:b8:d1  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

References

Bridge vs Macvlan
Enabling host-guest networking with KVM, Macvlan and Macvtap