In this second part of the article, we configure the
Configuring the service IP address
As already mentioned in the first part, using the “carp” kernel module -which implements the protocol with the same
The configuration consists of an extra line in the /etc/rc.conf file of each node.
For the first node, we will run the following:
1 |
root@server-1 # echo 'ifconfig_em0_alias0="inet vhid 1 pass secret alias 192.0.2.100/32"' >> /etc/rc.conf |
For the second, we will run:
1 |
root@server-2 # echo 'ifconfig_em0_alias0="inet vhid 1 pass secret alias 192.0.2.100/32 advskew 100"' >> /etc/rc.conf |
Let’s examine this in a generic way:
1 2 3 4 5 6 7 8 |
ifconfig_em0_alias0="<rest of parameters>" inet vhid 1 pass secret alias 192.0.2.100/32 # Only on the second node: advskew 100 |
The first line indicates the use of an IP alias. This means that the ’em0′ NIC will have an additional IP to the one already configured as the primary IP (192.0.2.5/24 in our case for node 1, and 192.0.2.6/24 for the second one). Everything that goes between quotes are the parameters that this alias will take.
The second line states that we are going to use the “inet” family (IPv4), and if it is not specified, it is the default value. Another option is “inet6” for the IPv6 protocol.
In the third line, the virtual host identifier is established. Each service IP address must have the same value in all the nodes where it is configured. If we had more service addresses, this field should change. The allowed values are from 1 to 255.
The fourth line establishes the authentication password with which the participating nodes will be identified in a “
In the fifth line, the IP address that we want as the service IP address is established.
Finally, and only for the secondary node, the value of “advskew” is set to 100. This value introduces a delay when the node is “announced” as a CARP node, modifying its order of precedence; and it is useful when we want to force a primary node automatically, or there are multiple secondary nodes.
At this time the change must be applied in each node.
For the first node we will execute:
1 |
root@server-1 # ifconfig em0 inet vhid 1 pass secret alias 192.0.2.100/30 |
And, in the second:
1 |
root@server-2 # ifconfig em0 inet vhid 1 pass secret alias 192.0.2.100/30 |
We can verify the correct functioning in several ways.
Using the “ifconfig em0” command in each node:
1 2 3 4 5 6 7 8 9 10 11 |
root@server-1:~ # ifconfig em0 em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM> ether 08:00:27:bf:d9:13 hwaddr 08:00:27:bf:d9:13 inet 192.0.2.5 netmask 0xffffff00 broadcast 192.0.2.255 inet 192.0.2.100 netmask 0xffffffff broadcast 192.0.2.100 vhid 1 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active carp: MASTER vhid 1 advbase 1 advskew 0 |
1 2 3 4 5 6 7 8 9 10 11 |
root@server-2:~ # ifconfig em0 em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM> ether 08:00:27:5b:00:66 hwaddr 08:00:27:5b:00:66 inet 192.0.2.6 netmask 0xffffff00 broadcast 192.0.2.255 inet 192.0.2.100 netmask 0xffffffff broadcast 192.0.2.100 vhid 1 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active carp: BACKUP vhid 1 advbase 1 advskew 100 |
If we observe the last line, in the first server it shows “CARP: MASTER” and, in the second, “CARP: BACKUP”.
Another option, where we will also see more information such as the choice of the “MASTER” node, state transitions, etc., is the / var / log / messages file of each node:
1 2 3 |
root@server-1:~ # grep carp /var/log/messages Oct 9 08:14:29 server-1 kernel: carp: 1@em0: INIT -> BACKUP (initialization complete) Oct 9 08:14:32 server-1 kernel: carp: 1@em0: BACKUP -> MASTER (master timed out) |
1 2 3 |
root@server-2:~ # grep carp /var/log/messages Oct 9 08:11:19 server-2 kernel: carp: 1@em0: BACKUP -> MASTER (master timed out) Oct 9 08:15:23 server-2 kernel: carp: 1@em0: INIT -> BACKUP (initialization complete) |
Verifying functionality
With the configuration already done and activated and, the nodes defined in master and slave, it is time to make the necessary tests to verify that the behavior is appropriate.
We’ll use the “ping” command from some system on the same 192.0.2.100/24 subnet to check the service IP address availability:
1 2 3 4 5 6 7 8 9 10 11 12 |
root@client-1:~# ping -c 5 192.0.2.100 PING 192.0.2.100 (192.0.2.100): 56 data bytes 64 bytes from 192.0.2.100: icmp_seq=0 ttl=64 time=0.301 ms 64 bytes from 192.0.2.100: icmp_seq=1 ttl=64 time=0.419 ms 64 bytes from 192.0.2.100: icmp_seq=2 ttl=64 time=0.621 ms 64 bytes from 192.0.2.100: icmp_seq=3 ttl=64 time=0.379 ms 64 bytes from 192.0.2.100: icmp_seq=4 ttl=64 time=0.281 ms --- 192.0.2.100 ping statistics --- 5 packets transmitted, 5 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 0.281/0.400/0.621/0.121 ms |
The first real test we will do is verify what happens when the master node becomes unavailable due to a reboot or power failure, and the second one consists
In case of active node reboot
Connected to both nodes, on the master we’ll reboot the server, and on the second we check what happens:
1 |
root@server-1:~# shutdown -r now |
1 2 3 |
root@server-2:~# tail -f /var/log/messages [...] Oct 9 08:28:19 server-2 kernel: carp: 1@em0: BACKUP -> MASTER (master timed out) |
The backup node becomes the master and the service IP address is assigned to it:
1 2 3 4 5 6 7 8 9 10 11 |
root@server-2:~# ifconfig em0 em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM> ether 08:00:27:5b:00:66 hwaddr 08:00:27:5b:00:66 inet 192.0.2.6 netmask 0xffffff00 broadcast 192.0.2.255 inet 192.0.2.100 netmask 0xffffffff broadcast 192.0.2.100 vhid 1 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active carp: MASTER vhid 1 advbase 1 advskew 100 |
The service IP address continues accessible from other systems on the network. The following output shows what perception a client had during the process since the first node failed until the secondary took the control and started giving service:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
root@client-1:~# ping -D 192.0.2.100 PING 192.0.2.100 (192.0.2.100): 56 data bytes 64 bytes from 192.0.2.100: icmp_seq=0 ttl=64 time=0.390 ms 64 bytes from 192.0.2.100: icmp_seq=1 ttl=64 time=0.426 ms 64 bytes from 192.0.2.100: icmp_seq=2 ttl=64 time=0.387 ms 64 bytes from 192.0.2.100: icmp_seq=3 ttl=64 time=0.657 ms ### REBOOT LAUNCHED AT THIS POINT ### 64 bytes from 192.0.2.100: icmp_seq=6 ttl=64 time=0.393 ms 64 bytes from 192.0.2.100: icmp_seq=7 ttl=64 time=0.457 ms 64 bytes from 192.0.2.100: icmp_seq=8 ttl=64 time=0.334 ms ^C --- 192.0.2.100 ping statistics --- 9 packets transmitted, 7 packets received, 22.2% packet loss round-trip min/avg/max/std-dev = 0.334/0.435/0.657/0.097 ms |
As can be seen, the virtual IP address has not been available for about 3 seconds.
In case of physical network failure
Is possible to simulate the loss of a network interface with the following command:
1 |
root@server-1:~# ifconfig em0 down |
We can verify that the slave node detects this fault in the master:
1 2 3 |
root@server-2:~# tail -f /var/log/messages [...] Oct 9 08:37:27 server-2 kernel: carp: 1@em0: BACKUP -> MASTER (master timed out) |
As in the previous case, the second server becomes MASTER:
1 2 3 4 5 6 7 8 9 10 11 |
root@server-2:~# ifconfig em0 em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM> ether 08:00:27:5b:00:66 hwaddr 08:00:27:5b:00:66 inet 192.0.2.6 netmask 0xffffff00 broadcast 192.0.2.255 inet 192.0.2.100 netmask 0xffffffff broadcast 192.0.2.100 vhid 1 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active carp: MASTER vhid 1 advbase 1 advskew 100 |
El estado actual del servicio sería como muestra la siguiente imagen:
The current service status is like the following image:

If we disable the network interface on the second node by:
1 |
root@server-2:~# ifconfig em0 down |
Having no enabled interface (either the first node or the second), the service would no longer be accessible.

Forcing a node as primary
We may want a particular node to always be the primary node.
For this task, we can use an automatic configuration that consists of adding a line to the /etc/sysctl.conf file of the node that we want it to be:
1 |
root@server-1 # echo "net.inet.carp.preempt=1" >> /etc/sysctl.conf |
If we do not want to restart the node at this time, we will activate the change in the following way:
1 |
root@server-1 # sysctl -w net.inet.carp.preempt=1 |
We can also temporarily set a node as primary using the following command in the current MASTER node:
1 |
root@server-1 # ifconfig em0 vhid 1 state backup |
Final remarks
For simplicity along these two articles related to the service IP address, we used a single physical network interface for both server management and providing the service.
It is advisable to use multiple physical network interfaces, each for a task. Ideally the physical network interface “em0” will be used to provide the service, while another network interface “em1” will be used to administer the server and where CARP exchanges the status of both nodes.
Similarly, for simplicity, we used only one connection for each task. On production environments, where high availability is essential, must have second network links using “link aggregation” with separate network cards; in the future, I will post about it.