Service IPs
Calico supports two approaches for assigning a service IP to a Calico-networked VM:
-
using a floating IP
-
using an additional fixed IP on the relevant Neutron port.
Both of these are standard Neutron practice - in other words, operations that have long been supported on the Neutron API. They are not Calico-specific, except insofar as the Calico driver needs to implement some of the low-level operations that are needed to make the expected semantics work.
The key semantic difference between those approaches is that:
-
With a floating IP, the target VM itself is not aware of the service IP. Instead, data sent to the floating IP is DNAT’d, to the target VM’s fixed IP, before that data reaches the target VM. So the target VM only ever sees data addressed to its fixed IP.
-
With the service IP as an additional fixed IP, the target VM is (and must be) aware of the service IP, because data addressed to the service IP reaches the target VM without any DNAT.
The use of floating IPs is already well known, so we won’t labour how to use those here. For some additional information on how Calico supports floating IPs, see Floating IPs.
The use and maintainance of additional fixed IPs, however, is not so well known, so in the following transcripts we demonstrate this approach for assigning a service IP to a Calico-networked VM.
We begin by creating a test VM that will be the target of the service IP.
Creating a test VM
-
Check the name of the available CirrOS image.
nova image-listIt should return a list of the images and their names.
-
Boot a VM.
nova boot --flavor m1.tiny --image cirros-0.3.2-x86_64 --nic net-name=demo-net testvm1The response should look similar to the following.
-
Check when the VM has booted:
nova listYou should see your VM with the following statuses.
-
Use the following command to obtain the status of the VM.
nova show testvm1It should return something like the following.
In this example, the VM has been given a fixed IP of 10.28.0.13.
-
Let’s look at the corresponding Neutron port.
neutron port-listIt should look something like the following.
Adding a service IP to the Neutron port as an extra fixed IP
Now we want to set up a service IP - let’s say 10.28.0.23 - that
initially points to that VM, testvm1.
-
One way to do that is to add the service IP as a second ‘fixed IP’ on the Neutron port.
neutron port-update --fixed-ip subnet_id=0a1221f2-e6ed-413d-a040-62a266bd0d8f,ip_address=10.28.0.13 \ --fixed-ip subnet_id=0a1221f2-e6ed-413d-a040-62a266bd0d8f,ip_address=10.28.0.23 9a7e0868-da7a-419e-a7ad-9d37e11091b8 -
It should return a confirmation message.
-
Use the following command to get more information about the port.
neutron port-show 9a7e0868-da7a-419e-a7ad-9d37e11091b8It should return a table like the following.
-
Now look at local IP routes.
ip rWe see that we have a route to
10.28.0.23.Note that, on the machine where we’re running these commands:
-
BIRD is running, peered with the BIRDs that Calico runs on each compute node. That is what causes VM routes (including
10.28.0.23) to appear here. -
192.168.8.3 is the IP of the compute node that is hosting
testvm1.
-
-
We can also double check that
10.28.0.23has appeared as a local device route on the relevant compute node.ip rIt should return something like the following.
We also need - because with this approach, data that is addressed to
10.28.0.23will be routed to the VM without any NAT - to tell the VM itself that it has the extra10.28.0.23address. -
SSH into the VM.
-
From inside the VM, issue the following command to list the interfaces.
ip aIt should return something like the following.
-
Next, issue the following command.
sudo ip a a 10.28.0.23/16 dev eth0 -
List the interfaces again.
ip aThe interfaces should now look more like the following.
-
Exit the SSH session.
-
And now we can access the VM on its service IP, as shown below.
Note that we already have security set up that allows SSH to the instance from our access machine (
192.168.8.1). -
You can check this by listing the security groups.
neutron security-group-listIt should return something like the following.
Moving the service IP to another VM
Service IPs are often used for HA, so need to be moved to target a different VM if the first one fails for some reason (or if the HA system just decides to cycle the active VM).
-
To demonstrate that we create a second test VM.
nova boot --flavor m1.tiny --image cirros-0.3.2-x86_64 --nic net-name=demo-net testvm2 -
List the VMs.
nova listYou should see the new VM in the list.
-
Check the ports.
neutron port-listIt should return something like the following.
-
Remove the service IP from the first VM.
neutron port-update --fixed-ip subnet_id=0a1221f2-e6ed-413d-a040-62a266bd0d8f,ip_address=10.28.0.13 9a7e0868-da7a-419e-a7ad-9d37e11091b8 -
And add it to the second.
neutron port-update --fixed-ip subnet_id=0a1221f2-e6ed-413d-a040-62a266bd0d8f,ip_address=10.28.0.14 \ --fixed-ip subnet_id=0a1221f2-e6ed-413d-a040-62a266bd0d8f,ip_address=10.28.0.23 7627a298-a2db-4a1a-bc07-9f0f10f58363 -
SSH into
testvm2. -
Tell
testvm2that it now has the service IP10.28.0.23.sudo ip a a 10.28.0.23/16 dev eth0 -
Now connections to
10.28.0.23go totestvm2 -
Remove the
known_hostsfiles.rm ~/.ssh/known_hosts -
Try again to SSH into the VM.
-
Check the host name.
hostnameIt should return:
-
Check the interfaces.
ip aThey should look something like the following.