A retired engineer and entrepreneur, now working with ROS

Updated:

This page contains notes from my various tests and experiments. It is a raw record of what I did, without correction for errors, or later update for things that I learn. Use at your own risk.

More OpenVPN DHCP tweaks

Continuing from yesterday, I want to make a standard install plan for an OpenVPN AWS gateway that pulls its IP address from the local (robot) network.

I successfully rebooted the OpenVPN server, so whatever caused my grief yesterday is gone.

First, let me remove the Cargo Culted stp: false from the netplan file, leaving this for /etc/netplan/90-rosbridge.yaml:

network:
  version: 2
  ethernets:
    tap0:
      match:
        name: "tap0"

  bridges:
    rosbridge:
      mtu: 1500
      interfaces: [tap0]
      dhcp4: true
      dhcp6: false

I’m also going to work on the openvpn config file directly, instead of going through the docker config. Here is the file that was generated by docker config by sudo ovpn_genconfig -u udp://$OPENVPN_URL -t -d -D at /etc/openvpn/openvpn.conf:

server 192.168.255.0 255.255.255.0
verb 3
key /etc/openvpn/pki/private/44.232.53.80.key
ca /etc/openvpn/pki/ca.crt
cert /etc/openvpn/pki/issued/44.232.53.80.crt
dh /etc/openvpn/pki/dh.pem
tls-auth /etc/openvpn/pki/ta.key
key-direction 0
keepalive 10 60
persist-key
persist-tun

proto udp
# Rely on Docker to do port mapping, internally always 1194
port 1194
dev tap0
status /tmp/openvpn-status.log

user nobody
group nogroup
comp-lzo no

### Push Configurations Below
push "comp-lzo no"

Issues to resolve:

  • Stop pushing routes (use route-noexec)

I added that to the conf, then

systemctl restart openvpn@openvpn

On the local system, I connect to openvpn, then add tap0 to the bridge. On the AWS gateway,

root@rosovpn:/etc/openvpn# ip link set down rosbridge
root@rosovpn:/etc/openvpn# ip link set ip rosbridge

resulted in:

root@rosovpn:/etc/openvpn# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc fq_codel state UP group default qlen 1000
    link/ether 02:fe:c5:16:53:e9 brd ff:ff:ff:ff:ff:ff
    inet 172.31.24.240/20 brd 172.31.31.255 scope global dynamic eth0
       valid_lft 3490sec preferred_lft 3490sec
    inet6 fe80::fe:c5ff:fe16:53e9/64 scope link 
       valid_lft forever preferred_lft forever
3: rosbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5a:3f:89:62:4d:9d brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.84/24 brd 192.168.0.255 scope global dynamic rosbridge
       valid_lft 7124sec preferred_lft 7124sec
    inet6 fe80::583f:89ff:fe62:4d9d/64 scope link 
       valid_lft forever preferred_lft forever
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:88:b2:00:17 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
6: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master rosbridge state UNKNOWN group default qlen 100
    link/ether 5a:3f:89:62:4d:9d brd ff:ff:ff:ff:ff:ff
root@rosovpn:/etc/openvpn# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.31.16.1     0.0.0.0         UG    100    0        0 eth0
0.0.0.0         192.168.0.11    0.0.0.0         UG    100    0        0 rosbridge
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.31.16.0     0.0.0.0         255.255.240.0   U     0      0        0 eth0
172.31.16.1     0.0.0.0         255.255.255.255 UH    100    0        0 eth0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 rosbridge
192.168.0.11    0.0.0.0         255.255.255.255 UH    100    0        0 rosbridge

That is, I have the offending default root. I bet that is coming from netplan and not from openvpn. Can I fix that? I see dhcp4-overrides has use-dns and use-routes options. What is the syntax? I see an example here. Use that:

root@rosovpn:/etc/netplan# cat 90-rosbridge.yaml 
network:
  version: 2
  ethernets:
    tap0:
      match:
        name: "tap0"

  bridges:
    rosbridge:
      mtu: 1500
      interfaces: [tap0]
      parameters:
        stp: false
      dhcp4: true
      dhcp4-overrides:
        use-dns: false
        use-routes: false
      dhcp6: false
root@rosovpn:/etc/netplan# netplan apply
root@rosovpn:/etc/netplan# ip link set down rosbridge
root@rosovpn:/etc/netplan# ip link set up rosbridge
root@rosovpn:/etc/netplan# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.31.16.1     0.0.0.0         UG    100    0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.31.16.0     0.0.0.0         255.255.240.0   U     0      0        0 eth0
172.31.16.1     0.0.0.0         255.255.255.255 UH    100    0        0 eth0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 rosbridge

So that worked. I can ping the remote rosbridge address from the local.

Next issue: modify the local config to add tap0 to the rosbridge by default. I think I can just do what I did on the remote. First, I confirmed that disconnecting the local from openvpn and reconnecting resulted in tap0 that is not under the rosbridge. Here’s the netplan mod file:

kent@ubutower:/etc/netplan$ cat 90-rosbridge.yaml 
network:
  version: 2
  ethernets:
    bridge-me:
      match:
        name: "*"

  bridges:
    rosbridge:
      mtu: 1500
      interfaces: [bridge-me]
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]
      parameters:
        stp: false
      dhcp4: true
      dhcp6: false

I would have expected that to work.

Try connecting, then sudo netplan apply. Nope.

Try rosbridge down/up. Nope.

Try tap0 down/up. Nope.

I’m going to try a more specific netplan file:

network:
  version: 2
  ethernets:
    estar:
      match:
        name: "e*"
    tap0:
      match:
        name: "tap0"

  bridges:
    rosbridge:
      mtu: 1500
      interfaces: [estar, tap0]
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]
      dhcp4: true
      dhcp6: false

I notice after netplan apply there is still an old bridge-me file in /etc/NetworkManager/system-connections. Remove it. I see the expected files under /run/NetworkManager/system-connections

Connect openvpn. Still tap0 not under bridge. Down/up tap0, no change.

OK, let’s use openvpn options to add tap0 to the bridge. Remove the tap0 stuff from the netplan file.

kent@ubutower:/etc/netplan$ cat 90-rosbridge.yaml 
network:
  version: 2
  ethernets:
    estar:
      match:
        name: "e*"

  bridges:
    rosbridge:
      mtu: 1500
      interfaces: [estar]
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]
      dhcp4: true
      dhcp6: false

Add to bottom of ros_local_gateway.ovpn:

script-security 2
up "/bin/bash -c 'ip link set master rosbridge dev tap0'"

Yep that works, and I can ping the remote bridge ip address.

I am still getting a unused address on tap0 though:

root@ubutower:~# ip a show tap0
```bash
14: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master rosbridge state UNKNOWN group default qlen 100
    link/ether 1a:52:25:3a:d1:e5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.255.2/24 brd 192.168.255.255 scope global tap0
       valid_lft forever preferred_lft forever
    inet6 fe80::1852:25ff:fe3a:d1e5/64 scope link 
       valid_lft forever preferred_lft forever

Try adding ifconfig-noexec to the local .ovpn as well. Ping to remote did not work, tap0 is down. Bring it up, Ping did not work, tried a few seconds later now it works. Need to add that to .ovpn Now lines at end are:

script-security 2
up "/bin/bash -c 'ip link set master rosbridge dev tap0 && ip link set up tap0'"
ifconfig-noexec

Ping the remote at 192.168.0.84. Took a few tries, but eventually it started working after maybe 10 seconds.

I connected tap0 to Wireshark. It took about 18 seconds until an ARP request went out, once that happened Ping started working. I also noticed two things: 1) the MAC address of the local rosbridge and ethernet adapter are the same, and 2) there is unneeded STP traffic every 2 seconds. Perhaps I should turn off STP.

I am going to try to create the local ROS bridge with its own MAC address. I used a website to get a uuid: c5f6a996-eb51-4277-b79f-0e7f2c2b41a5. I want a private MAC address, which means the second digit must be 2,6,a,, or e. So I will use 0e:7f:2c:2b:41:a5 as MAC address.

Yeah that works, but did not change the startup delay. I tried simply waiting 30 seconds after connect before trying the ping, that worked. So it just takes some time for the network to decide to work. I deleted the mac address from the bridge, tried the same thing, same result. So I don’t see any reason to set the mac address of the bridge. Actually Googling I find this where the answer says that the duplicated mac address is by design, so I will leave that.

On the delay, I tried ip neigh flush all no help.

So who should add tap0 to the bridge? On the AWS remote, the netplan config is doing it. On the local, I added an up command in openvpn. I think that the up approach is likely to be more reliable (and may be needed for other reasons). So let me use that in both places.

Back to the AWS remote, I modified netplan file root@rosovpn:/etc/netplan# cat 90-rosbridge.yaml:

network:
  version: 2

  bridges:
    rosbridge:
      mtu: 1500
      parameters:
        stp: false
      dhcp4: true
      dhcp4-overrides:
        use-dns: false
        use-routes: false
      dhcp6: false

I’m going to reconfigure OpenVPN using docker scripts. Took me awhile to get the quotes right, but this works:

OPENVPN_URL=$(curl icanhazip.com)
UP_COMMAND="/bin/bash -c 'ip link set master rosbridge dev tap0 && ip link set up tap0'"
sudo ovpn_genconfig -u udp://$OPENVPN_URL -t -d -D \
  -e "up \"${UP_COMMAND}\"" \
  -e "script-security 2" \
  -e "ifconfig-noexec"

but there is left over stuff from previous attempts. If I delete /etc/openvpn/ovpn_env.sh first, and restart the su session, junk is gone.

I start (or restart) openvpn with sudo systemctl start openvpn@openvpn. tap0 exists and is under the bridge. There are no routes.

Back at the local, we still have:

root@ubutower:~# cat /etc/netplan/90-rosbridge.yaml 
network:
  version: 2
  ethernets:
    estar:
      match:
        name: "e*"

  bridges:
    rosbridge:
      mtu: 1500
      interfaces: [estar]
      parameters:
        stp: false
      dhcp4: true
      dhcp6: false

Connecting. Back on the remote, rosbridge has no IP address. But wait, after some time it got it with no actiion from me! Ping from local to remote bridge address also works. I think this is all working. Let me summarize everybody.

Local:

root@ubutower:~# cat /etc/netplan/90-rosbridge.yaml 
network:
  version: 2
  ethernets:
    estar:
      match:
        name: "e*"

  bridges:
    rosbridge:
      mtu: 1500
      interfaces: [estar]
      parameters:
        stp: false
      dhcp4: true
      dhcp6: false

Added to .ovpn:

script-security 2
up "/bin/bash -c 'ip link set master rosbridge dev tap0 && ip link set up tap0'"
ifconfig-noexec

On AWS remote:

root@rosovpn:~# cat /etc/netplan/90-rosbridge.yaml 
network:
  version: 2

  bridges:
    rosbridge:
      mtu: 1500
      parameters:
        stp: false
      dhcp4: true
      dhcp4-overrides:
        use-dns: false
        use-routes: false
      dhcp6: false

Command to configure openvpn using docker scripts:

OPENVPN_URL=$(curl icanhazip.com)
UP_COMMAND="/bin/bash -c 'ip link set master rosbridge dev tap0 && ip link set up tap0'"
sudo ovpn_genconfig -u udp://$OPENVPN_URL -t -d -D \
  -e "up \"${UP_COMMAND}\"" \
  -e "script-security 2" \
  -e "ifconfig-noexec"

Now I am going to reboot both systems and try this again. Should only have to reconnect openvpn. Yep, that worked.

How about ROS2? On the local:

ros2 run demo_nodes_cpp talker

On the remote, create the cyclonedds config file and run:

ubuntu@rosovpn:~$ cat cyclonedds.xml 
<?xml version="1.0" encoding="UTF-8" ?>
  <CycloneDDS xmlns="https://cdds.io/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
    <Domain id="any">
        <General>
            <NetworkInterfaceAddress>rosbridge</NetworkInterfaceAddress>
        </General>
    </Domain>
  </CycloneDDS>
ubuntu@rosovpn:~$ CYCLONEDDS_URI=file://$PWD/cyclonedds.xml ros2 run demo_nodes_cpp listener
[INFO] [1635972343.097352004] [listener]: I heard: [Hello World: 1]
[INFO] [1635972344.097346401] [listener]: I heard: [Hello World: 2]

Works!

I want to automate the client extensions. The ovpn_getclient script uses an array variable OVPN_EXTRA_CLIENT_CONFIG got this. This in turn is set with the -E option in config. Try that. That is, run:

sudo rm /etc/openvpn/ovpn_env.sh
OPENVPN_URL=$(curl icanhazip.com)
UP_COMMAND="/bin/bash -c 'ip link set master rosbridge dev tap0 && ip link set up tap0'"
sudo ovpn_genconfig -u udp://$OPENVPN_URL -t -d -D \
  -e "ifconfig-noexec" \
  -e "script-security 2" \
  -e "up \"${UP_COMMAND}\"" \
  -E "ifconfig-noexec" \
  -E "script-security 2" \
  -E "up \"${UP_COMMAND}\""
sudo ovpn_getclient ros_local_gateway > ros_local_gateway.ovpn

Yep that worked. I think that I have all of the pieces for a summary post now.