Giving My Old PC a New Purpose

My old PC has just been sitting there for several years. I have been meaning to give it purpose for a while now, and have finally decided it will be better suited to some of the things I want to do that my Raspberry Pi just can't hack.

Although my raspi is good at doing some things, it isn't that great at doing many things at the same time, nor doing some specific things. For example, it is great at being a DNS and DHCP server, but if I add PBX, streaming large files from truecrypt encrypted external drives, and serving large image files (and php sites) on a webserver, DNS resolution takes a massive hit.

Current Setup

My home network has 3 DHCP servers running. My ADSL2+ connection goes to my O2 Wireless Box ii (my Zyxel modem was having speed issues) which serves some of the DHCP pool, my Fon 2.0n router serves another chunk of the pool, my raspi serves another chunk, and another two chunks are reserved for fixed IP addresses.

The reason my pi is now doing DHCP as well is because both my routers have needed rebooting at times because their DHCP servers crash. It took a while to narrow down the actual reason iOS devices kept having trouble connecting, but once I realised it was due to DHCP not functioning and let my pi do some of the work connectivity has been a lot better.

A lot better, but not perfect. My pi is also my 6in4 gateway, so IPv6 traffic goes through the pi as well. It also hosts some websites that are only accessible over IPv6 (my VPS acts as a proxy for IPv4 connections).

So, what do I want my old PC to do?

  • Be totally encrypted and able to be booted up remotely.
  • Act as a media server.
  • Act as a PBX.
  • Act as my IPv6 proxy (6in4 endpoint).
  • Be my primary mail server.
  • Be a backup database server.
  • Be a backup DNS server.
  • Other stuff.

A long list, but it is has a single-core Pentium 4 HT 2.8 Ghz processor (32 bit) and 4 GiB (3,549 MiB useable) RAM so is a lot more powerful than my pi. A lot more power-hungry and hot, but with no heating at the moment the internal case temp of 33.3 Celcius means my room isn't cold, but it will be an issue in the summer. It is also extremely loud. That old 200 GB hard drive is probably past it, and the PSU needs replacing, but that can be done further down the line.

Installation

So, the first thing was installation. I plugged the 200 GB drive that was previously in the case in to my eSATA caddy (the other drive, 250 GB, seems to have died) and after finding no evidence of files other than those potentially left on a 150 GiB partition that I couldn't access I decided it was safe to use.

How to install it? I didn't particularly want to burn a DVD (ended up having to burn one anyway although not the one I would have originally chosen) so I started hunting around my room for things. After finding the USB transceiver for my wireless keyboard and some batteries for the keyboard, locating another power cable, and connecting everything up (reminder to self: HDD still needs seating in HDD bay and screwing in) I started looking around the BIOS after a Google search and found what I was after: PXE boot support.

PXE Network Booting

This is something I have never done before, and was the first of several headaches to reach where I currently am. After attempting to download a windows zip file for tftpd32 and Chrome telling me it was malware, and instead downloading tftpd64, I downloaded the Debian network boot image file and tried to get it to work.

  • Create c:\tftpd\ and extract the contents of tftpd64...zip to that folder.
  • Create c:\tftpd\netboot\ and extract the contents of netboot.tar.gz to that folder.
  • Create c:\tftpd\netboot\pxelinux.cfg\ and copy c:\tftpd\netboot\debian-installer\i386\boot-screens\syslinux.cfg to that folder.
  • Rename c:\tftpd\netboot\pxelinux.cfg\syslinux.cfg to default (no file extenion).
  • Copy c:\tftpd\netboot\debian-installer\i386\pxelinux.0 to c:\tftpd\netboot\
  • Start tftpd64, settings button, DHCP tab...
    • IP pool start address: 192.168.1.201
    • Size of pool: 1
    • Boot file: netboot\pxelinux.0
    • Def. Router: <IP of O2 wireless box ii – my default gateway>
    • Mask: 255.255.255.0
    • DNS Servers: <IP of pi, Hurricane Electric DNS>
    • Bind DHCP to this address: <IP address of my laptop on network>
  • I may have changed some other settings, can't remember.

TFTP Error: Configuration File Not Found

The thing that took me the longest to work out was why it kept complaining about configuration file not found, thinking it was a problem with the path rather than a problem with the filename. Neither pxelinux.cfg nor syslinux.cfg are correct - has to be something that is searched for:

  1. A file with a name equal to its MAC address without a file extension, such as 00-00-00-00-00-00
  2. A file with its IP address converted to uppercase hexadecimal. 192 = C0, 0 = 00, 0 = 00, 201 = C9, therefore a file called C00000C9 (without a file extension).
  3. A file with the hex IP address, minus the last character (i.e. C00000C).
  4. Repeat looking for matching hex IP filename with one digit removed from the end each time until, in this case, it can't find a file called C
  5. A file called default (again, with no file extension).

Network Install

Once I got it to boot, I managed to get through everything (Googled some of the settings), and chose guided partitioning (encrypted LVM) with separate /boot, , /, /home, and swap partitions (/boot being outside the LVM).

I then waited and waited, and waited some more. Full formatting and data erasure takes a long time, unfortunately I wasn't asked if I wanted to use ATA Secure Erase to knock 12+ hours off the time it took.

So, I got to the password prompt, entered my password, and then got to work.

  • CTRL+ALT+F1
  • login
  • sudo su -
  • my password
  • apt-get update
  • apt-get dist-upgrade (shouldn't be any updates, just downloaded all the latest packages during network installation)
  • cd /opt
  • download sgfxi using wget.
  • Make sgfxi executable (chmod +x) and run it.
  • Shutdown, enter passphrase, CTRL+ALT+F1, login, sudo su -, run sgfxi again and install driver(s).
  • Start desktop
  • Open gnome-terminal, install busybox and dropbear.

Now I can't remember how I actually got busybox/dropbear to give me the ability to enter my passphrase over ssh, but I do know I had to download and burn a copy of the Debian Rescue Live CD (technically a DVD) because I ended up in a mess - after mounting /dev/sda5 it wanted the passphrase again to mount the UDID of that partition, complained it was already mounted, and asked for the passphrase again (and again, and again, and again - should really pay more attention to error messages).

So, after booting to the rescue "CD", using cryptsetup luksOpen and vgchange -ay, and then making directories under /mnt, mounting the relevant mountpoints, chrooting, deleting that line from /etc/crypttab, running update-grub and update-initramfs, and ignoring the errors and hitting CTRL+ALT+DEL (remove DVD, reboot), I managed to get to the point where I could enter the passphrase through ssh.

As I said, a lot of bother, and I still don't know how to get the passphrase thing back up remotely if the box decides to drop to the initramfs shell.

So here I am, and as expected when I saw the guided partitioning, my / partition is only 10 GB and only has 55% free, without anything (other than Debian and Nvidia drivers) installed yet. Having just read how difficult it is to resize volumes, I'm not presently going to bother.

Since I'm logged on, and speedtest.net requires flash, I'm going to install Google Chrome (since to my knowledge it is the only browser on Debian with the latest version of flash). Also needs a mouse and can't find any AAs, so keynav looks promising.

Issue #1: Chrome/Flash uses >100% of CPU running a speedtest. Only getting ~9.3 Mb/s down, ~1 Mb/s up for some reason. Worry about later.

OK, I am now at the point where I can start doing things. Which to do first?

SSH

Obviously, if I am going to be switching things from my pi to my old PC I will need a way to transfer files between my 3 machines.

Although I could re-use existing keys, I am going to switch to using separate keys on separate machines (I assume there is a way I can track logins more thoroughly as well).

ssh-keygen -t rsa -b 4096 -f ~/.ssh/OldPC -C "john@OldPC"
ssh-keygen -t rsa -b 4096 -f ~/.ssh/Laptop -C "John@Laptop"

I then want to copy OldPC.pub and Laptop.* to my laptop using WinSCP (for now, using a password) and then delete Laptop from the .ssh directory on OldPC. I also want to copy OldPC.pub to my pi and add it to authorized_keys.

Open /etc/ssh/ssh_config, and make the following changes:

  1. LogLevel VERBOSE
  2. PermitRootLogin no
  3. AuthorizedKeysFile /etc/ssh/%u/authorised_keys

Save the file and exit nano, then:

sudo su
cd ~
mv /home/john/.ssh /etc/ssh/john
ln -s /etc/ssh/john /home/john/.ssh

Repeat for any other usernames that need ssh access, then run service ssh restart

Run puttygen and convert Laptop to Laptop.ppk.

Back on OldPC:

cat /etc/ssh/john/*.pub >> /etc/ssh/john/authorised_keys
chown -R john:john /etc/ssh/john/
chmod -R 600 /etc/ssh/john/
chmod -R 644 /etc/ssh/john/*.pub
chmod 700 /etc/ssh/john/

If able to login using key in putty, disable password login in sshd_config (PasswordAuthentication no) and restart ssh. If not, look at auth.log (may need to change LogLevel to DEBUG1 or DEBUG3, try again, and look at the log again to find the issue).

Finally, restart ssh again, and exit the root shell.

Wake on LAN

Next up is Wake on LAN (WOL).

It took me a while to get it to work with my Asus P5KPL motherboard, but here it is.

  1. Enable Wake on LAN from PCI and PCI-e devices in the BIOS.
  2. Add the following to /etc/network/interfaces:
    iface etho inet dhcp
    	ethernet-wol g

That should ensure WOL is persistant across reboots (for some reason Debian/Linux disables it when bringing up connections).

Only issue I now see is if Busybox/Dropbear disable WOL.

  1. sudo halt
  2. Turn on using WOL.
  3. At the password screen (after dropbear displays network config), turn off by holding the power button.
  4. Attempt to turn on using WOL again.
  5. Be unsuccessful.
  6. TODO: Find out if I can enable WOL in busybox easily. If not, decide whether BIOS should prevent auto-on when AC power is restored, and whether init level 0/6 should (can?) be prohibited in dropbear.

get_iplayer

I just had a look at iPlayer's website and there is a new Jonathan Creek series (season?) so since this machine is also going to act as a media server, and streaming the same HD episode from iPlayer multiple times will use both mine and the BBC's bandwidth needlessly, get_iplayer is going to be installed first.

  • sudo su
    apt-get install git
    cd /opt
    git clone git://git.infradead.org/get_iplayer.git
    cd get_iplayer
    git tag -l
    git checkout tags/v2.85
    chmod 755 get_iplayer
    ln -s /opt/get_iplayer/get_iplayer /usr/local/bin/get_iplayer
    mkdir /usr/share/get_iplayer/
    ln -s /opt/get_iplayer/plugins /usr/share/get_iplayer/plugins
    chown -R john:users .
    exit
    mkdir ~/Videos/iPlayer/
  • add the following to ~/.bashrc export IPLAYER_OUTDIR="/home/john/Videos/iPlayer"
  • exit
  • Login, or open a new ssh session to the server
  • get_iplayer
    get_iplayer --modes=flashhd --subtitles --thumbnail --thumbsize=6 --metadata=generic --pvr-add=Jonathan_Creek "Jonathan Creek"
    get_iplayer --pvr
    sudo apt-get install rtmpdump
    get_iplayer --pvr

Samba File Sharing

In order to play that episode of Jonathan Crrek on my laptop, I need to set-up a samba share.

sudo apt-get install samba

Edit /etc/samba/smb.conf and check the following are there:

  • usershare allow guests = yes
  • map to guest = bad user

Add the following to the end of smb.conf:

[iPlayer]
comment = iPlayer Videos
path = /home/john/Videos/iPlayer
browseable = yes
writeable = no
create mask = 755
directory mask = 755
read only = yes
guest ok = yes
guest account = nobody
public = yes

sudo service samba restart

Voila. Downloads not automated yet, but that is because media's final home is a hard drive I haven't bought yet (as with mail, for which I am going to need a fair bit to store all old mail "in-house").

IPv6 Tunnels - 6in4 and IPSEC

Although mail is high on the list, I want to get IPv6 done first. There are several things I am going to have to do:

  1. Set-up firewall rules for IPv6 (probably can just copy from the pi).
  2. Set-up an IPv6 6in4 tunnel (probably can just copy from the pi).
  3. Set-up DHCPv6 (probably can just copy/modify the configuration on the pi).
  4. Set-up announcing public IPv4 IP address to Hurricane Electric (probably can just copy the cron job and script from the pi,

O2 Wireless Box ii (Thomson TG585v7) and Hurricane Electric Dynamic DNS (dynDNS)

A slight diversion first. At present my raspi visits the Hurricane Electric IPv4 endpoint update address every 15 minutes using a cron job and a script that uses wget.

Not only is the present way a waste of bandwidth (albeit tiny amounts), but the number of requests each time is sometimes double because the first request gets a 401 error.

Having had a look at the he.net forum, they now offer dynDNS-like endpoint updating. And after spending a couple of hours trying to get it to work on my O2 box, I have learnt the following:

  1. The O2 wireless box ii cannot do dynDNS over HTTPS (contact me on twitter @WatfordJC_LAN if you can find a way).
  2. The O2 wireless box ii does have a "custom" setting, but the only way to tell it where to go is to update that setting using telnet.
  3. Because it is unencrypted, it is probably best not to use my tunnelbroker.net password to do the updates.

Therefore, first connect to the O2 box using telnet. I am going to list the commands to do the following: change connection timeout setting, enable HTTPS, enable ping responder on WAN (and allow ping responses to tunnelbroker - can't remember which IPs were already on my list so will list them all here), enable NAT loopback, enable traffic passing between LAN and Wi-Fi, allow UDP hole punching, enable 6in4 (protocol 41), disable SIP ALG, customise dynDNS settings, change DNS settings, save changes.

  • :env set var=SESSIONTIMEOUT value=3600
  • :saveall
  • :service system modify name=HTTPs state=enabled port=443
  • :service system ifadd name=PING_RESPONDER group=wan
  • :service system list name=PING_RESPONDER expand=enabled
  • :service system ipadd name=PING_RESPONDER ip=127.0.0.1
  • :service system ipadd name=PING_RESPONDER ip=192.168.1.0/24
  • :service system ipadd name=PING_RESPONDER ip=66.220.2.74
  • :saveall
  • :ip config natloopback=enabled
  • :firewall config icmpchecks=disabled
  • :saveall
  • :connection bind application=CONE(UDP) port=0
  • :connection appconfig application=CONE(UDP) timeout=0
  • :nat mapadd intf=O2_ADSL2plus type=nat outside_addr=My.Public.IP inside_addr=192.168.1.33 protocol=6to4 mode=inbound
  • :nat tmpladd intf=O2_ADSL2plus type=nat outside_addr=0.0.0.1 inside_addr=192.168.1.33 protocol=6to4 mode=inbound
  • :nat tmpladd intf=O2_ADSL type=nat outside_addr=0.0.0.1 inside_addr=192.168.1.33 protocol=6to4 mode=inbound
  • :nat tmpladd intf=O2_Static type=nat outside_addr=0.0.0.1 inside_addr=192.168.1.33 protocol=6to4 mode=inbound
  • :connection unbind application=SIP port=5060
  • :saveall
  • :dyndns service modify name=custom server=ipv4.tunnelbroker.net port=www-http request=/nic/update updateinterval=86400
  • :saveall
  • :dns client dnslist
  • :dns client dnsadd addr=192.168.1.17 port=53
  • :dns client dnsadd addr=74.82.42.42 port=53
  • :dns client dnsadd addr=8.8.8.8 port=53
  • :saveall
  • :dns server route list
  • :dns server route add dns=192.168.1.33 metric=2 intf=LocalNetwork
  • :dns server route add dns=74.82.42.42 metric=5 intf=O2_ADSL2plus
  • :dns server route add dns=8.8.8.8 metric=8 intf=O2_ADSL2plus
  • :saveall
  • :dns client config state=disabled
  • :saveall
  • :connection stats
  • :connection clear

A later modification of the above commands list adds alternative DNS servers to the router's DNS client (i.e. dns client dnsadd adds DNS servers so the router doesn't rely on localhost), and then later the router's DNS server is disabled so that it doesn't announce its IP to the LAN as a nameserver. It should make Dynamic DNS more reliable and shave a couple of milliseconds off Wi-Fi DNS lookups. I also added templates for forwarding IPv6-in-IPv4 (protocol 41) packets for the other interfaces, since Sky Broadband migration might switch the interface used.

My O2 box stopped responding to telnet after adding the first DNS entry, so I accessed it over HTTPS and restarted it. Therefore, I have added the saveall command several times to the above. Restarting still prevented me from telnet access, so I had to power cycle it.

For DNS, a lower metric equals a higher priority. With the ISP DNS servers being given a metric of 10, that leaves 10 (0 through 9) possible high priority levels than ISP servers. Although my settings previously were metric=0, I deleted the custom settings one at a time re-adding them with their new metric.

Giving my pi a metric of 2 leaves me 2 higher priority slots (each of which can be shared), and Google DNS a metric of 8 (because I am more likely to give a new server a higher priority than Google). Hurricane Electric gets a priority of 5, because they also are a US-based company and the IP address is from ARIN, so although I currently use them for backup DNS resolution (and IPv6) I want some space each side for if I decide to ditch everything US-related.

The NAT routing of protocol 41 (apparently called 6to4 in the O2 box) was set to the IP address of my raspi above, which will need changing to the IP of my Old PC when I have configured IPv6 on it. The last line above (connection clear) "kill all connections" was needed for me because the 6to4 rules above stopped my IPv6 tunnel from working.

Now, create a random key for the tunnelbroker.net dynDNS (add it on the advanced tab for the tunnel, click outside the text box to save), and in the O2 box on HTTPS go to Toolbox, Dynamic DNS. Click configure.

  • Interface: O2_ADSL2plus (or whatever interface is the public one)
  • Username: same as tunnelbroker.net login username
  • Password: the key just entered on tunnelbroker.net
  • Service: custom
  • Host: one of the options tunnelbroker.net recognises. I used the tunnel host listed on Tunnel Broker's Main Page (i.e. [username]-[tunnel#].tunnel.[tunnel.server].ipv6.he.net). I could have done a rDNS lookup on my Client IPv6 Address, but I don't currently have rDNS setup.
  • Click Apply button.
  • If it works, "(Update successful)" will be appended to the Hostname on the Dynamic DNS -> Overview page.

Unfortunately, there is no way to set it to multiple interfaces, and this router doesn't allow adding new dyndns groups. If the interface used by my ISP changes (e.g. when Sky migrated me) I'll need to change the selected interface (easiest to do in the router's web control panel). Either that or modify 2 other groups (i.e. not the custom group) for use on O2_ADSL and O2_Static.

Internal IPv4 IP address

Access my Fon router, O2 router, and raspberry pi. Find the current DHCP pool settings.

  • Fon router: bridge mode, connected to O2 box, 192.168.1.19, default gateway O2 router, DNS raspi.
  • O2 router: 192.168.1.17, default gateway O2 router, DHCP Pool 192.168.1.65-192.168.1.126, DNS as set above
  • Raspberry Pi: 192.168.1.33, default gateway O2 router, DHCP Pool 192.168.1.129-192.168.1.191, DNS raspi, he.net, Google.

Looking at the IP addresses and ranges, I believe I set my network up like this:

  • 192.168.1.0 is Network ID.
  • 192.168.1.1-192.168.1.14 is reserved.
  • 192.168.1.15 is reserved.
  • 192.168.1.16 is reserved.
  • 192.168.1.17-192.168.1.30 is for static IPs for networking devices.
  • 192.168.1.31 is reserved.
  • 192.168.1.32 is reserved.
  • 192.168.1.33-192.168.1.62 is for static IP devices.
  • 192.168.1.63 is reserved.
  • 192.168.1.64 is reserved.
  • 192.168.1.65-192.168.1.126 is main DHCP Pool.
  • 192.168.1.127 is reserved.
  • 192.168.1.128 is reserved.
  • 192.168.1.129-192.168.1.191 is backup DHCP Pool.
  • 192.168.1.192-192.168.1.198 is for guest static IP addresses.
  • 192.168.1.199 is reserved.
  • 192.168.1.200 is reserved.
  • 192.168.1.201-192.168.1.206 is for temporary static IP addresses.
  • 192.168.1.207 is reserved.
  • 192.168.1.208 is reserved.
  • 192.168.1.209-192.168.1.223 is reserved.
  • 192.168.1.224-192.168.1.238 is reserved for static IP devices.
  • 192.168.1.239 is reserved.
  • 192.168.1.240 is reserved.
  • 192.168.1.241-192.168.1.254 is reserved for static IPs for networking devices.
  • 192.168.1.255 is Broadcast Address.

Having just experienced a 3 TB hard drive die on me, and 366 GB of data that wasn't backed up being irrecoverable, I shall change the reserved IP for OldPC in my pi's dhcp config from .201 to .34.

Time for an intermission while I move 2 3 terabyte drives into OldPC. These are the backups of my Media and Media 2 disks, and it was Media 2 that died on me. IPv6 will continue after dealing with this.

Media Backup Drives and Truecrypt

Since my drives are Truecrypt encrypted, open up putty and connect to OldPC at its new IP address.

sudo su
mkdir /mnt/MediaBackup
mkdir /mnt/Media2Backup
apt-get update
apt-get dist-upgrade
apt-get install truecrypt
  • Download the x86 truecrypt package from Truecrypt's website.
  • Fire up WinSCP, create a new profile, SCP, john, Laptop key, booted up port number (sshd_config has different ssh listening port so there isn't a fingerprint mismatch when switching between pre-boot and post-boot putty sessions).
  • Make sure Xming is running.
  • tar zxvf truecrypt[tab]
  • truecrypt[tab]set[tab]

Install Truecrypt.

  • mkdir Scripts
  • cd Scripts
  • nano mount-media-backups.sh
#!/bin

mediabackup = "/dev/sdb1"
media2backup = "/dev/sdc2"
deviceChosen=$1;

if [ -z "$deviceChosen" ];
then
	echo "Which Device?"
	echo "Media Backup [1]"
	echo "Media 2 Backup [2]"
	read deviceChosen
fi

case $deviceChosen in

	1)
		truecrypt --mount --slot=2 $mediabackup /mnt/MediaBackup;
		;;
	2)
		truecrypt --mount --slot=4 $media2backup /mnt/Media2Backup;
		;;
	*)
		echo "Invalid Choice";
		;;

esac

Create a new Putty session, and add ~/Scripts/mount-media-backups.sh 1 to the command line. Do likewise for one that mounts Media 2 backup.

Add the following line to /etc/sudoers:

%truecrypt ALL=NOPASSWD: /usr/bintruecrypt

Then do the following:

  • sudo groupadd truecrypt
  • sudo adduser $USER truecrypt
  • sudo smbpassword -a john
  • exit

Then add security = user in /etc/samba/smb.conf and add the following to the file:

[Media Backup]
comment = Media Backup
path = /mnt/MediaBackup
browseable = yes
writeable = yes
create mask = 0755
directory mask = 0755
read only = no
guest ok = no
public = no
valid users = thejc

Duplicate that for Media 2 Backup, then reload/restart samba.

... did some other stuff I cannot recall...

A number of notes:

  1. My old computer can't handle Truecrypt (high CPU usage), so the drives are now ext3 formatted.
  2. My LAN can't handle streaming and I believe it is the 100M links rather than hard drive speeds - a gigabit switch and some CAT6 cable is on my wish list.
  3. Security/Permissions policies need to be changed in Windows.
    1. My Windows user needs to be granted Full Control permissions to the backup directory.
    2. The Everyone group needs to be granted Read (Read & Execute, List Folder Contents, Read) permissions to the backup directory.

A note on Wake-on-LAN: For some reason, WOL only works for suspend, suspend-hybrid, hibernate, and reboot. I have yet to find a way to get it to work for shutdown - hardware limitation?


IPv6 Unique Local Addresses - Routing, Tunnelling, Firewalls

Annoyingly someone keeps turning the heating on constant and high, which combined with someone cooking in the kitchen resulted in my case hitting a temp of 41.8 degrees Celcius despite having bought a powerful fan for my room.

So, although I was going to do the IPv6 stuff on my old PC, I shall leave it on my pi for the time being.

It has been a while since I've messed with tunnels, routes, and firewall rules, so I'll get it working (a fair bit of research required first) and then I'll post how to do it.

One page I used for reference in setting this up is Multi-site private IPv6 networking using ULA and IPSEC, although I have had to make modifications.

It took some work to understand how it all works, so this is a condensed list of "need to knows" for future reference:

  1. Avoid firewall overblocking. If the device (raspberry pi in this case) only has one ethernet port, blocking traffic from the "WAN" with a source/destination of fc00::/7 will block traffic from the LAN.
  2. LAN IPv6 comes in on eth0.
    • fd..:...::/48 source/destination from outside the LAN comes in and goes out on ula-net.
      • he-net doesn't see the fd..:...::/48 (LAN/WAN) IPs because ula-net tunnels them.
        • he-net sees fd..::...:/48 traffic as to/from the 2...:...:/128 tunnel endpoints.

IP Addresses and Prefixes

I will use the following documentation IPv6 addresses for this section:

  • fd00:2001:db8::/48 for the ULA prefix.
  • fd00:2001:db8:a::/64 for the ULA prefix on my VPS.
  • fd00:2001:db8:b::/64 for the ULA prefix on my home network.
  • fd00:2001:db8:c::/64 for the ULA prefix on the ULA tunnel.
  • fd00:2001:db8:c::a/128 for the ULA tunnel endpoint on my VPS.
  • fd00:2001:db8:c::b/128 for the ULA tunnel endpoint on my home network.
  • 2001:db8:2001:a::/64 for the IPv6 prefix on my VPS.
  • 2001:db8:2001:a::2/128 for the IPv6 tunnel endpoint on my VPS.
  • 2001:db8:2001:b::/64 for the IPv6 prefix on my home network.
  • 2001:db8:2001:b::2/128 for the IPv6 tunnel endpoint of my home network.

Note: The IPv6 tunnel endpoints (2001:db8:2001:?::2) are the publicly routable addresses at each end of the tunnel. I am routing traffic between ULA addresses at different sites inside a double tunnel (ipv6-in-ipv6 inside ipv6-in-ipv4) because I plan on modifying the tunnel type to IPv6 IPSEC at a later time and this way sounds easiest to modify. These endpoint addresses could, through changing mode ip6ip6, use IPv4/IPv4 IPv6/IPv4, or another combination if desired (although doing so would probably require moving some of the IP6tables rules to IPtables).

ULA Network Tunnel Creation

Create the ULA tunnel on my VPS:

sudo ip -6 tunnel add ula-net mode ip6ip6 remote 2001:db8:2001:b::2 local 2001:db8:2001:a::2 ttl 225
sudo ip -6 link set ula-net up
sudo ip -6 addr add fd00:2001:db8:c::a/64 dev ula-net
sudo ip -6 route add fd00:2001:db8:b::/64 dev ula-net

Create the ULA tunnel on my raspberry pi:

sudo ip -6 tunnel add ula-net mode ip6ip6 remote 2001:db8:2001:a::2 local 2001:db8:2001:b::2 ttl 225
sudo ip -6 link set ula-net up
sudo ip -6 addr add fd00:2001:db8:c::b/64 dev ula-net
sudo ip -6 route add fd00:2001:db8:a::/64 dev ula-net

I believe the ULA tunnel endpoints only need to route both /128 endpoint addresses (fd00:2001:db8:c::a and fd00:2001:db8:c::b), however if I were to add a third site, I would create another ULA tunnel on my VPS and pi in the same manner. By utilising metrics it should be possible to have a mesh-type topology with multiple routes between sites and a preferred "most direct" route.

It might also be worth using a another 6in4 tunnel (not a double-tunnel) in case the Hurricane Electric tunnel goes down at either end, so both networks are still connected, and since my VPS has 3 IP addresses, perhaps 5 tunnels at each end for redundancy. For the time being that sounds like too much work.

IP6tables Rules For ULA Routing Between Sites

Add the following to the file I use for ip6tables-restore on my VPS:

# Allow some ICMP traffic
-A INPUT -i ula-net -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i ula-net -p ipv6-icmp -j ACCEPT

# Needed for IPv6 tunnels to function properly
-A INPUT -p ipv6-nonxt -j ACCEPT
-A INPUT -p udp -m udp --dport 32768:61000 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 32768:61000 ! --tcp-flags FIN,SYN,RST,ACK SYN -j ACCEPT

# Allow all ULA network traffic out on ula-net except local network prefix
#-A OUTPUT -o ula-net -d fd00:2001:db8:a::/64 -j REJECT --reject-with icmp-no-route
-A OUTPUT -o ula-net -d fd00:2001:db8::/48 -j ACCEPT

# Block traffic on interfaces it shouldn't be going in/out on.
-A INPUT -i eth0 -d  fc00::/7 -j REJECT --reject-with icmp6-no-route
-A OUTPUT -o eth0 -d  fc00::/7 -j REJECT --reject-with icmp6-no-route
-A INPUT -i he-ipv6 -d fc00::/7 -j REJECT --reject-with icmp6-no-route
-A OUTPUT -o he-ipv6 -d fc00::/7 -j REJECT --reject-with icmp6-no-route
-A INPUT -i netassist -d fc00::/7 -j REJECT --reject-with icmp6-no-route
-A OUTPUT -o netassist -d fc00::/7 -j REJECT --reject-with icmp6-no-route

# Route traffic between networks
-A INPUT -i he-ipv6 -s 2001:db8:2001:b::2 -d 2001:db8:2001:a::2 -j ACCEPT
-A FORWARD -i he-ipv6 -o ula-net -s 2001:db8:2001:b::2 -d 2001:db8:2001:a::2 -j ACCEPT
-A INPUT -i ula-net -s fd00:2001:db8::/48 -d fd00:2001:db8:a::/64 -j ACCEPT
-A OUTPUT -o ula-net -s fd00:2001:db8:a::/64 -d fd00:2001:db8::/48 -j ACCEPT

Add the following to the file I use for ip6tables-restore on my raspberry pi:

# Allow some ICMP traffic
-A INPUT -i ula-net -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i ula-net -p ipv6-icmp -j ACCEPT

# Needed for IPv6 tunnels to function properly
-A INPUT -p ipv6-nonxt -j ACCEPT
-A INPUT -p udp -m udp --dport 32768:61000 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 32768:61000 ! --tcp-flags FIN,SYN,RST,ACK SYN -j ACCEPT

# Allow all ULA network traffic out on ula-net except local network prefix
#-A OUTPUT -o ula-net -d fd00:2001:db8:b::/64 -j REJECT --reject-with icmp-no-route
-A OUTPUT -o ula-net -d fd00:2001:db8::/48 -j ACCEPT

# Block traffic on interfaces it shouldn't be going in/out on.
#-A INPUT -i eth0 -d  fc00::/7 -j REJECT --reject-with icmp6-no-route
#-A OUTPUT -o eth0 -d  fc00::/7 -j REJECT --reject-with icmp6-no-route
-A INPUT -i he-ipv6 -d fc00::/7 -j REJECT --reject-with icmp6-no-route
-A OUTPUT -o he-ipv6 -d fc00::/7 -j REJECT --reject-with icmp6-no-route

# Route traffic between networks
-A INPUT -i he-ipv6 -s 2001:db8:2001:a::2 -d 2001:db8:2001:b::2 -j ACCEPT
-A FORWARD -i he-ipv6 -o ula-net -s 2001:db8:2001:a::2 -d 2001:db8:2001:b::2 -j ACCEPT
-A INPUT -i ula-net -s fd00:2001:db8::/48 -d fd00:2001:db8:b::/64 -j ACCEPT
-A OUTPUT -o ula-net -s fd00:2001:db8:b::/64 -d fd00:2001:db8::/48 -j ACCEPT

Admittedly, I'm not 100% sure the FORWARD rules are required. If they are, it might be an issue if trying to tunnel the ULA traffic inside a 6-in-4 tunnel. Presumably using a 6in4 tunnel alone (i.e. IPv4 tunnel endpoints) would require forwarding protocol 41 packets from the router. With a more specific 6to4 forwarding rule in the router based on source address (foreign_addr for my router's nat rules) it should be possible to forward 6in4 (Hurricane Electric) traffic to one device, and 6in4 (ULA network) traffic to another device.

DHCP For ULA Prefix

Finally, hand out ULA addresses to LAN devices. I use RADVD on my pi:

interface eth0
{
        AdvSendAdvert on;
        MaxRtrAdvInterval 120;
        AdvLinkMTU 1480;
        AdvOtherConfigFlag on;
        prefix 2001:db8:2001:b::/64
        {
                AdvOnLink on;
                AdvAutonomous on;
                AdvRouterAddr on;
                AdvValidLifetime 300;
                AdvPreferredLifetime 120;
                DeprecatePrefix on;
        };
        prefix fd00:2001:db8:b::/64
        {
                AdvOnLink on;
                AdvAutonomous on;
                AdvRouterAddr on;
                AdvValidLifetime 300;
                AdvPreferredLifetime 120;
                DeprecatePrefix on;

        };
        RDNSS fd00:2001:db8:b::53 2001:db8:53:abcd:1234:5300
        {

        };
        route ::/0
        {

        };
};

Double-check that everything is now working:

C:\Users\John>ping fdd7:5938:e2e6:9660::2

Pinging fdd7:5938:e2e6:9660::2 with 32 bytes of data:
Reply from fdd7:5938:e2e6:9660::2: time=22ms
Reply from fdd7:5938:e2e6:9660::2: time=27ms
Reply from fdd7:5938:e2e6:9660::2: time=21ms
Reply from fdd7:5938:e2e6:9660::2: time=22ms

Ping statistics for fdd7:5938:e2e6:9660::2:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 21ms, Maximum = 27ms, Average = 23ms
john@vps2:~$ ping6 -c 4 fdd7:5938:e2e6:1::2
PING fdd7:5938:e2e6:1::2(fdd7:5938:e2e6:1::2) 56 data bytes
64 bytes from fdd7:5938:e2e6:1::2: icmp_seq=1 ttl=64 time=19.2 ms
64 bytes from fdd7:5938:e2e6:1::2: icmp_seq=2 ttl=64 time=19.3 ms
64 bytes from fdd7:5938:e2e6:1::2: icmp_seq=3 ttl=64 time=21.0 ms
64 bytes from fdd7:5938:e2e6:1::2: icmp_seq=4 ttl=64 time=19.1 ms

--- fdd7:5938:e2e6:1::2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 19.113/19.688/21.017/0.791 ms

Create Tunnels at Startup

As I previously stated (TODO: insert canonical link - can Google it using search term 'watfordjc ifup'), I can't change the network interfaces configuration on my VPS without also taking on full maintenance of various configuration files.

Although that is no longer the case (XenVZ/OpenITC now allow more granular control) taking control of just the network settings would require double-checking nothing needs to be updated/modified regularly. Because of that, I am going to stick with the current way of doing things for setting up my Hurricane Electric tunnels (differs on VPS and pi).

On the VPS:

sudo su
cd /etc/init/
cp he-ipv6-tunnel.conf ula-ipv6-tunnel.conf_
nano ula-ipv6-tunnel.conf_

And modify it so it looks like the following:

# ula-ipv6-tunnel
#

description     "Set-up ULA IPv6 tunnel and IPs"

start on (ipv6-tunnel-up IFACE=he-ipv6 and filesystem and net-device-up IFACE=eth0)
# Stop script probably doesn't run
stop on (net-device-down IFACE=ula-net or ipv6-tunnel-down IFACE=he-ipv6)

emits ipv6-tunnel-up
emits ipv6-tunnel-down

pre-start script
# Set-up the tunnel
ip -6 tunnel add ula-net mode ip6ip6 remote 2001:470:1f09:1aab::2 local 2001:470:1f09:38d::2 ttl 225
ip -6 link set ula-net up
ip -6 addr add fdd7:5938:e2e6:3::9660/64 dev ula-net
ip -6 route add fdd7:5938:e2e6:1::/64 dev ula-net
ip -f inet6 addr
initctl emit ipv6-tunnel-up IFACE=ula-net
end script

pre-stop script
ip -6 route del fdd7:5938:e2e6:1::/64 dev ula-net
ip -6 addr del fdd7:5938:e2e6:3::9660/64 dev ula-net
ip -6 link set ula-net down
ip -6 tunnel del ula-net
end script

post-stop script
initctl emit ipv6-tunnel-down IFACE=ula-net
end script

Save, close, and then:

mv ula-ipv6-tunnel.conf_ ula-ipv6-tunnel.conf
initctl list | grep ula
ip -6 route del fdd7:5938:e2e6:1::/64 dev ula-net
ip -6 addr del fdd7:5938:e2e6:3::9660/64 dev ula-net
ip -6 link set ula-net down
ip -6 tunnel del ula-net
service ula-ipv6-tunnel start
ifconfig -a
exit

Theoretically, the ULA tunnel will now start on reboots once (a) eth0 is up, (b) the filesystem is up, and (c) the Hurricane Electric tunnel is up. As noted by the comment that I have left in (since it is still the case) the stop script doesn't work when ifconfig ula-net down is called (and ifdown can't be used since ula-net isn't configured in network interfaces).

Not quite sure if ifconfig interface-name down emits anything. /etc/network/if-down.d/upstart suggests net-device-down IFACE=ula-net should be emitted, but even after removing the pre-down lines that might be returning non-zero if the interface is manually taken down, it still didn't seem to work. TODO: Find solution.

I previously reworked the Hurricane Electric script when I added a tunnel for netassist, so ipv6-tunnel-(up|down) is similar to net-device-(up|down) in that it contains an IFACE=x - saves me having to get creative with emit signal names.

IP6tables

Copy my ipv6tables init.d script from my pi to my old PC