Friday, July 27, 2012

The IPv6 DFZ just passed 10.000 prefixes


I noticed the IPv6 Default Free Zone just tipped over 10.000 prefixes. I think this is quite a milestone! Go IPv6!

The data was taken from the RING Looking Glass, which is part of the NLNOG RING project. The RING Looking Glass takes full BGP feeds from ± 20 organisations.

BTW, as an apples and peaches comparison: the IPv4 full routing table currently is roughly 415.000 prefixes. Live IPv4 and IPv6 graphs can be viewed here (RING) and here (Potaroo).

Wednesday, July 11, 2012

What is LISP DDT?

Some background on LISP

LISP (Locator/Identifier Separation Protocol) is a smart and novel method to create overlay networks with features such as multi-homing, mobility and VPN-segregation. These feats are possible because LISP makes a distinction between the 'who' and the 'where'.
"The separation of location and identity is a step which has recently been identified by the IRTF as a critically necessary evolutionary architectural step for the Internet."
- N. Chiappa in draft-chiappa-lisp-introduction-00 
An example would be that my IPv6 prefix 2001:67c:208c:10::/64 (the 'who') currently is located behind the following WAN IP addresses: 62.194.155.106, 217.8.107.2 and 2001:67C:21B4:1::2 (the 'where'). In this example my prefix is multi-homed behind 3 connections, and I'm doing IPv6 over IPv4 next to IPv6 over IPv6. This is possible because this single IPv6 prefix can have multiple Routing Locators (the 'where') and LISP is address-family agnostic.

Mapping systems for location information

As you can imagine, the key to protocols like LISP is locating who is where in a fast and efficient way.

To create more context: with the Border Gateway Protocol (BGP) all participating nodes (routers) have all information about everybody in memory. When an organisation announces a prefix to the internet, all routers attached to the default-free zone have to store how they can reach that prefix. This is called a "push-based" mechanism, because information is pushed towards all nodes regardless of whether they need that information. After all, not all networks have to send packets to all other networks on the planet.

The designers of LISP choose a vastly different approach: a "pull-based" mechanism. The moment a LISP node receives a packet it needs to forward to a certain destination, it will do a lookup in the mapping system where that destination is actually located. This ensures LISP routers only need to store a limited amount of information compared to routers running BGP. I imagine my CPE at home is only communicating with say 200 networks at any given time.

LISP DDT

To get a better understanding of what LISP DDT is, an analogy can be made between the DNS mapping system and LISP DDT. RFC 1034 states:
"The sheer size of the database and frequency of updates suggest that it must be maintained in a distributed manner, with local caching to improve performance. Approaches that attempt to collect a consistent copy of the entire database will become more and more expensive and difficult, and hence should be avoided."
DNS is not copying /etc/hosts around, it is an hierarchical distributed database maintained by many organisations. At the top there is the DNS root ".", and for instance the ccTLD "nl." is delegated to the Dutch Registry SIDN. Then in turn, that registry delegates a domain like "snijders-it.nl" to me.

LISP DDT stands for LISP Delegated Database Tree, it is an hierarchy for prefixes and other LISP related information. The root of the tree is authoritative for "key-ID 0, EID 0, AFI 0 and EID-prefix 0/0". This basically means 'everything'. At the root, prefixes within an address-family are delegated to Map Servers, which in turn could delegated smaller chunks of those prefixes down to other Map Servers.

Time for an example! The prefix 2001:67c:208c::/48 belongs to my organisation. I've asked the LISP DDT Root operators to delegate that prefix to certain Map Servers operators with which I have a (business) relation. These Map Servers (currently for instance 217.8.98.42 and 195.50.116.18) are authoritative for my /48, and it is to these Map Servers my LISP routers register 'who' and 'where' they are (think dynamic DNS!).

Now if some LISP router "LISP-CORP-B" wants to send packets towards a HOST-A (e.g. 2001:67c:208c:10::1) in my prefix, it will ask it's Map Resolver where to find that host. The Map Resolver will query the root where it can find more information, and the DDT Root will refer the Map Resolver for this /48 to 217.8.98.42 and 195.50.116.18. The Map Resolver will then query these Map Servers and eventually return all relevant information to the original request LISP-CORP-B.

The following image providers a simplified overview of how LISP-CORP-B learns how to reach an host at Job's office (click for larger image):



Status of LISP DDT

LISP DDT is currently being developed inside the IETF, this Internet-Draft describes how it works: draft-fuller-lisp-ddt. Cisco has developed a few implementations of LISP DDT, which are currently being used in production environments. In May 2012 the LISP Beta Network migrated from LISP-ALT to LISP-DDT and has been running smoothly ever since. The LISP DDT-ROOT is operated by  volunteers from various organisations such as Verisign, InTouch NV, Cisco and myself

Closing words

LISP DDT is the most promising mapping system in the LISP world, there is still some work to be done to develop this novel technology into a mature protocol. If you have any questions don't hesitate to contact me, I can help you assess what LISP and DDT can mean for your organisation. 


Tuesday, December 6, 2011

Example Puppet 2.7 git pre-commit script

I had a hard time finding a decent pre-commit script for puppet 2.7. This is composed of snippets I found at code.seas.harvard.edu. The pre-commit script will check the puppet syntax of the changed .pp files, and also check if an attempt has been made to properly document the .pp file.


#!/bin/sh
#
# install this as .git/hooks/pre-commit to check Puppet manifests
# for errors before committing changes.

rc=0

[ "$SKIP_PRECOMMIT_HOOK" = 1 ] && exit 0

# Make sure we're at top level of repository.
cd $(git rev-parse --show-toplevel)

trap 'rm -rf $tmpdir $tmpfile1 $tmpfile2' EXIT INT HUP
tmpdir=$(mktemp -d precommitXXXXXX)
tmpfile1=$(mktemp errXXXXXX)
tmpfile2=$(mktemp errXXXXXX)

echo "$(basename $0): Validating changes."

# Here we copy files out of the index into a temporary directory. This
# protects us from a the situation in which we have staged an invalid
# configuration using ``git add`` but corrected the changes in the
# working directory. If we checked the files "in place", we would
# fail to detect the errors.

git diff-index --cached --name-only HEAD |
grep '\.pp$' |
git checkout-index --stdin --prefix=$tmpdir/

find $tmpdir -type f -name '*.pp' |
while read manifest; do
puppet parser validate $manifest | sed "s#$tmpdir/##" >> $tmpfile1 2>&1

if ! head -1 $manifest | grep -q '^#'; then
echo $manifest | sed "s#$tmpdir/##" >> $tmpfile2
fi
done

if [ -s "$tmpfile1" ]; then
echo
echo Error: Puppet parse problem:
echo ----------------------------
cat $tmpfile1
echo ----------------------------
echo

rc=1
fi

if [ -s "$tmpfile2" ]; then
echo
echo Error: missing manifest documentation
echo see http://projects.puppetlabs.com/projects/puppet/wiki/Puppet_Manifest_Documentation
echo for more information.
echo Files with problems:
echo -------------------------------------
cat $tmpfile2
echo -------------------------------------
echo

rc=1
fi

exit $rc

Monday, April 4, 2011

LISP is serious business

Some perspective with regards to the maturity of the protocol specifications and implementations of the Locator/Identifier Separation Protocol (LISP):

The first LISP specification was published in January 2007 as an individual submission. After 13 revisions the Internet-Draft was adopted as an IETF LISP Working Group document. Within the LISP Working Group there have been 12 versions of the main Internet-Draft. Literally hundreds of contributors from a lot of different companies have made suggestions and fixed bugs to make the LISP specification what it is today.

The first implementation started at the Prague IETF conference in 2007. As of today there are about 10 implementations: Linux, Android, FreeBSD (OpenLISP), Zlisp, LISP-Click, FNSC FITELnet G21, IOS, NX-OS, IOS-XR and IOS-XE. Please note that not all of them have yet been released or are of production quality. I recommend using the implementations developed by Cisco because they are the most mature and feature rich implementations.

Better yet, recently Cisco announced to the world its first production software releases 15.1(4)M and 15.1(2)S which support the Locator/Identifier Separation Protocol (LISP). Cisco has committed to make LISP, as an emerging standard, available on all its major routers and switches in mainstream trains including the Nexus 7000.

So what does all this mean?

The LISP specification has received a lot of attention and review inside the IETF. This is a very good thing because it makes the protocol stronger and development less prone to for example tunnel vision. The academic world has developed various proof-of-concept implementations to discover its scalability properties and other testing purposes. Unlike other protocols such as ILNP, HIP, SHIM, and IRON-RANGER which have received significantly less review or have almost no real-world deployment experience.

Big vendors are investing lots of resources into the development of their respective LISP implementations, which means LISP is here to stay.

Tuesday, March 15, 2011

Nagios and IPv6 made easy with the mknagconf configuration generator

This article describes how to install Nagios3 and my mknagconf tool and how to use them. It should take about 30 minutes to install nagios3 and mknagconf and set it up to monitor a few hosts. The following has been tested with Ubuntu 10.04, 10.10 and 11.04 on an amd64 platform.

Nagios3 is an excellent monitoring engine, but the stock Nagios has some limitations in regard to dual-stack hosts. In the Nagios universe, one host is one ip address, and a secondary ipv6 address would require an extra host definition.

The Nagios packages which you are about to install have been patched to support this concept "one host = 1 ipv4 address + 1 optional ipv6 address". The mknagconf script makes it easier to maintain your Nagios installation. mknagconf takes small short, and simple definition files, parses them and generate the configuration files for Nagios. This will be explained after installing the required software.

Step 1: Install all dependencies
apt-get install apache2-mpm-prefork apache2-utils apache2.2-bin \
 apache2.2-common bsd-mailx libapache2-mod-php5 libapr1 \
 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap \
 libgd2-noxpm libjpeg62 libperl5.10 nagios-plugins-basic \
 php5-common postfix ssl-cert nagios-plugins-standard \
 nagios-plugins-extra git-core make
Step 2: obtain PGP key, configure apt to fetch updates from my repo

Add my PGP key to your apt setup and install the Nagios packages from my repository:
apt-key adv --keyserver keyserver.stack.nl --recv-keys 7E5BEC10
echo "deb http://snijders-it.nl/deb natty main" \
 > /etc/apt/sources.list.d/snijders-it.list
apt-get update
Step 3: install nagios

During the following step the Nagios installation will ask you for a password for the user 'nagiosadmin'. Don't forget to use a strong password!
apt-get install nagios3 nagios3-core nagios3-doc \
    nagios3-common nagios3-cgi
Step 4: download and install mknagconf

This step downloads the latest version of the script and example configuration files from github. After downloading you can run the ./install.sh script to copy all files to the correct place. This script will overwrite the current nagios configuration from Step 3.
cd /usr/srcgit clone git://github.com/SnijdersIT/mknagconf.git
cd mknagconf
sudo ./install.sh
Step 5 (optional): setup local repo for configs

It's usually good to do nagios stuff in some kind of repository, an example would be:
cd /etc/nagios3git init
git config --global user.name "your name"
git config --global user.email you@example.net
echo "conf.d/*_autogenerated.cfg" >> .git/info/exclude
git add *
git commit -am 'first nagios config'
Step 6: edit the hosts.def definition file

Open the following file in your texteditor:
/etc/nagios3/definitions/hosts.def
You will see a line like:
hpserv;192.168.23.4;2001:db8:1::65;coregate;central HP server

Which will be interpreted as: host 'hpserv' with ipv4 address "192.168.23.4" and ipv6 address "2001:db8:1::65", has 1 parent namely: "coregate" and is also called the "central HP server".

The ipv6 address, parents and human name are optional. You can specify multiple parents if you separate them with a comma. Don't forget to create a line specifying the parent itself!

Remove the hpserv line and add a (dual-stack) host you would like to monitor. Define one host per line. In the next step you can define what services you want to monitor on that host.

Step 7: edit the services.def definition file

Open the following file in your texteditor:
/etc/nagios3/definitions/services.def
You will see a line like:
hpserv = ping, ping6, ftp, ssh6
Which will be interpreted as: perform an IPv4 ping check, IPv6 ping check, an FTP connect over IPv4 and a SSH connect over IPv6 to host "hpserv". Define one host per line, and all services that should be checked on that host on the same line.

Step 8: configure correct contact

Open the following file and modify it according to your situation:
/etc/nagios3/conf.d/contacts.cfg
Step 9: generate configuration, reload nagios

Change to the nagios3 configuration directory, run the make commands to clean up the auto-generated files, compile the nagios configurations from the definition files and check if the configurations makes any sense.
cd /etc/nagios3
make cleanmake
make check && /etc/init.d/nagios3 reload
git commit -am 'added my first hosts' # optional step

For more information open the various files in /etc/nagios3/definitions/ and read their headers. Further recommended reading: the README file in /usr/src/mknagconf/

If at some later point you want to add more hosts or delete old ones, just go through step 6, 7 and 9.

That's it! Have fun using Nagios :-)

Tuesday, January 18, 2011

How Network Operators can cooperate: the NLNOG RING

In December 2010 I started a project with a few friends to make life for network engineers in the Netherlands better.

I noticed that there are a lot of friendly 'shell access' exchange deals between dutch network operators. This makes it easier for parties to debug network issues and troubleshoot from the outside. A point of view outside your network is absolutely essential, seeing what others see is a useful thing for a variety of network problems. Well known examples are "it works for even numbered ip address, but not for odd numbered ip address via this and this route".

The NLNOG RING tries to do this in a more organized way, basically the deal is "donate 1 machine, and gain access to all other machines in the ring". So far already 10 organisations are participating.

How useful is the ring exactly? A very nice example is executing a traceroute from ten different autonomous systems: nlnog ring example.

More information about the NLNOG RING can be found on the website we've launched today: ring.nlnog.net.

Wednesday, December 22, 2010

New Cisco IOS releases in an RSS feed

Over the last few days we've been spending some time on an RSS feed generator which can help you stay on top of new IOS releases. It takes regular expressions as input and can be useful for a quick search or generating an RSS feed for your favorite news reader.

The database is built upon a public md5 database Cisco publishes roughly every week. I cannot vouch for the accuracy of this information.

You can find the quick search & rss generator at http://snijders-it.nl/ios-rss/.