Tuesday, December 6, 2011

Example Puppet 2.7 git pre-commit script

I had a hard time finding a decent pre-commit script for puppet 2.7. This is composed of snippets I found at code.seas.harvard.edu. The pre-commit script will check the puppet syntax of the changed .pp files, and also check if an attempt has been made to properly document the .pp file.

# install this as .git/hooks/pre-commit to check Puppet manifests
# for errors before committing changes.


[ "$SKIP_PRECOMMIT_HOOK" = 1 ] && exit 0

# Make sure we're at top level of repository.
cd $(git rev-parse --show-toplevel)

trap 'rm -rf $tmpdir $tmpfile1 $tmpfile2' EXIT INT HUP
tmpdir=$(mktemp -d precommitXXXXXX)
tmpfile1=$(mktemp errXXXXXX)
tmpfile2=$(mktemp errXXXXXX)

echo "$(basename $0): Validating changes."

# Here we copy files out of the index into a temporary directory. This
# protects us from a the situation in which we have staged an invalid
# configuration using ``git add`` but corrected the changes in the
# working directory. If we checked the files "in place", we would
# fail to detect the errors.

git diff-index --cached --name-only HEAD |
grep '\.pp$' |
git checkout-index --stdin --prefix=$tmpdir/

find $tmpdir -type f -name '*.pp' |
while read manifest; do
puppet parser validate $manifest | sed "s#$tmpdir/##" >> $tmpfile1 2>&1

if ! head -1 $manifest | grep -q '^#'; then
echo $manifest | sed "s#$tmpdir/##" >> $tmpfile2

if [ -s "$tmpfile1" ]; then
echo Error: Puppet parse problem:
echo ----------------------------
cat $tmpfile1
echo ----------------------------


if [ -s "$tmpfile2" ]; then
echo Error: missing manifest documentation
echo see http://projects.puppetlabs.com/projects/puppet/wiki/Puppet_Manifest_Documentation
echo for more information.
echo Files with problems:
echo -------------------------------------
cat $tmpfile2
echo -------------------------------------


exit $rc

Monday, April 4, 2011

LISP is serious business

Some perspective with regards to the maturity of the protocol specifications and implementations of the Locator/Identifier Separation Protocol (LISP):

The first LISP specification was published in January 2007 as an individual submission. After 13 revisions the Internet-Draft was adopted as an IETF LISP Working Group document. Within the LISP Working Group there have been 12 versions of the main Internet-Draft. Literally hundreds of contributors from a lot of different companies have made suggestions and fixed bugs to make the LISP specification what it is today.

The first implementation started at the Prague IETF conference in 2007. As of today there are about 10 implementations: Linux, Android, FreeBSD (OpenLISP), Zlisp, LISP-Click, FNSC FITELnet G21, IOS, NX-OS, IOS-XR and IOS-XE. Please note that not all of them have yet been released or are of production quality. I recommend using the implementations developed by Cisco because they are the most mature and feature rich implementations.

Better yet, recently Cisco announced to the world its first production software releases 15.1(4)M and 15.1(2)S which support the Locator/Identifier Separation Protocol (LISP). Cisco has committed to make LISP, as an emerging standard, available on all its major routers and switches in mainstream trains including the Nexus 7000.

So what does all this mean?

The LISP specification has received a lot of attention and review inside the IETF. This is a very good thing because it makes the protocol stronger and development less prone to for example tunnel vision. The academic world has developed various proof-of-concept implementations to discover its scalability properties and other testing purposes. Unlike other protocols such as ILNP, HIP, SHIM, and IRON-RANGER which have received significantly less review or have almost no real-world deployment experience.

Big vendors are investing lots of resources into the development of their respective LISP implementations, which means LISP is here to stay.

Tuesday, March 15, 2011

Nagios and IPv6 made easy with the mknagconf configuration generator

This article describes how to install Nagios3 and my mknagconf tool and how to use them. It should take about 30 minutes to install nagios3 and mknagconf and set it up to monitor a few hosts. The following has been tested with Ubuntu 10.04, 10.10 and 11.04 on an amd64 platform.

Nagios3 is an excellent monitoring engine, but the stock Nagios has some limitations in regard to dual-stack hosts. In the Nagios universe, one host is one ip address, and a secondary ipv6 address would require an extra host definition.

The Nagios packages which you are about to install have been patched to support this concept "one host = 1 ipv4 address + 1 optional ipv6 address". The mknagconf script makes it easier to maintain your Nagios installation. mknagconf takes small short, and simple definition files, parses them and generate the configuration files for Nagios. This will be explained after installing the required software.

Step 1: Install all dependencies
apt-get install apache2-mpm-prefork apache2-utils apache2.2-bin \
 apache2.2-common bsd-mailx libapache2-mod-php5 libapr1 \
 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap \
 libgd2-noxpm libjpeg62 libperl5.10 nagios-plugins-basic \
 php5-common postfix ssl-cert nagios-plugins-standard \
 nagios-plugins-extra git-core make
Step 2: obtain PGP key, configure apt to fetch updates from my repo

Add my PGP key to your apt setup and install the Nagios packages from my repository:
apt-key adv --keyserver keyserver.stack.nl --recv-keys 7E5BEC10
echo "deb http://snijders-it.nl/deb natty main" \
 > /etc/apt/sources.list.d/snijders-it.list
apt-get update
Step 3: install nagios

During the following step the Nagios installation will ask you for a password for the user 'nagiosadmin'. Don't forget to use a strong password!
apt-get install nagios3 nagios3-core nagios3-doc \
    nagios3-common nagios3-cgi
Step 4: download and install mknagconf

This step downloads the latest version of the script and example configuration files from github. After downloading you can run the ./install.sh script to copy all files to the correct place. This script will overwrite the current nagios configuration from Step 3.
cd /usr/srcgit clone git://github.com/SnijdersIT/mknagconf.git
cd mknagconf
sudo ./install.sh
Step 5 (optional): setup local repo for configs

It's usually good to do nagios stuff in some kind of repository, an example would be:
cd /etc/nagios3git init
git config --global user.name "your name"
git config --global user.email you@example.net
echo "conf.d/*_autogenerated.cfg" >> .git/info/exclude
git add *
git commit -am 'first nagios config'
Step 6: edit the hosts.def definition file

Open the following file in your texteditor:
You will see a line like:
hpserv;;2001:db8:1::65;coregate;central HP server

Which will be interpreted as: host 'hpserv' with ipv4 address "" and ipv6 address "2001:db8:1::65", has 1 parent namely: "coregate" and is also called the "central HP server".

The ipv6 address, parents and human name are optional. You can specify multiple parents if you separate them with a comma. Don't forget to create a line specifying the parent itself!

Remove the hpserv line and add a (dual-stack) host you would like to monitor. Define one host per line. In the next step you can define what services you want to monitor on that host.

Step 7: edit the services.def definition file

Open the following file in your texteditor:
You will see a line like:
hpserv = ping, ping6, ftp, ssh6
Which will be interpreted as: perform an IPv4 ping check, IPv6 ping check, an FTP connect over IPv4 and a SSH connect over IPv6 to host "hpserv". Define one host per line, and all services that should be checked on that host on the same line.

Step 8: configure correct contact

Open the following file and modify it according to your situation:
Step 9: generate configuration, reload nagios

Change to the nagios3 configuration directory, run the make commands to clean up the auto-generated files, compile the nagios configurations from the definition files and check if the configurations makes any sense.
cd /etc/nagios3
make cleanmake
make check && /etc/init.d/nagios3 reload
git commit -am 'added my first hosts' # optional step

For more information open the various files in /etc/nagios3/definitions/ and read their headers. Further recommended reading: the README file in /usr/src/mknagconf/

If at some later point you want to add more hosts or delete old ones, just go through step 6, 7 and 9.

That's it! Have fun using Nagios :-)

Tuesday, January 18, 2011

How Network Operators can cooperate: the NLNOG RING

In December 2010 I started a project with a few friends to make life for network engineers in the Netherlands better.

I noticed that there are a lot of friendly 'shell access' exchange deals between dutch network operators. This makes it easier for parties to debug network issues and troubleshoot from the outside. A point of view outside your network is absolutely essential, seeing what others see is a useful thing for a variety of network problems. Well known examples are "it works for even numbered ip address, but not for odd numbered ip address via this and this route".

The NLNOG RING tries to do this in a more organized way, basically the deal is "donate 1 machine, and gain access to all other machines in the ring". So far already 10 organisations are participating.

How useful is the ring exactly? A very nice example is executing a traceroute from ten different autonomous systems: nlnog ring example.

More information about the NLNOG RING can be found on the website we've launched today: ring.nlnog.net.