I haven't blogged for a long time, but I've decided that I'm going to try to write again, at least about technical stuff.

My plan was to blog about the projects I've been working on lately, the main one being the setup of the latest version of Kolab with the systems we already have at work, but I'll do that on the next days.

Today I'm just going to make a list of the tools I use on a daily basis and my plans to start using additional ones in the near future.

Shells, Terminals and Text Editors

I do almost all my work on Z Shell sessions running inside tmux; for terminal emulation I use gnome-terminal on X, VX ConnectBot on Android systems and iTerm2 on Mac OS X.

For text editing I've been using Vim for a long time (even on Mobile devices) and while I'm aware I don't know half of the things it can do, what I know is good enough for my day to day needs.

In the past I also used Emacs as a programming editor and my main tool to write HTML, SGML and XML, but since I haven't really needed an IDE for a long time and I mainly use Lightweight Markup Languages I haven't used it for a long time (I briefly tried to use Org mode, but for some reason I ended up leaving it).

Documentation formats and tools

Since a long time ago I've been an advocate of Lightweight Markup Languages; I started to use LaTeX and Lout, then moved to SGML/XML formats (LinuxDoc and DocBook) and finally moved to plain text based formats.

I started using Wiki formats (parsewiki) and soon moved to reStructuredText; I also use other markup languages like Markdown (for this blog, aka ikiwiki) and tried MultiMarkdown to replace reStructuredText for general use, but as I never liked Markdown syntax I didn't liked an extended version of it.

While I've been using ReStructuredText for a long time, I recently found Asciidoctor and the Asciidoc format and I guess I'll be using it instead of rst whenever I can (I still need to try the slide backends and conversions to ODT, but if that works I guess I'll write all my new documents using Asciidoc).

Programming languages

I'm not a developer, but I read and patch a lot of free software code written on a lot of different programming languages (I wouldn't be able to write whole programs on most of them, but thanks to Stack Overflow I'm usually able to fix what I need).

Anyway, I'm able to program in some languages; I write a lot of shell scripts and I go for Python and C when I need something more complicated.

On the near future I plan to read about javascript programming and nodejs (I'll probably need it at work) and I already started looking at Haskell (I guess it was time to learn about functional programming and after reading about it, it looks like haskell is the way to go for me).

Version Control

For a long time I've been a Subversion user, at least for my own projects, but seems that everything has moved to git now and I finally started to use it (I even opened a github account) and plan to move all my personal subversion repositories at home and at work to git, including the move of all my debian packages from svn-buildpackage to git-buildpackage.

Further Reading

With the previous plans in mind, I've started reading a couple of interesting books:

• Learn You a Haskell by Miran Lipovača (http://learnyouahaskell.com/)
• Pro Git written by Scott Chacon and Ben Straub (http://git-scm.com/book/en/v2)

Now I just need to get enough time to finish reading them ... ;)

The last month I was supposed to work on a OpenStack related project, but for administrative reasons it has been delayed and I've tried to do small tasks to be able to finish them quickly and start the work on the main project when the issues get solved.

As the delay has been longer than expected last Wednesday I've realized than on the last weeks I did a lot of small system administration tasks:

• With a co-worker I started to work on a GNU/Linux version of our firewall based on Shorewall to handle the rules and conntrackd and keepalived to make it highly available (I had to stop my work on the Debian GNU/kFreeBSD based firewall a long time ago, and this summer the old firewalls' hardware started to fail, so a migration from Linux to Linux makes sense now, as it will be faster and a future migration will be simpler, as we will have a cleaner set of rules and better documentation),
• I installed and configured an instance of a web based File Exchange server (F*EX),
• I installed and configured an instance of a pastebin clone,
• I installed and configured an instance of ProFTPD that works only as a SFTP server using virtual users (without shell access),
• I also installed an instance of a web based event management system called indico that is being used to manage a conference and probably will be used for other events in the future,
• I installed and patched some plugins in our Trac servers,
• I tried a groupware system called SOGo that we will probable deploy in a week or two,
• And updated and fixed configurations of some other services,

With all the changes I did I noticed that I had to do something with our Intranet server; it is just a reverse proxy for a lot of different web services and its main page was one static HTML page with links to them, nothing else.

In the long term maybe we will replace it with something based on Drupal or Lifeay, but for now I just wanted something to be able to organize the links and provide some information about the services for the new users without having to write HTML (I really like Agile Documentation Tools that let me focus on the content and forget about the markup), and started to look at some of them.

My first idea was to use ikiwiki, as it has all the features I was looking for: I can use Markdown or reStructuredText to write the contents, the source pages are easily handled on a Version Control System, it supports the use of templates for the HTML, etc., but it seemed to me that using ikiwiki was like killing flies with a cannon (that's a Spanish say, I guess it's easy to understand it in English, ¿no?) and I decided to review other tools to build static web sites.

To make a long story short, I selected some tools that met my requirements and looked nice on their demo sites; after my first review I thought that Hyde was going to be my bet, as it uses technologies I'm already familiar with, but after trying it I saw that I was going to have a problem with documentation (the current Hyde version lacks it) and it was going to be more complicated that using ikiwiki.

Before giving up I decided to review simpler tools, just in case, and after looking some of them I ended up using poole, a simple python script (the source is just one file and it only requires python-markdown to work).

Before moving to the content I tried to adapt a couple of free themes to be used by the tool, but I didn't liked the result, so I went back to the plain style provided by the tool and added a logo and a background.

With that simple look and feel I started to work with the content, splitting it into eight markdown files and a python macro to include a file that has all the links used on the site.

While trying to make the main page look good I noticed how little I know about CSS, but using search engines I was able to build a two column block into the main page and publish the contents and with the help of some CSS enabled co-workers I changed the look and feel of the site in about 30 minutes.

In summary, if you want a really simple website, you know a little bit of python and don't want to spend much time learning how to use a website generator then Poole is a good option. If you want something more complex I still think that ikiwiki is a good option, but YMMV.

In this post I'll describe the changes made to the kernel and some of the Squeeze packages for the Freaky Wall.

The plan is to submit whishlist bugs to the BTS on the hope of having all what is needed for this project available on Debian after the Squeeze release, as my feeling is that a freeze is not the right time to push this changes... ;)

I'm giving access here to all the changes made to the source packages, but if anyone wants the binary packages (amd64 only) send me an email and I'll give you the URL of an apt repository that contains all the modified packages (it's the one at work, that contains other modified packages) or, if there is interest, I can put them on people.debian.org.

Kernel

To be able to build the firewall we need a kFreeBSD kernel with some options not compiled on the version distributed with Debian.

To compile the kernel I've followed the procedure described on the following debian-bsd mailing list post:

http://lists.debian.org/debian-bsd/2010/09/msg00023.html

Basically I've done the following:

    apt-get build-dep kfreebsd-8
    apt-get source kfreebsd-8
    cd kfreebsd-8-8.1
    cat >> debian/arch/amd64/amd64.config << EOF

    # Add pflog, pfsync, ALTQ and CARP support
    # ----------------------------------------

    # http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/firewalls-pf.html
    device      pf
    device      pflog
    device      pfsync

    options         ALTQ
    options         ALTQ_CBQ        # Class Bases Queuing (CBQ)
    options         ALTQ_RED        # Random Early Detection (RED)
    options         ALTQ_RIO        # RED In/Out
    options         ALTQ_HFSC       # Hierarchical Packet Scheduler (HFSC)
    options         ALTQ_PRIQ       # Priority Queuing (PRIQ)
    options         ALTQ_NOPCC      # Required for SMP build

    # http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/carp.html
    device      carp
    EOF
    vi debian/changelog 
    dpkg-buildpackage -B -uc

Once the package was built I installed the new kernel package and rebooted the machine.

Utilities

To be able to use some utilities related to pf I have built patched versions of three packages:

• freebsd-utils: I have included pflogd and ftp-proxy on the package and have added some options to allow the use of additional interface types with ifconfig (carp, pfsync, lagg, bridges, ...).

  There were a lot of changes needed:

  1. The pflogd and ftp-proxy Makefiles are missing from the original tarball, I modified the get-orig-source of the debian/rules, but I build my packages against the original tarball, with the extra files included directly on the package .diff.gz.

  2. The pflogd daemon needs a _pflogd user and group and a /var/run/_pflogd directory, so I've added the directory and the creation of the user and group to the package post-install.

  3. The ftp-proxy daemon uses the proxy user when dropping privileges and I've modified the code to make it use the /var/run/ftp-proxy directory for the chroot.

  4. Some C header files that are not available on -dev packages were missing and I put them on the packages' debian directory. I've created a debian/include directory and moved there the original debian/net directory and added the headers debian/include/net/if_bridgevar.h and debian/include/net/if_lagg.h to add bridging support to ifconfig and the debian/include/pcap-config.h and debian/include/pcap-int.h libpcap private headers to be able to compile the pflogd binary.

  As I'm not familiar with the way people changes code for kFreeBSD some of the patches are a little bit dirty, but at least things work; besides, probably I should also have had to include init.d scripts for pf, pflogd and ftp-proxy, but I have not done it at the package level yet, as what I got was enough to work with the tools.

  The debdiff against the freebsd-utils-8.1-2 source package is available here or here.

• libpcap: A test had to be removed in order to be able to support pflog on the library; the debdiff against the libpcap-1.1.1-2 package is available here or here.

• tcpdump: The package also had to be modified to include the code to print the pflogd entries on the pcap file; the debdiff against tcpdump_4.1.1-1 is available here or here.

On the next post I'll describe how I've configured the system, the network interfaces and the different utilities patched and compiled on this post.