I use a Qnap NAS as a central network storage for backing up mulitple Macs and therefore I want to define strict limits for the maximum size of each Time Machine backup. So after noticing the new image sizes, I tried to resize them with hdiutil:

hdiutil resize -size 500g MYBUNDLENAME.sparsebundle

Unfortunately Time Machine resizes the image size every time it runs, so it is necessary to prevent it from resizing the image.

The image size of a sparsebundle is stored in the Info.plist file inside the bundle directory, so I tried to remove write permissions to that file from the terminal:

cd /Volumes/MYSHARE/
chmod a-w MYBUNDLENAME.sparsebundle/Info.*

This wasn’t working either, because OSX seems to reset the permissions to Info.plist automatically.

The solution which is finally working for me was to login to my Qnap box via SSH and change the permissions there via chmod:

chmod a-w MYBUNDLENAME.sparsebundle/Info.*

Now, when Time Machine starts, it tries to resize the image but fails, as the Qnap server is preventing any changes to the Info.plist file. You can see this behavior in the system.log:

23.05.10 22:29:13	com.apple.backupd[378] Resizing backup disk image from 500.0 GB to 989.8 GB
23.05.10 22:29:13	com.apple.backupd[378] Could not resize backup disk image (DIHLResizeImage returned 35)

After this logging message, the rest of the backup runs fine. For me this is a nice workaround until there is an official way to limit the backup size.

Please take a look: See You, SQL – Hello Hadoop

But first, why do I want to switch from SVN to Mercurial? Basically the most appealing argument for me was the fact, that with Mercurial I am able to work offline with my repository. Besides that, I always had issues with the way SVN was handling tags and branches. Especially merging changes from a branch back into the trunk was always a pain. I did not need to use that functionality often but when I did, I always ended up doing it twice, because I could not remember which way to do it right.

So, in this article I will describe my setup of Mercurial served via FastCGI behind the nginx webserver. The approach is similar to the integration of PHP into nginx. You need the spawn-fcgi tool from the lighttpd distribution. The following steps should work on a recent Debian or Ubuntu distribution.

The first step is to install Mercurial and some necessary libraries to create the fcgi-wrapper for Mercurial:

aptitude install mercurial python-flup

Now you can already start defining your Mercurial repositories. Here are some steps to create a small example repository:

cd /tmp
hg init hgtest
cd hgtest
echo "Hello world." > readme.txt
hg add readme.txt
hg commit -m "Initial commit"

You now have a Mercurial repository with a single file. The command hg log should show you a single changeset with our commit comment.

Now, let’s start configuring nginx to integrate Mercurial. We will use FastCGI talk connect Mercurial to nginx. User authentication will be done via nginx. It is a good idea to use HTTPS for communication with Mercurial, but we will focus on a standard HTTP setup.

Let’s setup a virtual host for Mercurial. Open /etc/nginx/sites-available/your_domain_name and add the following server definition:

server {
        listen 80;
        server_name YOUR_MERCURIAL_DOMAIN;
 
        location / {
                auth_basic "Secure Login";
                auth_basic_user_file /tmp/mercurial_users;
                fastcgi_pass 127.0.0.1:9001;
                fastcgi_param SCRIPT_FILENAME /tmp$fastcgi_script_name;
                fastcgi_param PATH_INFO $uri;
                fastcgi_param REMOTE_USER $remote_user;
                include fastcgi_params;
        }       
}

The above settings will setup a new virtual host, where all traffic is redirected to the Mercurial FastCGI wrapper. It is important that you forward the PATH_INFO and REMOTE_USER variables. Mercurial will not work correctly without these.

Now reload the nginx configuration:

/etc/init.d/nginx reload

And create the password file:

htpasswd -c /tmp/mercurial_users MYLOGIN

Mercurial uses a central configuration file. In this file we can specify locations for our mercurial repositories. The following file will enable all mercurial repositories found in the /tmp directory. It also changes the theme to gitweb which is a bit clearer than the default theme. Create the file /tmp/hgweb.config with the following contents:

[collections]
/tmp = /tmp
 
[web]
style = gitweb
baseurl =

Mercurial uses a second configuration file for each repository where you may specify details about the repository and security settings like who may push changes into the repository. The configuration should be placed into the file /tmp/hgtest/.hg/hgrc and could look like this:

[web]
contact = YOUR NAME
description = DESCRIPTION OF PROJECT
style = gitweb
push_ssl = false
allow_archive = bz2 gz zip
allow_push = LOGIN_NAME

Now we need to grab the following script from the Mercurial repository and place it into the /tmp directory: http://selenic.com/repo/hg/raw-file/tip/contrib/hgwebdir.fcgi. Now edit this file and replace the line WSGIServer(hgwebdir('hgweb.config')).run() with WSGIServer(hgwebdir('/tmp/hgweb.config')).run().

The last step is to set the correct filesystem rights for your repository:

chown -R www-data.www-data /tmp/hgtest

That’s it. Now we can start the FastCGI process via:

spawn-fcgi -a 127.0.0.1 -p 9001 -u www-data -g www-data -f /tmp/hgwebdir.fcgi -P /var/run/fastcgi-mercurial.pid -C 1

It makes sense to write the above line into /etc/rc.local so that it will start up automatically when you reboot the server.

Further information about configuring your Mercurial server can be found in the Mercurial Wiki.

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
 
RewriteCond %{REQUEST_URI} !(\.gif)|(\.jpg)|(\.png)|(\.css)|(\.js)|(\.php)$
 
RewriteCond %{REQUEST_URI} ^(.*)$
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule .* sapphire/main.php?url=%1&%{QUERY_STRING} [L]
</IfModule>

So every file which does not end in .gif, .jpg, .png, .css, .js and .php and where the file does not exist will be rewritten.

I chose a somehow stripped down version of these rules which looks in Nginx notation like this:

if (!-f $request_filename) {
    rewrite ^/(.*?)(\?|$)(.*)$ /sapphire/main.php?url=$1&$3 last;
}

If a requested file is not found, the rewriting engine will parse the request string for all elements before a ‘?’. This substring will be pasted as the url parameter to main.php. Everything after ‘?’ will be added as additional parameters. This rewrite rule seems to be working and I haven’t encountered any problems so far.

One option is to move the SSH daemon to a non-standard port. But this means that you might get problems connecting yourself to the server if you are working from a restricted network. So another solution would be to use certificates for login. But then you need to make sure that you carry the certificates with you when you want to login to your server.

Now a good solution is to limit access to the SSH server. One way would be to use the so called port-knocking approach. Here the access to the SSH port is blocked until you use some kind of secret knock-sequence. Then the port will be unblocked for your IP for a certain time. This is very effective but has the downside that you always need to use this knock mechanism before connecting to your server.

What I prefer is a mechanism which works the other way around. The access to the SSH port is open until there are a number of failed login attempts detected. If this is the case, the IP address these login attempts came from will be blocked for a couple of hours. This approach is less secure then the port knocking approach but is a lot more convenient for me. As long is I don’t mess up multiple times with the login, I do not even notice any security restrictions. Brute force attacks on the other hand are blocked right away.

The most common tool for this task is the excellent fail2ban. On a Debian system it can be installed via aptitude:

aptitude install fail2ban

The default configuration is already useful. It will secure the SSH daemon with a blocking time of 10 minutes after 6 failed login attempts. The configuration files can be found in the directory /etc/fail2ban.

You can change the blocking time in the file /etc/fail2ban/jail.conf:

[DEFAULT]
bantime  = 7200
maxretry = 4

This will change the default settings which apply if not specified otherwise in the application settings. The setting bantime is specified in seconds and defines how long the blocked IP will not be able to connect to the blocked service. maxretry is the number of failed login attempts.

Lets take a look at the ssh setting:

[ssh]
enabled = true
port    = ssh
filter  = sshd
logpath  = /var/log/auth.log
maxretry = 3

You can enable or disable the monitoring of specific services (ports). The ssh monitoring uses the auth.log file to detect failed login attempts.

To activate changed configuration settings, just reload the service:

/etc/init.d/fail2ban force-reload

So what happens when an IP is blocked? Once the login attempt limit has been reached, the IP address will be blocked via iptables. Here is the output of iptables on my system:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
fail2ban-ssh  tcp  --  0.0.0.0/0            0.0.0.0/0           multiport dports 22
 
Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
 
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
 
Chain fail2ban-ssh (1 references)
target     prot opt source               destination
DROP       all  --  123.456.789.012        0.0.0.0/0
RETURN     all  --  0.0.0.0/0            0.0.0.0/0

I have replaced the blocked IP address with some default numbers, but you can see that all traffic from this IP address to the port 22 will be blocked. This iptables rule will be automatically removed after the specified blockout time.

fail2ban is an effective tool to lock out brute force login attempts. It is really easy to setup and can be used to monitor multiple services besides SSH.

First you could try to install the default Debian package to check whether you need to recompile PureFTPD. This can be done via:

apt-get install pure-ftpd-common pure-ftpd

The following steps are needed to recompile the package with the necessary options to make it run on a virtual server. You only need to do these steps if you find the following message in your syslog when trying to connect to the ftp server:

 pure-ftpd: (?@?) [ERROR] Unable to switch capabilities : Operation not permitted

In that case, make sure that you have enabled the Debian source repositories in your /etc/apt/sources.list and fetch the source files via:

apt-get source pure-ftpd

The source files have now been downloaded to your current directory. There should now some new files and a directory pure-ftpd-1.0.21. Enter this directory and edit the file debian/rules you should change the line starting with optflags and add --without-capabilities to that line, so that it looks like:

optflags=--with-everything --with-largefile --with-pam --with-privsep --with-tls --without-capabilities

Now grab the dependencies needed to build the source and compile the Debian installation package:

apt-get build-dep pure-ftpd
dpkg-buildpackage -uc -b

The resulting Debian package should now have been built without errors and you can install it via:

dpkg -i ../pure-ftpd_1.0.1-8_i386.deb

The filename of the package might vary depending on the current patch level in the Debian repository.

When installing the package you will be asked whether to use a chrooted setup which you should do and if you prefer a standalone installation or inetd-based operation. For low volume sites I would prefer the inetd installation.

Now let’s get on to the configuration of PureFTPD. I prefer a setup using virtual users which are mapped to a central user account on the server. So I usually create a new account without a shell or home directory which belongs to the Apache user group to have access to hosted webs:

useradd -g www-data -d /dev/null -s /bin/false ftpuser

The configuration of PureFTP is done via distinct files in the directory /etc/pure-ftpd/conf. Each file contains a single configuration setting and we can setup a decent configuration with the following commands:

cd /etc/pure-ftpd/conf
echo yes > ChrootEveryone
echo no > PAMAuthentication
echo no > UnixAuthentication
echo 1 > TLS
cd ../auth
ln -s ../conf/PureDB 50pure

With the above settings we disable all authentication methods except the internal PureFTPD user database. We also lock down each user into a chroot-environment and enable secure authentication via TLS. PureFTPD excepts a certificate in the file /etc/ssl/private/pure-ftpd.pem. You could copy your apache certificate to that file, or generate a new one via:

openssl req -new -x509 -days 4312 -nodes -keyout /etc/ssl/private/pure-ftpd.pem -out /etc/ssl/private/pure-ftpd.pem

The user administration and setup is now done with the PureFTPD tool pure-pw. You can add a new user via:

pure-pw useradd myusername -u ftpuser -d /path/to/homedir

The above command will ask you for a password and register the user which is mapped to the system user ftpuser within PureFTPD. Every time you modify the user database you need to rebuild the PureFTPD user configuration via:

pure-pw mkdb

Now you should be able to connect to your server with secure authorization with the newly created user.

The first thing is to open the prefenreces panel and set the “Pro” theme as the standard theme. I also like to activate text antialiasing. My font of choice is “Monaco 12pt.”.

To enable the page down, page up, home and end keys you need to go to the keyboard tab and set the following key actions:

Home = \033[1~
End = \033[4~
Page Up = \033[5~
Page Down = \033[6~

To activate colored output for ls with a decent color scheme that works well on a dark background, create a file .profile in your home directory with the following contents:

export CLICOLOR=1
export LSCOLORS=cxexcxdxbxfxfxbxbxcxcx

The idea behind this setup is that nginx will listen on port 80 for incoming connections, identify whether the client requests a static file or a dynamic webpage. In case of a static file it will deliver the file itself. In case of a dynamic request it will forward that request to the Apache server.

So let’s get started. First we need to download and unzip the lates stabel version of nginx. Currently this is verions 0.6.32. Compilation and installation is a done with the usual steps:

./configure
make
make install

This will install nginx in the directory /usr/local/nginx. I usually like to have all my configuration files under /etc, so let’s copy the configuration folder over:

cp -r /usr/local/nginx/conf /etc/nginx

The sample configuration file nginx.conf is well suited as a starting point. I would recommend to uncomment / change the following settings in the main section:

user www-data www-data;
worker_processes 2;
pid /var/run/nginx.pid;

In the http-section you could alter the following settings:

tcp_noauth on;
gzip on;

The English nginx wiki contains a very good documentation on these settings.

Nginx has full support for name based virtual hosts and you need to create a server-section in the config for every virtual host that is configured in Apache. But first create a new configuration file /etc/nginx/proxy.conf which contains the basic proxy settings as found in the nginx wiki:

proxy_redirect          off;
proxy_set_header        Host            $host;
proxy_set_header        X-Real-IP       $remote_addr;
proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size    10m;
client_body_buffer_size 128k;
proxy_connect_timeout   90;
proxy_send_timeout      90;
proxy_read_timeout      90;
proxy_buffers           32 4k;

These settings will be reused in every virtual host. The default virtual host is used as fallback in case no specific configuration for that virtual host can be found. To configure this default host, replace the server-section in the file /etc/nginx.conf with the following block:

server {
    listen       XXX.XXX.XXX.XXX:80 default;
    server_name  _;
    access_log /var/log/nginx/default.access.log main;
 
    location / {
        proxy_pass http://127.0.0.1:80;
        include /etc/nginx/proxy.conf;
   }
}

Replace XXX.XXX.XXX.XXX with the extern IP address of the server. The above configuration is a pure proxy configuration which will pass all the traffic to the Apache server that is listening on 127.0.0.1:80.

To be flexible in the virtual host configuration, I like to maintain one configuration file per virtual host in a separate directory. We can include all configuration files from a certain directory into the nginx configuration by adding the following line after the server-section:

include /etc/nginx/sites-enabled/*;

So every time you want to setup a new virtual host, you just need to add a new configuration file to the directory /etc/nginx/sites-enabled. Here is a template for that file:

server {
    listen XXX.XXX.XXX.XXX:80;
    server_name foobar.com www.foobar.com;
 
    location / {
        proxy_pass http://127.0.0.1:80;
        include /etc/nginx/proxy.conf;
    }
 
    location ~* ^.+.(jpe?g|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js|swf|avi|mp3)$ {
        expires 30d;
        root /var/www/foobar.com/htdocs;
    }
}

This configuration will setup the virtual host foobar.com. You can define all alias addresses with the configuration directive server_name. All requests that match one of the above file extensions will be delivered directly by nginx from the directory /var/www/foobar.com/htdocs. All other requests are forwarded to the Apache server.

Before nginx can be started we need to make sure that Apache only listens on the address 127.0.0.1 for requests and not on the external IP address. In Debian this is done in /etc/apache2/ports.conf. Change this file to:

Listen 127.0.0.1:80

Also make sure, that the VirtualHost directives in the Apache configuration files do not include an IP address. They should look like this:

<VirtualHost *:80>
    ServerName foobar.com
...
</VirtualHost>

The directive NameVirtualHost should look like this (also without an IP address):

NameVirtualHost *:80

Now you can restart Apache:

/etc/init.d/apache2 restart

And start nginx:

/usr/local/nginx/sbin/nginx -c /etc/nginx/nginx.conf

You can reload the nginx configuration without stopping nginx:

kill -HUP `cat /var/run/nginx.pid`

On my server some quick benchmarks have shown that nginx can deliver static content up to 10 times faster than Apache. The amazing thing is that not only did it deliver the content faster, there was nearly no impact on CPU or memory. With combining Apache and nginx we can have the best of both worlds, nginx for static files and Apache for dynamic content.

As a workaround for this bug, you can copy the files manually to the correct location:

sudo cp -r /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/* /Library/Python/2.5/site-packages/django/contrib/

The first step in setting up an OpenVZ server is to install a minimal setup of Ubuntu server 8.04. When installing Ubuntu you should select LVM in the partitioning options as this will enable backups of your OpenVZ containers without downtime while they are running. The only service that we will install is OpenSSH to access the installation remotely.

Once the base system is setup and you can login remotely via SSH you could update the whole system to make sure you have the latest versions of all installed applications:

sudo apt-get update
sudo apt-get dist-upgrade

The OpenVZ-Kernel does not work well with the AppArmor package, which is installed by default, so disable it:

sudo update-rc.d -f apparmor remove

Now it is time to install the OpenVZ-kernel:

sudo apt-get install linux-openvz

The above package is a meta package which will install the kernel, the kernel modules and any required header-files and libraries. Now take a look at the file /boot/grub/menu.lst and make sure that the new OpenVZ kernel is the default kernel when booting. Now reboot and check that the correct kernel is loaded:

uname -r

If you can see something like 2.6.24-18-openvz when running the above command, the correct kernel has been booted. If not, check your boot config.

The next step is to install the OpenVZ tools:

sudo apt-get install vzctl vzquota

Now you can run the OpenVZ daemon:

sudo /etc/init.d/vz start

Basically that’s it. OpenVZ is now running and you can setup your VZ containers. The OpenVZ project page has a lot of different container templates you can use as a starting point. Take a look at the list on http://wiki.openvz.org/Download/template/precreated and download all templates you want to use to /var/lib/vz/template/cache. You can choose from a wide selection of Linux distributions.

You can now start creating your virtual machine containers and run them:

sudo vzctl create 101 --ostemplate ubuntu-8.04-i386-minimal
sudo vzctl start 101

Of course you need to configure your containers, but this is a different topic which I will describe in a followup to this article.