HOWTO: Set Up A Private Git Server

By RT REDDY

As the company I worked for started growing, we needed to set up a version control server to make it easier for our geographically dispersed developers to collaborate. I decided to go with Git running on Ubuntu 12.04. Here's how I did it.

Install Ubuntu

First, I grabbed the ISO that I wanted to use. I went with Ubuntu Server 12.04.1. Being an LTS version, I figured 12.04.1 should work for quite a while. I used Startup Disk Creator in Ubuntu 12.10 to install the ISO on a USB flash drive and make it bootable. This avoids having to burn a DVD that I'm going to end up tossing anyway.
The USB install was ok, although it did require a little babysitting as it would ask various questions along the way. It's a text mode interface, and the final install is about 1.5GB.
I went with LVM (Logical Volume Manager) as it sounded quite professional. In the end, I think this was a mistake. When Ubuntu sets up LVM, it gives your "/boot" directory a tiny 200MB filesystem which quickly fills up over time until the system can no longer upgrade the kernel. You'll need to run this periodically to clean up the older kernels:

$ sudo apt-get autoremove

I think I'm going to skip LVM in the future and just partition the whole drive for EXT4 and a swap partition. This should be more maintenance free for me anyway. I don't really anticipate using the features of LVM.
The install asked me if I wanted to install the ssh server. I said yes, and it installed it for me. If you forget, you can install it later:

$ sudo apt-get install ssh

Configure Networking

I had to configure the server for wireless as that was the only convenient way to hook it up to the Internet in my house. Fortunately, this turned out to be incredibly easy. I plugged in my ALFA USB wireless adapter and then went in to /etc/network/interfaces and added the appropriate lines to make sure the network would come up automatically at boot. Setting up Ethernet in here is really easy if you go that route. Just add the following:

# The Ethernet interface
auto eth0
iface eth0 inet dhcp

Setting up wireless is only slightly more difficult. Here's what my added entries looked like:

# The wireless interface
auto wlan0
iface wlan0 inet dhcp
    wpa-driver wext
    wpa-ssid WirelessRouterName
    # 2 => SSID is hidden
    wpa-ap-scan 2
    # RSN => WPA2
    wpa-proto RSN
    wpa-pairwise CCMP
    wpa-group CCMP
    wpa-key-mgmt WPA-PSK
    # ASCII version requires quotes.  Hex doesn't.
    wpa-psk "passphrase"
    # wpa_passphrase can be used to convert an ASCII passphrase to hex.

Put your wireless router SSID (name) where you see "WirelessRouterName" and your WPA2 passphrase where you see "passphrase". This assumes you are using WPA2 and PSK mode. See the man page for wpa_supplicant.conf(5) for the details on each of the above fields.
Next you'll want to configure the firewall. I used ufw, which is probably not a good idea for a server that isn't protected by another hardware firewall. Setting up ufw is very easy:

$ sudo ufw enable
$ sudo ufw status

Install Updates

Use apt-get and install the latest updates.

$ sudo apt-get update
$ sudo apt-get upgrade

Check Running Servers

Use one of the following (or try all of them) to see what servers are running and listening on ports. Ubuntu tends to have almost nothing running after a clean install. Very nice.

$ sudo lsof -i
$ sudo netstat -lptu
$ sudo netstat -tulpn

Securing the Secure Shell

Using git through ssh is simple and secure, so I decided to go with that approach. My main concern was securing ssh so that it would be very difficult to hack. These steps are probably not complete, so I recommend that you check other sources to find out how to secure a ssh server.
Make sure the admin account has an ssh key so that we can get back in after we lock things down. If you are working from the console, this might not be a big deal, but it will probably make your life a lot easier.
The /etc/ssh/sshd_config file has all the settings for sshd that you can use to make it more secure. In no particular order...
Restrict the encryption ciphers to the best by adding this line to sshd_config:

Ciphers aes256-ctr

Pick a random port number between 10000 and 65535 (see random.org) and move the server to that port. Add or adjust the following line in sshd_config, replace "port" with the port number you've picked:

Port port

Restrict authentication so that only public/private key authentication is allowed. Add or adjust the following lines in sshd_config:

PasswordAuthentication no
ChallengeResponseAuthentication no
PermitRootLogin no
UsePAM no

Turn off tcp tunneling so your users can't use your server to bounce around the Internet and cause trouble. Add or adjust this line in sshd_config:

AllowTcpForwarding no

Note that this is pretty extreme as it blocks tunneling even to other ports on the server. If you want to selectively allow connections, e.g. to 192.168.1.200:3000 only, do this:

AllowTcpForwarding yes
PermitOpen 192.168.1.200:3000

Finally, once you've made all your changes, ask sshd to reload the config file (this sends it a good ol' SIGHUP):

$ sudo initctl reload ssh

Open a hole in the server firewall

So that ssh traffic can get in through the random port we picked above, we'll need to poke a hole through the server's firewall. With ufw (replace "port" with the random port number we selected for sshd):

$ sudo ufw allow port
$ sudo ufw status

Now we can test ssh from the local network. From another computer on the network, try logging into the server with ssh. (Replace "port" with the random port number we selected for sshd.):

$ ssh -p port user@host

Install and Configure Git

Install git.

$ sudo apt-get install git

Set up a "git" group. Members of this group will be able to push to the repo. Everyone else will be able to fetch only.

$ sudo groupadd git

Then add your user to that group so that you can test. Change "ted" to your userid on the server.

$ sudo gpasswd -a ted git

Now we need to create the /srv/git directory and set it up properly. (Be sure to replace "ted" with your server userid. "root" is probably ok too.)

$ sudo mkdir -p /srv/git
$ cd /srv
$ sudo chown ted:git git
$ chmod 775 git
$ chmod g+s git

Tell git that we want to do sharing with the git group. This should preserve the "git" group on all files along with the setgid bit on all directories. We'll double-check later just in case.

$ sudo git config --global core.sharedRepository group

Now you can clone your first repo into the /srv/git directory. One way is to clone from a repo in another directory. In this example, we assume the repo is in a directory called "source-repo-dir". Replace "projectname" with an appropriate name for the git project.

$ git clone --bare source-repo-dir /srv/git/projectname.git

Setting core.sharedRepository to "group" should have ensured that the group and setgid bits were handled properly when we did the clone. Check to make sure this is the case. Go through the new repo and make sure all files and directories belong to the "git" group and that the setgid bit is on for all directories. If this isn't the case, fix up the permissions with chown and chmod:

$ cd /srv/git
$ chown -R ted:git projectname.git
$ chmod -R 775 projectname.git
$ find projectname.git -type d -exec chmod g+s '{}' \;

Now we can test git from another machine on the network. Clone the repo and see if it works. Replace "user" with your server userid, "host" with the IP or hostname of the server, and "port" with the random sshd port we selected. Also replace "projectname" with the name of the project.

git clone ssh://user@host:port/srv/git/projectname.git projectname

Internet Firewall

Set up a hole in your Internet firewall so that people outside of the local network can get to your git server. If you have a dynamic IP, set up Dynamic DNS so that others can find you. With my Asus router, this was built-in and very easy to set up. Next, poke a hole through your firewall to allow data for the sshd port we selected above to get to the server. E.g. if the port we picked was 2222, and the server was at 192.168.5.15, set up port forwarding on the firewall to send connections for port 2222 to 192.168.5.15:2222.
Once this is set up, try testing from the outside. Or test from the inside with the dynamic DNS name. This should be almost as good.

Add Users

To add a user to the server:

$ sudo adduser --disabled-password --shell /usr/bin/git-shell username

Optionally, add them to the "git" group if you want them to be able to push:

$ sudo gpasswd -a username git

For the git-shell to work, you must create a git-shell-commands directory. Not sure why, but if you don't, you'll get errors that it is missing.

$ sudo mkdir /home/username/git-shell-commands

Ask them to generate a public key for you with ssh-keygen and email it to you. Append their public key to their authorized_keys file.

$ sudo mkdir /home/username/.ssh
$ sudo sh -c "cat id_rsa.pub >>/home/username/.ssh/authorized_keys"

And finally, for good measure, normalize the owner/group:

$ sudo chown -R username:username /home/username

Now they should be able to clone from their machine:

git clone ssh://username@host:port/srv/git/projectname.git projectname

To troubleshoot, the user can use ssh to connect and they should get a "git>" prompt if all is configured properly. "quit" will exit.

ssh -p port username@host

Updates

When installing, I remember telling it to go ahead and automatically upgrade packages for me. That appears to be working for security updates, but not for regular updates. I need to take a look at that. See the Automatic Updates section of the Ubuntu Server Guide for more.
Sometimes a reboot is needed after an upgrade. This is not done automatically. If you login from the console, and the system decides that the load isn't too great, you'll see this in the message of the day (/etc/motd):

*** System restart required ***

If you login remotely, the motd doesn't appear to be updated or displayed. (Probably because I turned off PAM under ssh. See update_motd(5).) In that case, check for:

/var/run/reboot-required

If that exists, then you need to reboot to finish the updates. See the weekly maintenance scripts in the next section which will do this automatically.

Periodic Maintenance

For weekly maintenance, I put the following scripts in /etc/cron.weekly. First, a script called apt-upgrade:

#!/bin/bash

# apt upgrade script.  There is probably a better way to do this.
# I currently have it set up to do regular installs of security
# updates.  I think there's a config file someplace where I can
# ask for non-security updates too.

# Remove older kernels to keep the /boot fs from filling up.
apt-get -y autoremove

apt-get update

# Might want to use "dist-upgrade" instead.  This doesn't always
# upgrade everything.
apt-get -y upgrade

Next is a script called git-gc. You'll need to tweak "projectname.git" as needed. Plus, you might need to gc multiple repos.

#!/bin/bash

# Git garbage collection script.
# Dump this in /etc/cron.weekly and call it git-gc.

# Do git garbage collection
cd /srv/git/projectname.git
git gc --aggressive

Finally, a rather dangerous script called zz-reboot-required:

#!/bin/bash

# Reboot if updates require a reboot.
# Add this to cron.weekly and name it zz-reboot-required so it is
# run last.

# if reboot required,
if [ -e /var/run/reboot-required ]
then
  # Reboot in 5 minutes
  shutdown -r +5
fi

The "zz" on the front of the name ensures that this script is the last thing that is run in the cron.weekly directory. Since this script reboots the system at the end, we don't want that to happen before all the weekly scripts are run.
It might be possible to eliminate the apt-upgrade script by configuring automatic updates to do all updates, not just security updates. See the Automatic Updates section of the Ubuntu Server Guide. The cron script above has the advantage of only performing the upgrades at a predictable date/time (Sunday morning). (The "unattended-upgrades" package can probably be configured to do its work at a specific time also, so I'm not sure whether this is actually an advantage.)

Monitoring the Server

A server that is exposed to the public Internet needs to be monitored to make sure it continues to be secure. There are many ways to check this. This is definitely not an exhaustive checklist. In fact, it's a pretty lousy one based on a few things I've researched. I highly recommend looking elsewhere for advice on this.
The /var/log/auth.log file shows attempted ssh logins. You can monitor this to detect attempted break-ins.
denyhosts is a program that will watch for excessive login failures and ban the host that is causing them. This should block attackers before they get in by brute-force.
Use lsof -i to make sure no unexpected servers are running.
"logcheck" is a program that might be useful, but I've not looked into it.
An Intrustion Detection System like tripwire is also a good idea. It will detect modifications to the system that might be an indication of a break-in.

Building Git

If you want to keep up with the latest releases of Git, but still run the LTS version of Ubuntu, you'll need to download the Git source and build it.
First, you'll need to install some software to do the build:

$ sudo apt-get build-dep git
$ sudo apt-get install autoconf

You can find the Git source on github. The tags on the Git project have .tar.gz files with the source:

http://github.com/git/git/tags

You can use wget to download the file you want. Or you can use the text browser "w3m" to find and download the latest version:

$ w3m github.com/git/git/tags

Move the cursor on top of a link and hit enter to download. Press q to exit out of w3m.
Expand the code archive with tar:

$ tar -xf git-1.8.2.3.tar.gz

Within the git source directory, you will find an INSTALL file with instructions on how to build and install. For 1.8.2.3 the "configure" set of steps seemed to work best:

$ make configure
$ ./configure --prefix=/usr
$ make all doc
$ sudo make install install-doc install-html

And you've got the latest Git.

Technical Knowledge Overview

Thursday, 5 May 2016