As Go community slowly moving towards established and well understood patterns and practices of dependency management, there are still some confusing moments. One of them is automating repeatable build process using containers along with using dependencies in private repositories.
Private repositories on Github are often is a source of confusion when using
go get, but it has easy workaround by adding two lines to your
[url "firstname.lastname@example.org:"] insteadOf = https://github.com/
or as a oneliner:
git config --global url."email@example.com:".insteadOf "https://github.com/"
But the most confusing part is trying to make the whole build process work inside the container. I will use Docker as an example, as it’s most popular container at the moment.
Imagine, you have two packages
foo.go: import "github.com/company/bar"
In normal workflow, you setup GOPATH, you SSH keys, gitconfig and you’re done - simple
go get github.com/company/foo will work and download both packages:
$ go get -v github.com/company/foo github.com/company/foo (download) github.com/company/bar (download)
But now, you want to make the build process reproducible on any machine, even on CI instance, so you pack everything in Docker container. You will probably use simple Dockerfile based on official
FROM golang:1.6 ADD . /go/src/github.com/company/foo CMD cd /go/src/github.com/company/foo; go get github.com/company/bar && go build -o /go/bin/foo
docker build -t foo-build . # build image docker run --name=foo-build foo-build # compile binary docker cp foo-build:/foo foo # copy binary to fs docker rm -f foo-build # remove container docker rmi -f foo-build # remove image
This setup will not work because Docker container used for building (foo-build) doesn’t container
bar dependency, SSH keys and proper gitconfig. And, apparently, it’s not trivial simply to add the keys - you have to deal with a bunch of obstacles, mainly on the SSH side. So, let’s go through quickly and setup working solution.
ssh vs https
First of all, on the building stage (
docker run ...) you will encounter the following error message:
# cd .; git clone https://github.com/company/bar /go/src/github.com/company/bar Cloning into '/go/src/github.com/company/bar'... fatal: could not read Username for 'https://github.com': No such device or address package github.com/company/bar: exit status 128
What it does mean, is that your access to github is granted using SSH keys, but
git command, which is invoked by
go get, is trying to clone repository using HTTPS form and you don’t have credentials set up.
Workaround for this is easy, and described in the beginning of this post, so we just have to add this to our Dockerfile right before calling
RUN echo "[url \"firstname.lastname@example.org:\"]\n\tinsteadOf = https://github.com/" >> /root/.gitconfig
The next error you’ll see is host key verification error:
# cd .; git clone https://github.com/company/bar /go/src/github.com/company/bar Cloning into '/go/src/github.com/company/bar'... Host key verification failed. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. package github.com/company/bar: exit status 128
Well, it’s simply because our Docker container doesn’t have SSH keys yet. And the right approach is not trivial, so let’s go through it.
First of all, we want every developer or CI to use it’s own keys for accessing private repo. If person has access to
foo, she’s definitely has an access to
bar and the keys are usually in
You can’t copy keys into container, though. Dockerfile’s ADD and COPY commands can copy files from the current directory only, so you can’t just add
ADD ~/.ssh/ /root/ssh to your Dockerfile. One of the solution is to write wrapper script that will copy private key to local directory and then to the container, but it’s still not very safe and elegant solution.
What we can do, is to mount volume using docker’s
-v command line flag. The first approach will be probably to mount the whole
~/.ssh directory, but it’s tricky
docker run --name=foo-build -v ~/.ssh:/root/.ssh foo-build
This command will work as expected on MacOS X (using latest Docker Beta, at least), but not in Linux box. The reason is the files ownership for
~/.ssh/config file. The
ssh (which is invoked by
git, which is invoked by
go get) expects this file to have the same user ownership as a running user. Inside the container the user is
root, but the mounted directory most probably has ovnership of your normal Linux user, say,
developer and inside the container it looks like:
$ ls ~/.ssh/config -rw-r--r-- 1 1000 1000 147 Jun 1 19:20 /root/.ssh/config
making SSH to complain and abort:
Bad owner or permissions on ~/.ssh/config
The solution is to mount only the key and workaround host checking later.
docker run --name=foo-build -v ~/.ssh/id_rsa:/root/.ssh/id_rsa foo-build
Error will be the same, though, but rerunning it with
-t option, we’ll see the reason:
$ docker run --name=foo-build -v ~/.ssh/id_rsa:/root/.ssh/id_rsa -t foo-build The authenticity of host 'github.com (126.96.36.199)' can't be established. RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48. Are you sure you want to continue connecting (yes/no)?
Of course, we don’t want to interact manually with ssh prompt, so we have to find a way to force it. There is an SSH client option for that, called StrictHostChecking.
Typically, you have file named
~/.ssh/known_hosts, which holds information about, well, known hosts. But in our container, there is no such file, so we have to use the client option to supress those checks. The easiest way to do this is to put this option into the
~/.ssh/config file - yes, the one we had ownership problems with.
But, we only need one option, so it’s ok to create this file on the fly inside the container. Add to Dockerfile:
RUN mkdir /root/.ssh && echo "StrictHostKeyChecking no " > /root/.ssh/config
docker run step and you’ll finally have success!
The final Dockerfile:
FROM golang:1.6 RUN echo "[url \"email@example.com:\"]\n\tinsteadOf = https://github.com/" >> /root/.gitconfig RUN mkdir /root/.ssh && echo "StrictHostKeyChecking no " > /root/.ssh/config ADD . /go/src/github.com/company/foo CMD cd /go/src/github.com/company/foo && go get github.com/company/bar && go build -o /foo
and the build steps:
docker build -t foo-build . docker run --name=foo-build -v ~/.ssh/id_rsa:/root/.ssh/id_rsa foo-build docker cp foo-build:/foo foo docker rm -f foo-build docker rmi -f foo-build
You may put those steps to the Makefile or custom build script, and can safely use it locally or in CI or whatever.
Private SSH key is copied once into the temporary container, used for building, which is removed immediately. Nice and safe solution.