Darktable for Windows using Vagrant

I have an Olympus TG5 camera, which has RAW support for Darktable, but only in the very latest (currently unreleased!) 2.3.0 version.  Since I have Windows, I’ll have to build Darktable directly from source to be able to manipulate it.  Here’s how you can do that.

First, I assume you have Cygwin/X running.  You’ll also need Vagrant installed, along with VirtualBox.  With all that in place, doing the rest is pretty straightforward.  Create a folder, and throw this Vagrantfile into it;

# -*- mode: ruby -*-
# vi: set ft=ruby :

VMBOX = "bento/ubuntu-16.04"
VMHOSTNAME = "darktable"
VMRAM = "1024"


Vagrant.configure("2") do |config|
  # Configure the hostname for the default machine
  config.vm.hostname = VMHOSTNAME

  # Configure the VirtualBox provider
  config.vm.provider "virtualbox" do |vb, override|
    # The default ubuntu/xenial64 image has issues with vbguest additions
    override.vm.box = VMBOX

    # 1gb RAM, 2 vCPU
    vb.memory = VMRAM
    vb.cpus = VMCPU

    # Configure vbguest auto update options
    override.vbguest.auto_update = false
    override.vbguest.no_install = false
    override.vbguest.no_remote = true

  # Mount this folder as RW in the guest, use this for transferring between host and guest
  config.vm.synced_folder "shared", "/srv/shared", :mount_options => ["rw"]

  # Build the server from a provisioning script (which will build Darktable for us)
  config.vm.provision "shell", inline: <<-SHELL
    # Install essential and optional dependencies
    apt-get update
    apt-get install -y gcc g++ cmake intltool xsltproc libgtk-3-dev libxml2-utils libxml2-dev liblensfun-dev librsvg2-dev libsqlite3-dev libcurl4-gnutls-dev libjpeg-dev libtiff5-dev liblcms2-dev libjson-glib-dev libexiv2-dev libpugixml-dev
    apt-get install -y libgphoto2-dev libsoup2.4-dev libopenexr-dev libwebp-dev libflickcurl-dev libopenjpeg-dev libsecret-1-dev libgraphicsmagick1-dev libcolord-dev libcolord-gtk-dev libcups2-dev libsdl1.2-dev libsdl-image1.2-dev libgl1-mesa-dev libosmgpsmap-1.0-dev

    # Install usermanual and manpage dependencies
    apt-get install -y default-jdk gnome-doc-utils libsaxon-java fop imagemagick docbook-xml docbook-xsl
    apt-get install -y po4a

    # Install this for Cygwin/X to work properly
    apt-get install -y xauth

    # Pull the master repo
    git clone https://github.com/darktable-org/darktable.git
    cd darktable
    git checkout master

    # Pull the submodules
    git submodule init
    git submodule update

    # Build Darktable
    ./build.sh --prefix /opt/darktable

    # Build documentation
    cd build
    make darktable-usermanual
    make darktable-lua-api
    cd ..

    # Install Darktable
    cmake --build "/home/vagrant/darktable/build" --target install -- -j2

    # Copy documentation into shared area
    cp build/doc/usermanual/*.pdf /srv/shared/

  # This piece here is run when we use 'vagrant ssh' to configure the SSH client appropriately
  if VAGRANT_COMMAND == "ssh"
    config.ssh.forward_x11 = true


Make a shared folder in that folder, and vagrant up followed by vagrant ssh.

Assuming everything is configured correctly, you can then start Darktable with;


And off you go.  You can add some more mounts into the VM as required to share your picture library or whatever with it so you can manipulate it with Darktable.

Ansible with Vagrant on Windows

Since I’m converting all my builds and other things to use Ansible, the idea of using Ansible to customize a Vagrant box is very attractive.

I’ve chosen to use the ansible-local provisioner in this case, so that Ansible runs inside the Vagrant box.  I’ll do an example later where this isn’t the case.

Have a look at this gist for some info about how to do this.  Or read on.

Step 1 – the Vagrantfile

In a blank directory, edit a new Vagrantfile.  Make it look something like this;

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
 # The default ubuntu/xenial64 image has issues with vbguest additions
 config.vm.box = "bento/ubuntu-16.04"

 # Set memory for the default VM
 config.vm.provider "virtualbox" do |vb|
   vb.memory = "1024"

 # Configure vbguest auto update options
 config.vbguest.auto_update = false
 config.vbguest.no_install = false
 config.vbguest.no_remote = true

 # Configure the hostname for the default machine
 config.vm.hostname = "ansible-example"

 # Mount this folder as RO in the guest, since it contains secure stuff
 config.vm.synced_folder "vagrant", "/vagrant", :mount_options => ["ro"]

 # And finally run the Ansible local provisioner
 config.vm.provision "ansible_local" do |ansible|
   ansible.provisioning_path = "/vagrant/provisioning"
   ansible.inventory_path = "inventory"
   ansible.playbook = "playbook.yml"
   ansible.limit = "all"


There’s a few things going on here.  First up, we define the default box we’re going to use, the memory allocated to it, our auto-update options and the hostname.

Next up is we define a synced folder that will appear in the Vagrant box.  There is a default, which is for the folder the Vagrantfile is in to appear as /vagrant.  However, this is shared on VirtualBox with R/W access, which means that the box can modify your original files (including its own Vagrantfile).  Not necessarily bad, but I don’t like the idea of that very much.

Lastly, we define the Ansible provisioner.  This will simply run the playbook that’s in the vagrant/provisioning subfolder of the Vagrantfile against all hosts.

Step 2 – Create Playbook

Do the following to create the rest of the structure (from within the directory your Vagrantfile is in);

mkdir -p vagrant/provisioning

Now, you’ll need to create an ansible.cfg in that directory, like this;

host_key_checking = no

ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes

The parameters are necessary to avoid Ansible having a freak-out about SSH keys and whatnot when deploying.  Of course, if you have just one host, you don’t need to worry about it.

Next, you need an inventory spec;

ansible-example ansible_connection=local

This forces deployments against the machine we’re deploying to use the local connection type.

And lastly, a really basic playbook to test it out;


- hosts: ansible-example
 - copy: content="IT WORKS!\n" dest=/home/vagrant/ansible_runs


Step 3 – Run it!

Now we’ve set up the most basic structure, bring it up!

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'bento/ubuntu-16.04'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'bento/ubuntu-16.04' is up to date...
==> default: Setting the name of the VM: ansible-example_default_1472187535117_41803
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
 default: Adapter 1: nat
==> default: Forwarding ports...
 default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
 default: SSH address:
 default: SSH username: vagrant
 default: SSH auth method: private key
 default: Vagrant insecure key detected. Vagrant will automatically replace
 default: this with a newly generated keypair for better security.
 default: Inserting generated public key within guest...
 default: Removing insecure key from the guest if it's present...
 default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
 default: The guest additions on this VM do not match the installed version of
 default: VirtualBox! In most cases this is fine, but in rare cases it can
 default: prevent things such as shared folders from working properly. If you see
 default: shared folder errors, please make sure the guest additions within the
 default: virtual machine match the version of VirtualBox you have installed on
 default: your host and reload your VM.
 default: Guest Additions Version: 5.0.26
 default: VirtualBox Version: 5.1
==> default: Setting hostname...
==> default: Mounting shared folders...
 default: /vagrant => C:/cygwin64/home/username/ansible-example/vagrant
==> default: Running provisioner: ansible_local...
 default: Installing Ansible...
 default: Running ansible-playbook...

PLAY [ansible-example] ****************************************************************

TASK [setup] *******************************************************************
ok: [ansible-example]

TASK [copy] ********************************************************************
changed: [ansible-example]

PLAY RECAP *********************************************************************
ansible-example : ok=2 changed=1 unreachable=0 failed=0


And to prove that the playbook actually, really, did run;

$ vagrant ssh
Welcome to Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-31-generic x86_64)

 * Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/advantage
Last login: Fri Aug 26 05:01:58 2016 from
vagrant@ansible-example:~$ cat ansible_runs

You can then re-run the playbook any time you like with vagrant provision .

The main catch with running Ansible like this is that it actually installs Ansible on the Vagrant box.  You can get around this by running Ansible on your Vagrant host.  More on this later.

Vagrant on Cygwin/Virtualbox Quickstart

So, you want to try out Vagrant, and you’re using Windows with Cygwin?  Have I got something for you!

Preparing the Environment

Firstly, get Oracle VirtualBox installed.  I personally prefer VMware Workstation, but VirtualBox works better for this.  Also get the extensions while you’re at it.

Next, go and install Vagrant, and use the default settings.  Now we’re going to have to manually patch a file in the Vagrant source.  Go to /cygdrive/c/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.5/plugins/guests/linux/cap in Cygwin, and edit public_key.rb .  At line 57, make the code look like the bit that’s highlighted here;

if test -f ~/.ssh/authorized_keys; then
  grep -v -x -f '#{remote_path}' ~/.ssh/authorized_keys > ~/.ssh/authorized_keys.tmp
  mv ~/.ssh/authorized_keys.tmp ~/.ssh/authorized_keys
  chmod 0600 ~/.ssh/authorized_keys

This won’t be necessary in a newer version of Vagrant, but it is required in 1.8.5 for some boxes to work.

Next up, bring up your Cygwin prompt, and do this.  This will remove the default VMware provider (if it’s installed), and put in a plugin that automatically updates VirtualBox Guest Additions (optional, but very useful)

vagrant plugin uninstall vagrant-vmware-workstation
vagrant plugin install vagrant-vbguest
vagrant version

It should spit out that you’re running an up-to-date Vagrant.  Great.

Bringing up your first Vagrant box

Now, I’m a CentOS fan, so we’ll be bringing up a CentOS box first.  From your Cygwin prompt, do this;

vagrant box add centos/7 --provider virtualbox
mkdir vagrant-test && cd vagrant-test
vagrant init centos/7
vagrant up
vagrant ssh

If everything’s been done correctly, you’ll find yourself in a shell on your new Vagrant box.  By default, the VM will be using NAT.  Poke around, and when done, exit and do;

vagrant destroy -f
cd ..
rm -rf vagrant-test

To clean everything up.  After cleanup, you’ll still be left with the centos/7 box cached, you can ditch that with vagrant box remove centos/7 .

All done!  You’ve got a working Vagrant environment on Windows, running under Cygwin against a VirtualBox provider.  Magic!

Vagrant Quickstart on Ubuntu Xenial 16.04 with Libvirt

There’s a few issues with running Vagrant with Libvirt on Ubuntu 16.04 .  Namely, the bundled version of Vagrant is broken.  Whoops!

Here’s how you can get it running using the upstream Vagrant (currently 1.8.4), get a basic libvirt running, and bring up a VM just to prove that it works (we’ll use openSUSE because they provide a box that works with libvirt).

Install the libvirt essentials

sudo apt-get update
sudo apt-get install ubuntu-virt-server ubuntu-virt-mgmt virt-manager libvirt-dev
sudo adduser YOURUSERNAME libvirtd

Fetch and install upstream Vagrant

DANGER:  Don’t run any vagrant plugins with sudo, it will probably trash permissions on your ~/.vagrant.d/ directory and go badly for you.

sudo apt purge vagrant
sudo apt autoremove
wget https://releases.hashicorp.com/vagrant/1.8.4/vagrant_1.8.4_x86_64.deb
sudo dpkg -i vagrant_1.8.4_x86_64.deb
sudo apt-get install -f
vagrant plugin install vagrant-libvirt

Bring up a test VM

Showtime!  Bring up a test VM and connect to it with ssh…

mkdir testvm
cd testvm
vagrant init opensuse/openSUSE-42.1-x86_64
vagrant up --provider libvirt
vagrant ssh

Get rid of the test machine

vagrant destroy

Phew.  Next up, making your own box.