Rumble Functionality with XB1 Wireless on Steam Link

If you have the newer (model 1708) XBox One Wireless Controller, you can get it to work with a Steam Link on native Bluetooth pretty easily.  But if it’s straight out of the box, the rumble motor won’t work on the controller.

In order to make it work, you’ll need a PC with Windows 10 and the Anniversary Update installed.  Go to the Windows Store and install ‘Xbox Accessories’ by Microsoft.

Then, plug in your XB1 controller with a USB cable into your PC (shut down Steam first).  The PC will then prompt you to update firmware on the controller.  Do so.  You’ll then have to repair it with the Steam Link, but once that’s done rumble functionality should work.

Threading a X-Tek Pure Buckle the Right Way

In case this is useful for anyone besides me, here is the correct way to thread the waist buckle onto a Scubapro X-Tek Pure Harness;

My sincere apologies for the Microsoft Paint drawing.  The long end of the red line represents the end of the webbing that is attached to the rest of the harness, and the short end is the end that you have in your hand.  The rest should be self-explanatory.  Threading it this way results in the buckle pulling tight properly when buckled in, and also means the other end of the webbing can slide over the buckle properly when putting it on.

You don’t want to know how many times I put it on wrong before I figured this out.  I swear that thing exists in n-dimensional space or something.

Converting a bunch of OGG music to MP3, preserving metadata

Quick one.  If you have a heap of OGG music that you want to convert to MP3 format, and also want to conserve the metadata that’s in the music, run this from Ubuntu;

for name in *.ogg; do ffmpeg -i "$name" -ab 128k -map_metadata 0:s:0 "${name/.ogg/.mp3}"; done

Done and dusted!

Adding an RTC to your Raspberry Pi

I use a RPi 3 as a secondary DNS and DHCP server, and time synchronization is important for that.  Due to some technicalities with how my network is set up, this means that I need a real-time clock on the RPi so that it can have at least some idea of the correct time when it powers up instead of being absolutely dependant on NTP for that.

Enter the DS3231 RTC (available on eBay for a few bucks).  The Pi Hut has an excellent tutorial on setting this up for a RPi, which I’m going to summarize here.

Configure I2C on the RPi

From a root shell (I’m assuming you’re using Raspbian like me);

apt-get install python-smbus
 apt-get install i2c-tools

Then, edit your /boot/config.txt and add the following down the bottom;

dtparam=i2c_arm=on
dtoverlay=i2c-rtc,ds3231

Edit your /etc/modules and add the following line;

i2c-dev

Now reboot.  If you do an i2cdetect -y 1 you should see the DS3231 listed as device 0x68.  If you do, great.

Configure Raspbian to use the RTC

After rebooting, the new device should be up, but you won’t be using it yet.  Remove the fake hardware clock with;

apt-get --purge remove fake-hwclock

Now you should be able do hwclock -r to read the clock, and then hwclock-w to write the current time to it.

And lastly, to make it pull time from the RTC on boot, put the following into /etc/rc.local before the exit 0;

hwclock -s

And you can then add a cronjob in /etc/cron.weekly to run hwclock -w once a week.

Done!

Backing up KVM Virtual Machines with Duplicity + Backblaze

As part of my home DR strategy, I’ve started pushing images of all my virtual machines (as well as my other data) across to Backblaze using Duplicity.  If you want to do the same, here’s how you can do it.

First up, you will need a GnuPG keypair.  We’re going to be writing encrypted backups.  Store copies of those keys somewhere offsite and safe, since you will absolutely need those to do a restore.

Secondly, you’ll need a Backblaze account.  Get one, then generate an API key.  This will be comprised of an account ID and an application key.  You will then need to create a bucket to store your backups in.  Make the bucket private.

Now that’s done, I’m assuming here that you have your /var/lib/libvirt library where your VMs are stored on its own LV.  If this isn’t the case, make it so.  This is so you can take a LV snapshot of the volume (for consistency) and then replicate that to Backblaze.

#!/bin/bash

# Parameters used by the below, customize this
BUCKET="b2://ACCOUNTID:APPLICATIONKEY@BUCKETNAME"
TARGET="$BUCKET/YOURFOLDERNAME"
GPGKEYID="YOURGPGKEYIDHERE"
LVNAME=YOURLV
VGPATH=/dev/YOURVG

# Some other parameters
SUFFIX=`date +%s`
SNAPNAME=libvirtbackup-$SUFFIX
SOURCE=/mnt/$SNAPNAME

# Prep and create the LV snap
umount $SOURCE > /dev/null 2>&1
lvremove -f $VGPATH/$SNAPNAME > /dev/null 2>&1
lvcreate --size 10G --snapshot --name $SNAPNAME $VGPATH/$LVNAME || exit 1

# Prep and mount the snap
mkdir $SOURCE || exit 1
mount -o ro,nouuid $VGPATH/$SNAPNAME $SOURCE || exit 1

# Replicate via Duplicity
duplicity \
 --full-if-older-than 3M \
 --encrypt-key $GPGKEYID \
 --allow-source-mismatch \
 $SOURCE $TARGET

# Unmount and remove the LV snap
umount $SOURCE
lvremove -f $VGPATH/$SNAPNAME
rmdir $SOURCE

# Configure incremental/full counts
duplicity remove-all-but-n-full 4 $TARGET
duplicity remove-all-inc-of-but-n-full 1 $TARGET

Configure the parameters above to suit your environment.  You can use gpg --list-keys to get the 8-digit hexadecimal key ID of the key you’re going to encrypt with.  The folder name in your bucket you use is arbitrary, but you should only use one folder for one Duplicity target.  The 10G LV snap size can be adjusted to suit your environment, but it must be large enough to hold all changes made while the backup is running.  I picked 10Gb, because that seems OK in my environment.

Obviously this means I need to have 10Gb free in the VG that the libvirt LV lives in.

Retention here will run incrementals each time it’s run, do a full every 3 months, ditch any incrementals for any fulls except the latest one, and keep up to 4 fulls.  With a weekly backup, this will amount to a 12 month recovery window, with a 3-monthly resolution after 3 months, and a weekly resolution less than 3 months.  Tune to suit.  Drop that script in /etc/cron.daily or /etc/cron.weekly to run as required.

Important.  Make sure you can do a restore.  Look at the documentation for duplicity restore for help.

Splunkd High CPU after leap second addition?

Had my alerting system yell at me about high CPU load on my Splunk Free VM;

A bit of examination revealed that it was indeed at abnormally high load average (around 10), although there didn’t appear to be anything wrong.  Then a quick look at dmesg dropped the penny;

Jan 1 10:29:59 splunk kernel: Clock: inserting leap second 23:59:60 UTC

Err.  The high CPU load average started at 10:30am, right when the leap second was added.

A restart of all the services resolved the issue.  Load average is back down to its normal levels.

Minecraft Local Server Discovery – Can’t find LAN games?

I had a problem where I couldn’t find LAN games automatically on my local network in Minecraft.  Turns out that the problem was due to the interface priority on my network interfaces, and Minecraft was binding to the wrong interface!

Minecraft uses UDP multicast (on 224.0.2.60, port 4445) to advertise local games.  If you have more than one network adapter on your machine (in my case, a VirtualBox Host-Only adapter), it’s possible that Minecraft has bound to the wrong adapter.

You can reveal this with netsh interface ip show joins – if you see the join on 224.0.2.60 on the wrong interface, that’s your problem.  Here’s how to fix it.

Open an administrative Powershell prompt.  Run get-netipinterface and review.  You should see two entries for the offending adapter.  Look at the InterfaceMetric value for that adapter and for the adapter you want to be the default.  In my case, both were 25.

You can now adjust the interface metric for the offending adapter to be higher than the correct adapter;

get-netipinterface | where-object { $_.InterfaceAlias -like "VirtualBox*" } | set-netipinterface -interfacemetric 40

And voila!  Minecraft local server discovery works again!

Powershell Remoting for Non-Domain Test Machines

NOTE – This isn’t particularly secure, but it works.  It’s a bit better than configuring WinRM in unencrypted mode though.

Got some non-domain joined Windows machines and you want to get WinRM running in a hurry so you can do some stuff remotely?  Do this.

On the server (the thing you are remoting to);

Invoke-WebRequest -Uri https://github.com/ansible/ansible/blob/devel/examples/scripts/ConfigureRemotingForAnsible.ps1 -OutFile ConfigureRemotingForAnsible.ps1
.\ConfigureRemotingForAnsible.ps1
winrm quickconfig

That script is taken from Ansible, and configures a host with a self-signed SSL cert for use with WinRM.  The final line then configures up the WinRM listeners and firewall rules.

Then, on the client (the thing you’re remoting from);

# enter local admin creds here
$creds = get-credential  

$so = New-PSSessionOption -SkipCACheck -SkipCNCheck
Invoke-Command -Computername YOURSERVERHERE -UseSSL -SessionOption $so -Credential $creds -ScriptBlock { get-childitem env: }

You should see a dump of the local environment variables on the target machine, indicating that the invoke worked.  You can now do whatever Powershell remoting stuff you want to do.

Note, this doesn’t actually check the CA cert provided, so you can be MITM’ed and have your credentials captured.  For better security you should use a properly signed certificate on the server and trust it on the client correctly, but this will work fine for a home setup where you’re in control of all the layers (network, client and server).

Good luck.

Netflow Collector on Splunk – Interesting Bug

The Splunk Add-on for Netflow appears to have a bug.  If you run through the configure.sh script accept all the defaults, it refuses to ingest any Netflow data.

This is because its script deletes all ASCII netflow data that’s older than -1 day old.

You can easily fix this by either rerunning configure.sh again and typing in every value, or edit /opt/splunk/etc/apps/Splunk_TA_flowfix/bin/flowfix.sh and change the following line;

# Cleanup files older than -1
find /opt/splunk/etc/apps/Splunk_TA_flowfix/nfdump-ascii -type f -mtime +-1 -exec rm -f {} \;

Change the +-1 to +1.  This tells the script to clean up all ASCII netflow data older than 1 day (ie, not everything older than some time in the future).

Splunk integration with Docker

I’ve changed over my log aggregation system from ElasticStack to Splunk Free over the past few days.  The primary driver for this is that I use Splunk at work, and since Splunk Free allows 500Mb/day of ingestion, that’s plenty for all my home stuff.  So, using Splunk at home means I gain valuable experience at using Splunk professionally.

What we’ll be talking about here is how you integrate your Docker logging into Splunk.

Configure an HTTP Event Collector

Firstly, you’ll need to enable the Splunk HTTP Event Collector.  In the Splunk UI, click Settings -> Data Inputs -> HTTP Event Collector -> Global Settings.

Click Enabled alongside ‘All Tokens’, and enable SSL.  This will enable the HTTP Event Collector on port 8088 (the default), using the Splunk default certificate.  This isn’t enormously secure (you should use your own cert), but this’ll do for now.

Now, in the HTTP Event Collector window, click New Token and add a token.  Give it whatever details you like, and set the source type to json_no_timestamp.  I’d suggest you send the results to a new index, for now.

Continue the wizard, and you’ll get an access token.  Keep that, you’ll need it.

Configure Docker Default Log Driver

You now need to configure the default logging method used by Docker.  NOTE – Doing this will break the docker logs command, but you can find everything in Splunk anyway.  More on that soon.

You will need to override the startup command for dockerd to include some additional options.  You can do this on CentOS7 by creating a /etc/systemd/system/docker.service.d/docker-settings.conf with the following contents;

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --log-driver=splunk --log-opt splunk-token=PUTYOURTOKENHERE --log-opt splunk-url=https://PUTYOURSPLUNKHOSTHERE:8088 --log-opt tag={{.ImageName}}/{{.Name}}/{{.ID}} --log-opt splunk-insecureskipverify=1

The options should be fairly evident.  The tag= option configures the tag that is attached to the JSON objects outputted by Docker, so it contains the image name, container name, and unique ID for the container.  By default it’ll be just the unique ID, which frankly isn’t very useful post-mortem.  The last option allows the use of the Splunk SSL certificate.  Get rid of this option when you use a proper certificate.

Getting the driver in place

Now you’ve done that, you should be able to restart the Docker host, then reprovision all the containers to change their logging options.  In my case, this is a simple docker-compose down followed by docker-compose up, after a reboot.

The docker logs command will be broken now, but you can instead use Splunk to replicate the functionality, like this;

index=docker host=dockerhost | spath tag | search tag="*mycontainer*" | table _time,line

That will drop out the logs from the last 60 minutes for the container mycontainer running on the host dockerhost.

You can then start doing wizardry like this;

index=docker | spath tag | search tag="nginx*" 
| rex field=line "^(?<remote_addr>\S+) - (?<remote_user>\S+) \[(?<time_local>.+)\] \"(?<request>.+)\" (?<status>\d+) (?<body_bytes>\d+) \"(?<http_referer>.+)\" \"(?<http_user_agent>).+\" \"(?<http_x_forwarded_for>).+\"$"
| rex field=request "^(?<request_method>\S+) (?<request_url>\S+) (?<request_protocol>\S+)$"
| table _time,tag,remote_addr,request_url

To dynamically parse NGINX container logs outputted by Docker, split up the fields, and then list them by time, remote IP, and the URL requested.

I’m sure there’s better ways of doing this (such as parsing the logs at index time instead of at search time), but this way works pretty well and should function as a decent starting point.