Zigbee2MQTT on RPI with CC2530/CC2591

So, if you’ve flashed a CC2530/CC2591 from my previous post, you now probably want to get it talking to something. Here’s how you can do that.

Assumptions

I will assume you are wanting to do the following;

  • Use a Raspberry Pi 2/3 running Raspbian to act at the bridge between the CC2530/CC2591 and MQTT.
  • You’re using Raspbian Stretch.
  • You want to use Zigbee2MQTT to get this thing talking to something like HomeAssistant.
  • You want to use the RPI’s built-in UART and directly wire the module to the RPI.
  • You don’t care about Bluetooth on the RPI.
  • You already have Docker installed on the RPI.
  • You already have Docker-Compose installed on the RPI.
  • You already have your CC2530/CC2591 flashed with KoenKK’s Z-Stack firmware, and it’s a recent version.
  • You have a mqtt server somewhere already.

Phew. Now with all that in line, let’s get moving.

Hardware Setup

CC2530/CC2591 Pinout Diagram
RPI2/3 GPIO Pinout

Using the two charts above, you will need to make the following connections;

PIN PURPOSECC2591RPI PIN
VCC (Supply)VCC3V3
GND (Ground)GNDG
RXD (Receive Data)P0_2RPI Pin 8 (TXD)
TXD (Transmit Data)P0_3RPI Pin 10 (RXD)

This is the minimum set of pins required. Note that RXD on the CC2591 gets connected to TXD on the RPI. This is normal.

Do not connect the CC2530 to the 5V lines on the RPI. Doing so will likely destroy the CC2530.

Configure UART on the RPI

What we’ll be doing is using the UART on the RPI. There are a number of ways to do this, but we’ll use the method which disables Bluetooth and puts the UART on the high-performance device /dev/ttyAMA0.

Edit your /boot/config.txt and add the following;

dtoverlay=pi3-disable-bt

Then edit /boot/cmdline.txt and remove the following bit from the only line;

console=serial0,115200

Reboot your Pi, and you should now have the UART pins listed above manifest on /dev/ttyAMA0. Don’t try and use minicom or similar to connect to it, you won’t see much useful.

Set up a HomeAssistant Docker-Compose File

We’re going to use Docker Compose to run zigbee2mqtt in a container. Make a directory somewhere for it, and a data directory, like so;

mkdir -p /srv/zigbee2mqtt/data

Then edit /srv/zigbee2mqtt/docker-compose.yml, and fill it in like this;

version: '2'
services:
zigbee2mqtt:
image: koenkk/zigbee2mqtt:arm32v6
restart: always
volumes:
- /srv/zigbee2mqtt/data:/app/data
devices:
- /dev/ttyAMA0:/dev/ttyAMA0

Now, this will spin up a zigbee2mqtt service when you start it, which will always restart when stopped, using /dev/ttyAMA0 as we defined earlier. Lastly, create a /srv/zigbee2mqtt/data/configuration.yaml and fill it in like this;

homeassistant: true
permit_join: true
mqtt:
base_topic: zigbee2mqtt
server: 'mqtt://YOURMQTTSERVERHERE:1883'
include_device_information: true
serial:
port: /dev/ttyAMA0
advanced:
log_level: info
baudrate: 115200
rtscts: false

I strongly suggest you change the network key, and disable permit_join when you have all your devices up. There’s various other things to do here too, but this should get you started.

Once that’s done, a simple;

docker-compose up

Should bring up your container. Press Ctrl-\ to break out without terminating it.

AnyCubic i3 Mega Upgrades

After setting up my printer and doing some test prints and other things, I immediately set forth printing out upgrades for the printer. The intention here is to resolve some deficiencies in the design.

Filament Guide

Filament Guide

I printed a filament guide from Thingiverse. This needed to be reprinted slightly oversize and trimmed to fit, but it clips around the right-hand side (from the front) allowing filament to go through it and be guided in a straighter line into the filament sensor. A pretty easy print.

Filament Sensor Guide

Next, the filament sensor default setup is pretty awful. The magnetic ball disconnects all the time, leaving the filament sensor bouncing up and down a lot. So I printed a replacement filament sensor guide from Thingiverse.

Filament Sensor Guide

The design as given has no tolerances built in, so it may be difficult to fit. In particular, I had to trim the flat below the screw hole and widen the hole itself so it would fit. In addition, the far end makes contact with the adjustment screw for the pinch grip on the stepper motor, so I had to cut that off with a knife to make clearance. If I knew more what I was doing with Fusion 360 I’d redesign it to have more clearance, but there you go.

As it is now, it works great – it retains the filament sensor and allows it enough movement to not cause any problems.

Y-Axis Tensioner

Next is a Y-axis tensioner. I noticed that there was some effects caused by a slightly wobbly belt, so I printed a tensioner mechanism which can adjust the belt tension from Thingiverse.

Y-Axis Tensioner

Now there’s some notes about this thing. First of all, you MUST have a lock nut on the adjustment screw, otherwise vibration will work it loose and it’ll ruin your print. I found out this the hard way. Second, don’t tension up the belt too tight, it needs to be tight enough but not enough to bend the stepper pulley. I did it enough that if I push the belt together with my fingers near the middle, I can still get them to touch without undue force.

I had to do a reasonable amount of trimming and fitting to get it to all fit together, sizing holes with a drill bit held in the hand and trimming faces with a craft knife. The assembly order is quite important, so carefully examine how it all works before you try and put it together. Don’t overtighten anything, getting too brutal will result in the PLA failing.

I’ll likely reprint this in ABS when I get that dialed in.

Squash Ball Feet

Everything from here required turning the printer on its side and removing the bottom plate. Ghosting is a printing phenomenon where “echoes” of features on a print appear later on the print. It is worsened by higher print speed. Here’s an example from Calibration Cube;

Ghosting example

What’s happening here is that vibration from the print head changing direction travels through the surface the printer is installed on, then reflects and comes back. When the echo arrives it leaves the ghost you see. So it’s interesting because tightly mating the printer to the surface it’s on (for example a table) may actually make the ghosting worse, and a larger table provides more time for the echo to travel before it comes back. The solution here is to isolate the printer from the surface that it’s on. Enter these feet, printed from Thingiverse, with squash balls inserted into them.

Squash Ball Feet

The feet were easy to print, and work well. They do make the printer quite a lot higher though (~51cm total from base to top), but they dramatically reduce ghosting effects.

PSU Replacement Fan Cover

The PSU fan on this unit is horribly loud. I put this down to there being hardly any clearance between the thin aluminium shell of the PSU and the metal bottom plate, so it vibrates awfully. So, I printed a replacement PSU fan cover from Thingiverse (the 60mm version) and installed the recommended Noctua FLX fan.

Replacement PSU Cover

I’m not terribly impressed with the print, it fits, but it’s off by a little. It works OK though. I’m also not enthralled with using PLA for something that may be exposed to heat, but I wanted to be sure of the amount of cooling I had before I did a big print in ABS – one of my first projects with the fire retardant ABS I have on order is to print a replacement cover for this in that. But it dramatically cuts down noise while improving airflow.

Controller and Stepper Drivers

I haven’t replaced these, but here’s a view of the Trigorilla control board with the default stepper drivers;

Trigorilla Board with default stepper drivers

I have a set of replacement stepper drivers on order, which should improve print quality and resolution markedly. I’ll install those when I have replacement internal components printed in ABS.

Stepper Driver Cooling Duct

The default cooling setup for the stepper drivers is woeful. It features a fan with about 1mm of clearance to a solid steel section of the bottom plate, meaning there is virtually no airflow over the stepper drivers. Considering these generate a lot of heat, this design is pretty poor. So, we simply print a stepper driver cooling duct from Thingiverse.

Stepper Driver Cooling Duct

This uses a 40mm Noctua FLX fan, and due to the design you can only actually fit it while the power supply is out. But it does fit, and provides a directed flow of air straight over the stepper drivers and out of the case. I also intend to reprint this in fire retardant ABS.

Final Notes

It’s pretty ingenious that you can print upgrades and fixes for your own printer. However, note that your ability to print upgrades depends on the upgrades you have already printed working! So I would strongly suggest that when you find a working upgrade that is critical to the functioning of the printer (eg, tensioners) that you print two of them. I also ordered spare drive belt (GT2x6 pulley belt) and some spare pulleys, since I expect a belt will go at some stage, and I can’t reprint that.

Next up I’m printing some cable chains to help keep things neater. I also printed a side mount for an OctoPi which houses a camera, but I don’t know if I’ll use that in the end, it depends how I do my final setup.

I also made a Wemos-powered status monitor that can sit on my desk, but that warrants its own post.

Lastly, I’m looking into making an enclosure using some IKEA Lack tables and printed fittings for them, but that will also warrant its own post.

Flashing Z-Stack on a CC2530+CC2591 using a Wemos D1 mini

I’m messing about with Zigbee for a comms protocol to various temperature sensors, and this requires a Zigbee Coordinator. There’s a few ways of doing this, but ultimately I settled on a zigbee2mqtt bridge and a cheapie AliExpress CC2530+CC2591 module.

This module incorporates an RF amplifier, but does not have the normal debug header that the CC2530 Zigbee transceivers have and also lacks the USB-TTL adapter chip. Not a problem if you’re using a RPi as the bridge, which is what I plan on doing.

However first, you need to get Z-Stack firmware on it, so you can use it as a coordinator. This proves to be… non-trivial. Especially if you want to use a Wemos D1 Mini as the flashing device (these Wemos things are really good, incidentally).

First Steps – Getting CClib-Proxy onto the Wemos

Assuming you have a Wemos D1 mini, your first steps are to install the Arduino IDE (available from the Windows Store). Once that’s in, in Preferences, add the following URL to the Additional Boards Manager URL field;

http://arduino.esp8266.com/stable/package_esp8266com_index.json

From there, you should now be able to go to the Boards Manager, and install the esp8266 package. Once that is installed, configure your board as a “LOLIN(WEMOS) D1 R2 & Mini” and select the correct COM port.

Now it’s as simple as downloading CCLib-Proxy from this link. Open up CCLib_Proxy.ino, then change the following lines for the pinout;

 int CC_RST   = 5;
int CC_DC = 4;
int CC_DD_I = 14;
int CC_DD_O = 12;

These mappings are required. Upload to your device. You now have CClib-Proxy onto the Wemos and ready to go.

Wiring up the Wemos to the CC2530+CC2591 Module

You will need to map various pins on the Wemos to pins on the module, using the following chart;

PIN PURPOSENUMBER ON CC2591NUMBER ON WEMOS
DC (Debug Clock)P2_2D2 (GPIO4)
DD (Debug Data)P2_1D5 (GPIO14)
D6 (GPIO12)
RST (Reset)RSTD1 (GPIO5)
VCC (Supply)VCC3V3
GND (Ground)GNDG
RXD (Receive Data)P0_2RPI Pin 8 (TXD)
TXD (Transmit Data)P0_3RPI Pin 10 (RXD)
CTS (Clear To Send)P0_5RPI Pin 11 (RTS)
RTS ( Ready To Send)P0_4RPI Pin 36 (CTS)

When using a Wemos as the flashing device, it’s safe to tie the two I/O pins together (D5 and D6) and connect them to the DD pin on the CC2530. It works fine. The P0_2 through P0_5 pins are used when you’re using the finished device, not when flashing (so you don’t need to connect them up).

Pinout for CC2530+CC2591 module

The above diagram shows the pin mappings on the CC2530+CC2591 module itself. Follow those numbers and the pins above to wire it up.

Pinout of Debug Header on CC2530 (not present on combined module)

This diagram shows the pinout of the debug header (which is not present on the CC2591). However, it does show which pins on the CC2591 marry up to what purposes on the debug header (which correspond to pins on the Wemos).

After this is done, you need to use CClib to flash the firmware.

Flashing the Z-Stack Firmware

Get the firmware from this link. You will also need to install Python 2.7 for your system. Once that’s done, install pyserial with;

pip install pyserial==3.0.1

Edit the firmware .hex you downloaded, and remove the second to last line (it won’t work with the flasher you’re using). Once that is done, connect your Wemos to your computer, and then from the Python directory in your CClib download, run;

python cc_info.py -p COM9

Assuming that COM9 is your Wemos. You should see output giving you data on the CC2530. If so, fantastic. Now flash it;

python cc_write_flash.py -e -p COM9 --in=YOURFIRMWAREHERE.hex

This will take an extremely long time. Several hours. But you should see progress fairly quickly. Just hang tight. Once that’s done, you have a coordinator!

Next post will deal with testing the coordinator out.

References and Links

AnyCubic i3 Mega Tuning

After having my AnyCubic for a week, I’ve done a fair bit of tuning and mucking about. I’ve printed several benchmarking prints and calibrated the steppers. It’s printing fairly well now. Here’s what I did.

Initial Calibration

Firstly, you’ll need a set of calipers – a ruler isn’t good enough. Then, consult the following guide;

3D Printer Calibration Guide

The very first step is to get the first layer adjusted. Fortunately with Marlin, this is quite easy and you can use mesh bed levelling. Get the bed fairly flat initially using the normal levelling technique that the AnyCubic includes, then follow this guide. Use a thin piece of paper (thermoprint receipt paper is perfect), and do it while the bed and nozzle is hot.

After you’re done, send the following G-Code commands to print a levelling test (200 degree hot end, 60 degree bed);

G28
G26 C H200 P5 R25 Q4.2 Z4

The next part is to calibrate the extruder stepper. I measured out 130mm of filament, and told it to extrude 100mm. A longer length gives better accuracy. Read the calibration guide for instructions.

After doing that, measure the thickness of your filament in a few points and adjust the filament thickness in Cura to suit. In my case, my filament is a LOT thicker than 1.75mm (it’s actually 1.98mm), which makes a difference to extrusion and can throw off the next test.

Now you have your extruder calibrated, you can print a calibration cube. After that’s printed, you can measure its dimensions and then adjust your XYZ steppers.

At this point, all your axes should be calibrated, your bed is level, and you know how thick your filament is – so the majority of the dimensional calibrations are done.

Temperature Tuning

You next need to print a temperature tower. However, this will require some manipulation in Cura to make it work. Consult the readme, and in there you will find a list of the layer Z heights that should correspond to what temperatures.

From there, you can use Cura extensions to fix it. Set your print temperature to 180 (which corresponds to the “floor” temperature setting in the readme), then click Extensions->Post-Processing -> Modify G-Code.

Click Add a Script, then ChangeAtZ. Specify the height as per the README, then check the “Change Extruder 1 Temp” box, and enter the temperature for that height from the readme. Click Add a script again, and repeat. You’ll need to do this a bunch. Sorry.

Print it off, and then examine it. Select the temperature that looks best (for me that was 190 degrees), and that will be your new default print temperature.

Cura Profile Tuning

Right. Now you’ve done that, it’s time to adjust your Cura profile. What works for you may be different from what works for me, but I adjusted the following settings and got fairly good results so far (this is all based from the default Normal profile);

  • Quality
    • Layer Height: 0.2mm
    • Initial Layer Height: 0.3mm
  • Shell
    • Wall Thickness: 1.2mm
    • Top Surface Skin Layers: 1
    • Top/Bottom Thickness: 1.2mm
    • Enable Ironing: OFF
  • Infill
    • Infill Density: 20%
  • Material
    • Default Printing Temperature: 190
    • Build Plate Temperature: 60
    • Build Plate Temperature Initial Layer: 70
    • Retraction Distance: 6.5mm
    • Retraction Speed: 50mm/s
  • Speed
    • Print Speed: 60mm/s
    • Infill Speed: 60mm/s
      • Outer Wall Speed: 45mm/s
      • Inner Wall Speed: 60mm/s
    • Top/Bottom Speed: 30mm/s
    • Support Speed: 80mm/s
      • Support Interface Speed: 40mm/s
    • Print Acceleration: 1800mm/s
    • Print Jerk: 8mm/s
  • Travel
    • Combing Mode: Not in Skin
    • Z Hop When Retracted: YES
  • Cooling
    • Fan Speed: 80%
    • Initial Fan Speed: 0%
    • Regular Fan Speed at Layer: 4
  • Support
    • Generate Support: YES
    • Support Placement: Touching Buildplate
    • Support Overhang Angle: 60
    • Support Density: 15%
  • Build Plate Adhesion
    • Build Plate Adhesion Type: Brim

Some words about why these settings were picked is relevant – some of them I haven’t actually tweaked, but I know they are relevant.

  • Quality: Layer Height is picked according to the resolution you want. The initial layer is selected to be 0.3mm to aid plate adhesion.
  • Shell: Thicknesses can be reduced if you want a faster print at the expense of speed. 0.8mm is a common choice. Ironing will get the nozzle to wipe over the top layer to smooth it off.
  • Infill: Change density as desired for part strength.
  • Material: 190/60 fits my filament well. The initial layer temperature promotes good adhesion of the first layer to the bed, hence the elevated temperature. Retraction distance and speed were tweaked to reduce stringing effects.
  • Speed: Print speeds can be adjusted downwards, but I wouldn’t adjust them upwards. These fit the recommended speeds of the AnyCubic. The outer walls and top/bottom are slower to promote better surfaces. I haven’t tweaked the acceleration and jerk much, but they are enabled so I can mess with them later.
  • Travel: Changing Combing Mode will reduce the effect where strings are placed just under the skin layers, leading to ugly looking tops/bottoms. There is a small cost in print speed, but in my opinion it’s worth it.
  • Cooling: The fan at 100% is rather excessive, hence ratcheting it down. The initial fan speed and layer settings promote better bed adhesion by reducing cooling for the first few layers, which then ramps up as the print proceeds.
  • Support: I set Touching Buildplate so I don’t get internal supports, which are horrible to remove. This needs to be tweaked as per your needs.
  • Build Plate Adhesion: Brim adhesion prints a single-layer brim around your piece. It greatly improves adhesion and reduces warping, and is pretty easy to remove from a finished piece with your fingernails. Worth having as a default.

Start G-Code Customization

Ok. After all the above is done, you should have your temperatures dialed in, Cura set up with some useful parameters, and your axes all calibrated. Your bed should be flat. Next up is to tweak the G-Code used to start the build.

I’ve selected to perform a nozzle wipe procedure after initial setup. This procedure prints a short line of filament at the front of the bed before wiping the nozzle away to clear it.

The G-Code I’m using for Start G-Code looks like this;

;;;;; cura zeroing with nozzle wipe ;;;;;
G21 ;metric values
G90 ;absolute positioning
M82 ;set extruder to absolute mode
M107 ;start with the fan off
G28 X0 Y0 ;move X/Y to min endstops
G28 Z0 ;move Z to min endstops
M501 ;marlin load eeprom
M420 S1 ;marlin use mesh levelling
G0 X0 Y0 Z0.15 F{speed_travel} ;move 0.15mm from front left corner of platform
G92 E0 ;zero the extruded length
G1 X40 E25 F{speed_layer_0} ;extrude 25mm of filament in a 4cm line at initial layer print speed
G92 E0 ;zero the extruded length again
G1 E-1 F{wipe_retraction_speed} ;retract 1mm
G1 X80 F4000 ;quick wipe from the filament line
G1 Z15.0 F{speed_travel} ;move the platform down 15mm
G1 F{speed_travel}
M117 Printing...
G5

This is pretty similar to the default Cura settings for the AnyCubic. A few things warrant talking about though.

The M501 / M420 steps enable the Mesh Levelling you set up in Marlin. This should be done just after reaching the home position.

The steps after mesh leveling and before the final platform position change (setting the head 15mm above the platform) perform the wipe.

If you don’t want to do this, replace that section (including the G1 Z15.0 step) with;

G1 Z15.0 F{speed_travel} ;move the platform down 15mm
G92 E0 ;zero the extruded length
G1 F200 E3 ;extrude 3mm of feed stock
G92 E0 ;zero the extruded length again

That is the Cura defaults. The above settings should produce decent results.

Phew! What now?

Ok, that was a lot. Now that’s done, I’m printing various other bits and pieces for the printer – more will come as I get it done.

Oh, and if you’ve set up the printer and want to give it a stress test, print a 3dBenchy with no supports enabled. See how it goes!

AnyCubic i3 Mega Marlin Firmware

So I’ve been doing more mucking about with my printer, and I elected to install a custom firmware onto the unit. This firmware is a port of the Marlin 1.1.9 firmware to be compatible with the AnyCubic i3 Mega v3.

Oh, it turns out there are four versions of the AnyCubic i3 Mega kicking around. You can find a breakdown of them here. Mine is a v3 – identifiable because it has the Ultrabase, v1.1 firmware, and the SD card board is separate from the mainboard (visible through the slots in the bottom plate). The difference is very important when it comes to firmware updating.

The custom firmware has a number of benefits – most notably for me is mesh bed levelling. My bed is pretty flat, but there is a slight concavity in the center (of the order of less than 0.1mm, but it affects adhesion in the very center). So before I went too crazy with calibrating the printer, I wanted to have a better firmware onboard.

OctoPrint and OctoPi

First of all, firmware updating requires a terminal. Since I have many Raspberry Pis kicking around, the easiest way to get what I want is to install OctoPrint. This comes in a distribution called OctoPi, and has a lot of awesome features including, critically, a firmware updater plugin.

Installing OctoPi was a breeze. Then after that, I installed the Firmware Updater, and followed the instructions to set it up. Specifically, the programmer settings are;

  • AVR MCU: Atmega2560
  • Programmer Type: wiring

After applying the firmware update and restarting the printer, the About on the TFT does not show any change. This is expected. You can, however, see the firmware version information in OctoPi on the Terminal display when the printer starts up.

Once that is installed, it is critical that you reset the printer to factory defaults with;

M502 ; Load default values
M500 ; Save to EEPROM

Once that is done, you can then proceed with mesh bed levelling and so-on.

I also picked up a Microsoft LifeCam H5D-00016, which can be assembled into a housing for OctoPi with camera here. I’ll be putting that together as a project pretty soon.

3D Printing – First Steps

So I bought myself a 3d printer. I decided on getting an AnyCubic i3 Mega printer, which was pretty cheap considering. Reviews were fairly decent.

Anyway, the box arrived, and it was packed pretty well, and proved to be quite painless to do initial setup;

The base unit (holding electronics), and the manuals and tools
The spool of PLA the unit comes with, and the gantry that holds the extruder
The tools and spare parts the printer came with

The printer came with all the tools required to assemble it, as well as various quality of life tools such as a scraper, tweezers, cutters, SD card and USB-to-SD adapter, USB cables, and even some spare parts such as a spare hot end and a spare limiter microswitch.

Assembly was painless, though the plugs were pretty hard to press in. After that, I fed in the PLA, and ran the test print (which came on the SD card);

The test print worked out fine. A small amount of stringing, but it ‘just worked’, no calibration was done here.

I also noticed a few defects which can be fixed. First up, the fans in this unit are REALLY LOUD. The reason for this is that the fans (on the PSU and on the control board) are much too close to the bottom plate, and the bottom plate obstructs the control board fan.

The fix for this is to print a new PSU cover, and replace the control board cooler with a ducted fan. I’ll print these shortly.

I did print an extruder knob, but I had to reprint it at 103% scale to get it to fit the NEMA17 stepper. This is likely because the printer is just at factory calibration right now.

Next up for me is some calibration stuff, rotating the bottom plate to clear the obstruction on the cooling fan, and getting the fans / printing replacement covers and cooling ducts to make it work better.

AussieBroadband CheckMK Plugin

I’ve recently changed my ISP to AussieBroadband.  Since I’m now working under a quota, I want a way to monitor my quota in CheckMK.  Enter a bit of python.  If you want to use this, you’ll need to adjust the hashbang to suit your actual OMD site, and then pass a parameter which is your username:password to get onto your ABB account.

#!/omd/sites/checkmk/bin/python2.7
#
# Parses AussieBroadband Quota Page to generate a CheckMK alert and stats pages
#

import requests
import re
import time
import sys
import json

status = 0
statustext = "OK"

try:
creds = sys.argv[1].split(":")

# Create a new session
s = requests.Session()

# Process a logon
headers = {
'User-Agent': 'abb_usage.py'
}
payload = {
'username': creds[0],
'password': creds[1]
}
s.post('https://myaussie-auth.aussiebroadband.com.au/login', headers=headers, data=payload)

# Fetch customer data and service id
r = s.get('https://myaussie-api.aussiebroadband.com.au/customer', headers=headers)
customer = json.loads(r.text)
sid = str(customer["services"]["NBN"][0]["service_id"])

# Fetch usage of the first service id
r = s.get('https://myaussie-api.aussiebroadband.com.au/broadband/'+sid+'/usage', headers=headers)
usage = json.loads(r.text)
quota_left = usage["remainingMb"]
quota_up = usage["uploadedMb"]
quota_down = usage["downloadedMb"]

# Derive some parameters for the check
total = quota_left + quota_up + quota_down
critthresh = 0.10*total
warnthresh = 0.25*total

# Determine the status of the check
if quota_left < critthresh:
status = 2
statustext = "CRITICAL"
elif quota_left < warnthresh:
status = 1
statustext = "WARNING"

# Format the output message
print "{7} - {1} MB quota remaining|left={1};{2};{3};0;{4}, upload={5}, download={6}".format( \
status, \
int(quota_left), \
int(warnthresh), \
int(critthresh), \
int(total), \
int(quota_up), \
int(quota_down), \
statustext)

except:
print "UNKNOWN - Unable to parse usage page!"
status = 3
statustext = "UNKNOWN"

sys.exit(status)

Enjoy.  It’s pretty quick and dirty, but it works.  You put this into your site’s local/lib/nagios/plugins directory, then add it as a classical check.

Darktable on Windows through WSL

EDIT:  No longer required.  Since Darktable 2.4.0, there’s an official native Windows installer for it.  Use that instead.  Easy!

With the Windows Subsystem for Linux (WSL) now being much more stable and useable, it turns out it’s possible to install Darktable on Windows with very little fuss.

This will require you to have Windows 10, and also to have at least the 1709 Fall Creator’s Update (run winver and see your version, it should be 1709 or higher).

Follow the instructions here to install WSL, and then go ahead and install Ubuntu from the Windows store.  Don’t bother starting a prompt yet, we have more to do.

Next, you’ll need an X server of some type, to display graphical UI from Ubuntu apps on the screen.  Unless you have something else, I suggest you install VcXsrv, it’s straightforward to install and run.  When running this, just select all the defaults and go ahead.  This will give you an X server on :0, which we will use in a moment.

Now, start up Ubuntu, then type nano ~/.profile and press enter.  Enter the following text down the bottom,

# set display
if [ "$DISPLAY" == "" ]; then
  export DISPLAY=localhost:0
  #export LIBGL_ALWAYS_INDIRECT=1
fi

Press Ctrl-O and Y in order to save.  Now exit that Ubuntu window and start it up again.  If you type echo $DISPLAY you should see the variable above printed out.  This tells programs in your Ubuntu window how to find your X server.

Next, go to this PPA repository, and install Darktable like this;

sudo add-apt-repository ppa:pmjdebruijn/darktable-release
sudo apt-get update
sudo apt-get install darktable

Wait a bit, and Darktable will be installed.  You can now run it by simply typing darktable into that prompt.

No mess, no fuss.  Enjoy.

 

Stopping DNS leakage with pfSense

I’ve recently changed my core router over from OpenWRT to pfSense.  I was pretty happy with OpenWRT, but I wanted something more powerful since it was running in a VM anyway.

A few days ago, CloudFlare announced their new 1.1.1.1 service.  This is a public DNS service very much like Google’s 8.8.8.8 DNS service, with a notable difference.  It supports TLS.

Why should you care?  Because DNS requests are normally not encrypted, and therefore visible to your ISP to record, use for research / marketing purposes, or even (in the case of some nefarious actors) manipulate or change.  Running DNS over TLS prevents that, by encrypting your DNS traffic so that it can’t be manipulated or collected.

In this post, we’ll be configuring pfSense to do three things – provide a local standard unencrypted port 53 DNS resolver which uses CloudFlare’s 1.1.1.1 encrypted service on the WAN end, and then set up a NAT redirect so any attempts on the internal network to use port 53 DNS servers outside the network instead are intercepted and resolved by the internal resolver.  Lastly, it will also make sure that it blocks any outbound requests to port 53 just to be sure.

NOTE:  There’s one piece here I haven’t figured out yet.  How to pin a cert for the DNS endpoints listed here, so it’s not perfect.  When I figure that out, I’ll edit this post.

Let’s get started.

Configuring the pfSense Local Resolver

In pfSense, go to Services -> DNS Resolver, then put the following block into Custom Options:

server:
ssl-upstream: yes
do-tcp: yes
forward-zone:
    name: "." 
    forward-addr: 1.1.1.1@853
    forward-addr: 1.0.0.1@853
    forward-addr: 2606:4700:4700::1111@853
    forward-addr: 2606:4700:4700::1001@853

You will also need to make sure that the DNS Query Forwarding option is NOT selected, otherwise the above settings will conflict.  It’s OK to set the resolver to listen on all interfaces, since the firewall rules on the WAN will prevent Internet hosts from using your resolver anyway.  Follow the prompts, then test it with something like;

dig www.google.com @yourrouter.local

You should see a resolve against your router’s local DNS resolver that works.  If you really want, use Diagnostics -> Packet Capture, and capture port 853 to verify that requests are being triggered.

Redirect all DNS requests to outside DNS servers to pfSense

Follow the article you can find here.  You will need to do this once for each of your interfaces (in my case, LAN, DMZ, and VPN).  Obviously don’t configure this for the WAN interface.  This then causes any requests to addresses that are not on your internal network to be resolved through the local pfSense resolver (which goes out to port 853 anyway).

To test this, try and dig something against an IP that you know is not internal and is not a DNS server.  It should work, since the request will be NATted.  Something like;

dig www.google.com @1.2.3.4

Assuming that’s all fine, you should now be able to configure a broad block rule to bar all outbound port 53.

Block all outbound non-encrypted DNS

This shouldn’t really be required if the NAT rule is working, but we’ll do it anyway to be sure we’re stopping any DNS leaks.

In pfSense, go to Firewall -> Rules, and for the WAN interface, define a new rule at the top of the list.  This rule should use these settings;

Action: Block
Interface: WAN
Address Family: IPv4+IPv6
Protocol: TCP/UDP
Source: any
Destination: any
Destination Port: DNS (53)
Description: Block outbound insecure DNS

After doing this, verify that you can still resolve against the local resolver (your router’s IP), and that you can still resolve against what seems to be external resolvers (eg, 8.8.8.8).  You should also check that when you do so that nothing passes on the WAN interface on port 53.

If that all passes, you’re done.   It’s up to you if you use the ‘Block’ target or the ‘Reject’ target.  Block causes a simple timeout if something hits 53 (which shouldn’t happen anyway), Reject causes an immediate fail.

Resources

Ubuntu replaces /bin/sh with Dash

Trap for young players.  Ubuntu replaces the default interpreter for /bin/sh from Bash to Dash.  This was done for performance reasons, but certain scripts really don’t like this.  You can easily change it back with;

sudo dpkg-reconfigure dash

Information about this can be found here.  This was done quite a long time ago, apparently, but for whatever reason scripts that ultimately wind up calling the PHP 7.1 interpreter while under Dash break badly under some circumstances (resulting in PHP segfaulting).