ENVI-R – Data Broker Setup

James Young · May 1, 2011

Continuing my previous post on this topic, I realized I needed a message broker in order to permit me to have a daemon polling the ENVI-R for data, processing it, and then making it available in a palatable form for various scripts and for MRTG.  It would have been possible for me to simply have a cron job dump a summary table to disk every few seconds, but since my Linux box uses a CompactFlash card for disk storage, that would quickly kill the flash chips.  I needed something that held the messages in memory only.  That’s where a MQTT message broker steps in.


One of my colleagues at work put me on to MQTT, and in particular he put me onto IBM’s Really Small Message Broker (RSMB).

RSMB is a really tiny implementation of an MQTT compliant message broker.  When they say tiny, they aren’t kidding - the broker is a 78k executable which requires an included 73k library.  (config file is optional but a good idea).

Setup Notes

Given that RSMB is such a tiny thing, setting it up is very easy.  However, I did a few extra steps to make it a little more secure and integrated;

  • Dump broker (the executable) into /usr/local/bin.
  • Do the same with stdoutsub and stdinsub .
  • Copy libmqttv3c.so to /lib as libmqttv3c-1.2.0.so and then softlink libmqttv3c.so to it.  This conserves the normal sanity with libraries that ld.so expects, and it should be a bit more maintainable.  Probably not the best way to do things, but it works.
  • Create a new user with no special rights named broker.  After creation, edit /etc/shadow (there’s probably a better way to do this) and change the second field on the broker line to *.  That ensures nobody can log in using the account.
  • Create a config file in /usr/local/etc/broker.cfg (config text follows).
  • Create a new directory in /var/local/broker .  chown that directory to broker:broker.
  • Create a new /etc/init.d/broker script and add it to startup with update-rc.d .

The config file contents are;

# config file for IBM’s RSMB (Really Small Message Broker)</p>

port 1883
max_inflight_messages 500
max_queued_messages 3600
persistence_location /var/local/broker/</span>

</blockquote>

The init.d script is;

#!/bin/bash
### BEGIN INIT INFO
# Provides:          broker
# Required-Start:    $network $local_fs
# Required-Stop:     $network $local_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Should-Start:      broker
# Should-Stop:       broker
# Short-Description: start really small message broker (broker)
### END INIT INFO</p>

pushd /usr/local/bin
su -c "nohup /usr/local/bin/broker /usr/local/etc/broker.cfg >> /dev/null &" broker
popd</span>

</blockquote>

Testing the Install

Broker comes with a couple of other binaries, which can be used to publish messages to a channel, and also to view messages on a channel.  Testing broker is quite easy.

  • Start up broker either manually (good idea the first time) or with the init.d script.
  • In one window, run stdoutsub example to start listening on channel "example".
  • In another window, run stdinpub example and then start typing in lines.  You should see each "message" you enter appear on the stdoutsub window.
  • If you do, great.  If you don't, verify that the ports are all OK and opened on your firewall (if any).

Next Steps

Right, getting the broker up provides the underpinning that gets used by the rest of the software that I'm discussing

From CPAN, you'll need to go and install Websphere::MQTT - that's the CPAN module for interfacing with IBM Websphere's MQTT implementation.  But because it's MQTT compliant, it'll work just fine with RSMB.  That module will be used fairly heavily in the other scripts.  You'll also need Device::SerialPort and Clone , for other parts of the scripts.

Followup posts will be outlining how to set up THTTPD and MRTG, and then tying it all together with Perl and Bash glue.

Twitter, Facebook