ENVI-R & MRTG – Overview

I recently had an ENVI-R wireless power monitor installed, and I set it up to record data to an always-on Ubuntu Linux box I have sitting around using MRTG.  The setup required a fair bit of scripting in Perl, Bash, and a couple of extra bits of software.

This post is the first in a series outlining just how I set up the monitor and what was required.

The ENVI-R and accessories

I bought the ENVI-R unit in a pack which included the transmitter, a receiver LCD display, power supply and one current clamp.  I also bought an additional two current clamps and the USB connector cable for the receiver.

The USB connector is actually a specially wired Prolific PL2303 USB to RS232 serial port adapter, with an RJ45 connector on the end.  I don’t know the exact pinout, but that’s not required.  Anyway, the adapter “just works” with Ubuntu 10.10.

Each transmitter unit can handle up to three current clamps, and one receiver can handle up to ~10 transmitters.  I didn’t want to buy additional transmitters, and there isn’t a huge amount of space in my switchboard, so I just got an additional two current clamps bringing me to three.  That allows for the monitoring of up to three loads.

Clamp Installation

WARNING – Mains voltage can be lethal.  In addition, tampering with your switchboard without an electrician’s license may be illegal, as well as dangerous.  Get an electrician to install the clamps for you.

The clamps themselves are no-contact types, and go around the active wire of whatever feed you want to monitor.  They should be clamped entirely around the wire so that the ferrite core of the clamp encircles the active wire.  The clamp operates by picking up the EM field around the active wire as current passes.

Be aware that the clamp cannot identify the direction that current is flowing.  This isn’t a problem in the case of a house like mine that has no solar power, but if you have solar power you’ll want to install a clamp onto the feed coming from the solar cells, and then put your main power clamp after the location where the solar power feed connects to your main feed.

In my case, my electric hot water is on a different circuit from the main power, so the three clamps were connected as so;

  • Main power:  Clamp connected after main breaker, between breaker and switchboard.  Registers all power going into the house (except hot water)
  • Lights:  Clamp connected after breaker for the lights (I only have one).
  • Hot water:  Clamp connected after breaker for the hot water.

Be aware as well that the ENVI-R assumes if you have multiple clamps connected to one transmitter that you’re measuring 3-phase power, and it therefore just adds together all the currents.  In the case of mains + hot water that’s OK, but since my “main” clamp is actually registering the sum of lights and everything else, it’ll over-read if the lights are on.  For that reason, if you’re setting up like me, don’t trust the display, use the serial data feed.

ENVI-R Receiver Installation

The receiver is pretty straightforward, and it’s just plugged in, and then the USB serial cable is hooked up.  I’ll run through the software to actually interpret the data in a useable format for MRTG later.

The ENVI-R communicates at 57600 8N1 speed.  If you want to just see the raw output, run this command from the command line;

$ stty -F /dev/ttyUSB0 speed 57600
$ cat /dev/ttyUSB0

You should see, about once every five seconds, output something like this;

CC128-v1.310002222.00033221000000029100000

If you do, great.  The ENVI-R is all hooked up.  Note that it also monitors temperature, and that temperature is the temperature of the receiver.  Note, if you don’t have a transmitter attached, you won’t see any output from the ENVI-R at all.

The Software

For this kind of thing, most people seem to use Cacti.  While it’s definite that Cacti can do more, the box I’m using has very limited memory, so I need to keep the number of running daemons to an absolute minimum.  MRTG can run as a Cron job, and doesn’t require a database daemon to be running as well.  For the small number of graphs I’m talking, Cacti is overkill.

In order to pass data to MRTG, I used a message broker written by IBM (Really Small Message Broker), since it does the job and it’s tiny.  Then I wrote a couple of Perl scripts to handle converting the raw ENVI-R data into moving averages for MRTG to slurp up.  So, the software required is;

  • IBM’s RSMB or other MQTT-compatible message broker
  • MRTG for generating the graphs
  • A web server of some type.  I used THTTPD, since it’s very small and fast.
  • Perl.  It’ll come with Ubuntu, but you will need a number of CPAN libraries.

Up Next

If you get to this point, you should have a connected up ENVI-R with a few clamps, and it should be hurling data out to /dev/ttyUSB0 on your Linux recording box.  Fantastic.

Now for the software….

PowerCLI: Get-VMSize

I’m employed as a server admin, with most of my time spent working with VMware and managing a reasonably sized fleet of machines.  As such, I have a range of various Powershell scripts I’ve written to take advantage of VMware’s PowerCLI interface for Powershell.  PowerCLI is, in a word, great.  It provides some pretty good in-depth insight to what’s going on in vCenter, and since it ties into Powershell, it’s easy to script up whatever you want to do.

Anyhow, below is a filter script, which was cobbled together from some code from a source I can’t recall (if you know the original source, let me know so I can attribute it properly!).  The purpose of this script is to take a bunch of VMs, calculate the Size and Used disk space of those VMs, and dump that out.

Begin {

}

Process {
    $vm = $_

    $report = $vm | select Name, Id, Size, Used
    $report.Size = 0
    $report.Used = 0

    $vmview = $vm | Get-View
    foreach($disk in $vmview.Storage.PerDatastoreUsage){
           $dsview = (Get-View $disk.Datastore)
        #$dsview.RefreshDatastoreStorageInfo()
        $report.Size += (($disk.Committed+$disk.Uncommitted)/1024/1024/1024)
        $report.Used += (($disk.Committed)/1024/1024/1024)
    }

    $report.Size = [Math]::Round($report.Size, 2)
    $report.Used = [Math]::Round($report.Used, 2)

    Write-Output $report
}

End {

}

 An example of its use follows;

Get-VM | .\Get-VMSize.ps1 | Measure-Object -Property Size -Sum

The input should be a VMware.VimAutomation.ViCore,Impl.V1.Inventory.VirtualMachineImpl object, such as what gets returned by Get-VM.  Each output object from the script will contain the following fields;

  • Name – The text name of the VM as given by the Name field in the input object.
  • Id – The ID of the VM as given by the Id field of the input object.
  • Size – The sum of the sizes of all disks used by the VM.  This size is the figure set when provisioning the disk, not the actual on-disk allocation (ie, it will be bigger than the disk use if you are using thin provisioning).
  • Used – The amount of actual disk space being used by all disks on the VM.  This size is smaller than Size if the VM is thin provisioned.

Enjoy.

WIDCOMM Bluetooth = Crap

So, I got hold of an i-Mate Smartflip smartphone for work, and wanted to configure it for ActiveSync over Bluetooth. I’ve got a Compaq nc8430 laptop. Here’s where the fun begins.

In a nutshell, ActiveSync doesn’t work using the default WIDCOMM Bluetooth stack that comes with the HP. And the Bluetooth chip that’s on the nc8430 doesn’t work with Microsoft’s Bluetooth stack (that comes with Windows XP SP2).

That’s not entirely true. The chip is completely compatible with Microsoft’s Bluetooth driver and stack. It’s just not in the INF file, so the MS driver won’t install. To fix, we crack out Notepad, open up C:\WINDOWS\INF\BTH.INF, and add in a line for the transceiver, as follows;

[HP.NT.5.1]
“HP USB BT Transceiver [1.2]”= BthUsb, USB\Vid_03F0&Pid_0C24
“HP Integrated BT Transceiver [2.0]”= BthUsb, USB\Vid_03F0&Pid_171D

The bolded part is the bit you add. This then allows the Microsoft Bluetooth driver to think that the BT chip in the nc8430 is compatible, and then allows you to install it (when you select Advanced and pick the driver). When you go and install the driver again, the Microsoft Bluetooth stack installs itself and you’re rolling.

Once that’s done, create an incoming COM port on your laptop. Then turn on discovery, and in ActiveSync on the phone, tell it to connect via Bluetooth, and follow the instructions to pair. Fixed.

Of note, the Microsoft stack doesn’t seem to work too well with the Nokia 6230i. You need to use the WIDCOMM stack for that.

Fixing W32Time in a Guest OS

NOTE: For informative purposes only. I take no responsibility at all for any harm that may result to your environment as a consequence of this information. Use at your own risk, and research appropriately!

Sometimes you must run W32Time on a guest OS, but it’s not a good idea to run it at the same time as using VMWare Tools time synchronization. A good example of this is a domain controller – it must have W32Time running, must have accurate time, and must supply time to member servers.

First, a note. Don’t just go and point your PDC at some dummy NTP source that doesn’t exist. If you do that, after some period W32Time will just shut down and stop serving time. Instead, we need to find a way to get W32Time and VMWare Tools to co-exist peacefully.

The solution is to set W32Time so it only tries to slew the clock very occasionally, so the adjustments made by VMWare Tools dominate the clock and keep it in sync. Since W32Time is still in contact with a valid time source, it doesn’t commit seppuku.

You can do this by changing this registry key to “Weekly”. Data type is REG_SZ, if you need to create it;

HKLM\SYSTEM\CurrentControlSet\Services\W32Time\Parameters\Period

Restart the W32Time service when you do that, and then all should be well. Oh yeah, and after giving it a little while to settle down, don’t forget to check your event viewer, and do a;

w32tm /stripchart /computer:ANOTHER_DC

to check that your machine is still in sync with the rest of your domain.

VMWare & the System Clock

The system clock behaves … strangely … under VMWare in a guest OS. It tends to run slow, and the amount by which it runs slow varies wildly from day to day. Without a fixed time synchronization source, guests will quickly fall out of synchronization and time-critical mechanisms such as Kerberos will break.

This happens because the guest OS assumes that there is a constant period of time between instructions – ie, it has all of the processor’s attention. The system clock is a high-precision clock maintained by this fact. When you put a guest OS into a virtual environment this goes out the window – the guest no longer has a consistent period of time between instructions, in general this time is longer than expected. Therefore means that individual ticks of the system clock take longer than the fixed (outside) time periods they should, and the system clock runs slow.

So, how do we stop this? Well, one solution is to use a guest-internal NTP client such as W32Time. This is a very bad idea. NTP clients adjust the system clock by slewing (speeding up or slowing down) the clock progressively so they can converge the system clock with NTP time. Since the system clock is running at an unpredictable rate, slewing the clock is a recipe for disaster – it causes the clock to swing to and fro and never stabilize. An unstable clock can cause very strange things to happen.

We could also just keep setting the time as we need to. This is also a very bad idea. Modern OSes rely on the continuity of time. If you just keep resetting the clock, the system ‘loses time’, and tasks that were supposed to run in the lost time just don’t happen.

The solution is to let VMWare Tools handle the problem, and check the box that lets it synchronize with the host. When you do this, VMWare Tools slews the clock appropriately so it doesn’t break anything, and your time converges as you’d expect. When you do this, you must turn off any other kind of time sync software such as W32Time, otherwise they will fight over the clock and much havoc will ensue.

There are certain times when you must run an NTP time server such as W32Time (such as on a domain controller). How you go about preventing W32Time and VMWare Tools fighting is an issue for another post.

NTP Time Synchronization in VI3

Time synchronization is of critical importance in a VMWare infrastructure. If it goes wrong, all hell breaks loose, especially in a Windows 200x environment using Kerberos.

Due to how system clocks work in VMWare, this means that you need to use VMWare Tools’ sync capability to keep your VMs right on time. This means that all your hosts need to be properly synchronized.

So, how do you do this? If you’re using any fancy deployment solution like Altiris, disable time synchronization in it. Why? Because if you don’t, you’ll forget six months down the track, virtualize your deployment solution, and then wonder why all your clocks go crazy.

Read this article and implement it. That’ll get your NTP daemon sorted out, but that’s not quite enough. You need to get your machine’s system clock and hardware clock in sync before NTP can slew the clock and keep it synchronized.

In order to do that, get into a console on your VI3 server, and do the following (I assume that firewall.contoso.com is one of your NTP sources, change to suit);

service ntpd stop
ntpdate firewall.contoso.com
hwclock –systohc
service ntpd start
watch “ntpq -p”

That will configure your system and hardware clocks to be close to the NTP source you named, and then start a watch process showing you the state of your NTP peers.

After a while, you should see an asterisk appear next to one of the peers (not LOCAL, that’s your host’s internal clock). When that happens, you’re all good.

Making a transform for any MSI installer

Many MSI installers will let you generate unattended installs, using command-line arguments, but they may not permit the use of a standard transforms (MST) file to make the unattended install. This is a major problem if you are attempting to deploy software via GPO, since you can’t specify a command line.

There’s a way around this, though. Go and get the Windows 2003 SP1 Platform SDK, and install ORCA from it. The SDK is a big download, but c’est la vie.

Once you’ve got ORCA up and running, make a copy of the MSI file you’re customizing (we’ll call them install.msi and install-cust.msi). Then open up install-cust.msi in ORCA. You will see a VAST number of tables. Don’t worry about them too much. Go find the Property table.

Editing the copied MSI

Now, when you use command-line arguments, what actually happens is the MSI inserts those into the Property table when it runs. So, let’s say you needed to add a TARGETDIR=c:\ argument into the Property table. Go look for the TARGETDIR property, and if you find it, edit it. Otherwise add it by right-clicking on the right-hand pane and clicking Add Row. Enter the values as appropriate. When you’re done, save and close ORCA.

Generating the transform MST

From a command prompt, get to a directory that has the two MSI’s in the same location. What we’ll run here is msitran.exe, a Microsoft tool that came with the SDK that generates an MST that’s the diff of two MSI’s.

Run the following command, and you’ll get a transform named install.mst;

“c:\program files\microsoft platform sdk\bin\msitran.exe” -g install.msi install-cust.msi install.mst

Voila! You now have an MST for your original MSI that incorporates the changes you wanted!

Manually running the MSI with the MST

In order to test deploy, you just run the following command. That runs the MSI, applies the MST you created, and does so in basic mode (which is what you’d typically use in an unattended install);

msiexec /i install.msi TRANSFORMS=install.mst

Assuming that works fine, go ahead and deploy via your method of choice.