Thursday, June 11, 2015

Real-Time Web Interface to MQTT using and Node.js

First, all credit for this tutorial goes to Robert Hekkers Blog.  I've altered it slightly to pick up newer versions of the various javascript libraries.

If you've followed along with my earlier post, you now have MQTT running on your Raspberry Pi, and an Arduino IoT client that can publish and subscribe to MQTT packets.  The next step is developing a real-time web interface that can control your MQTT network.

Why and Node.js?
Web browsers typically operate by pulling data from a server when you click on a link.  Servers don't usually keep an open connection to the browsers it has serviced, so if some event happens on the server side, the server cannot push that event to your browser, unless you refresh the page.

That's where comes in handy. maintains an open connection between the server and the browser, which enables the server to push updates to the browser as they happen.  This is useful so you can see changes to your IoT network as they happen, and not have to wait for a page refresh.

Step 1: Setup a Web Server on your Raspberry Pi
sudo apt-get install apache2 -y
hostname -I
Now test your web server by using a web browser to navigate to your Raspberry Pi's web address

Step 2: Get Socket.IO, Node.js and the MQTT client
sudo apt-get install nodejs
cd /var/www
sudo npm install mqtt
sudo npm install

Step 3: Test Your Node.js and MQTT client
create the file /var/www/mqtt_test.js
node mqtt_test.js
Then change the server URL from to your Rpi's IP address
node mqtt_test.js

Step 4: Get Thomas Reynolds' iOS Style Jquery Checkboxes
cd ~/
cd iphone-style-checkboxes-1
sudo cp -pr jquery /var/www
sudo cp -pr images /var/www
sudo cp style.css /var/www

Step 5: Create a Node.js script to link to MQTT
create the file /var/www/pusher.js
node pusher.js &

Step 6: Create your web page
create the file /var/www/iot_demo.html

Step 7: Test your real-time web interface
mosquitto_sub -t "led"
browse to http://your_rpi_ip_address/iot_demo.html
toggle the checkbox, and you should see messages on your MQTT network Now open multiple browsers and try toggling the checkbox

Thursday, June 4, 2015

Creating your own Internet of Things

These instructions will help you setup your own Internet of Things.

What is the Internet of Things?  Moore's Law accurately predicted that computing density would roughly double every 18 months.  Today we have credit-card sized, fully-functional, general purpose computers that cost $35.  Eventually we will reach a point where embedded computing will be so small, cheap and low-power that many everyday objects (light switches, garage doors, thermostats, kitchen appliances, etc) will include an embedded processor/micro-controller.  These smart objects will be able to communicate, collaborate, coordinate with other objects and human interfaces to automate existing tasks, and enable new capabilities.

Some example ideas:

  • Set the room brightness with your smartwatch
  • Have your lights automatically turn on/off as you travel from room to room
  • See the remaining time on the clothes dryer
  • Track the internal temperature of the pot roast in the oven
  • lower your heating and cooling bills by reducing the climate control in unused rooms
  • unlock your door by simply grasping the handle

While there's many ways to go about it, I'll describe the approach I used.

Things you'll need for a basic setup:

  • Raspberry Pi 2 model B
    • 4GB+ SD card
    • power adapter
    • HDMI cable
    • Ethernet cable or compatible WiFi adapter
    • monitor, keyboard, mouse
  • Arduino
    • Ethernet shield + Ethernet cable
    • USB programming cable
    • PC for programming the Arduino

Step 1: Setup your Raspberry Pi

  1. Follow this guide to install Raspbian Wheezy onto your Raspberry Pi
  2. Install MQTT
    curl -O
    sudo apt-key add mosquitto-repo.gpg.key
    rm mosquitto-repo.gpg.key
    cd /etc/apt/sources.list.d/
    sudo wget
    sudo apt-get update
    sudo apt-get upgrade
    sudo apt-get install mosquitto mosquitto-clients
  3. Subscribe to a  MQTT topic
    mosquitto_sub -h YOUR_RPI_IP_ADDRESS -t hello/world
  4. Publish to a topic
    In another terminal window execute:
    mosquitto_pub -h YOUR_RPI_IP_ADDRESS -t hello/world -m "Hello World"

Step 2: Setup your Arduino

  1. Download the arduino pubsubclient from GitHub
    • Unzip the file and copy it into your Arduino/libraries directory
  2. Build the circuit
    • Connect an LED to pin 3 of the arduino through a 220 Ohm resistor
    • Connect an SPST (normally open momentary contact) switch between pin 2 and ground
  3. Program your Arduino
    • Grab this example Arduino sketch
    • change the server[] to your Rasberry Pi's IP address
    • load the code onto your Arduino
  4. Setup a subscriber to monitor the "led" topic
    • On your Raspberry Pi, open a terminal and subscribe the the "led" topic
      mosquitto_sub -h YOUR_RPI_IP_ADDRESS -t led
  5. Power up your Arduino 
    1. You should see "Hello World" published to the "led" topic
    2. When you press the switch you should see "on" or "off" on the "led" topic.
    3. When "on" appears, the LED should turn on and vice versa.
Here's a code example for the Spark Core:

Useful Links:

Wednesday, July 3, 2013

Arduino Controlled RFID Door Strike

RFID Door Strike

This door controller was built for a vacation property that may someday be rented out.  The goal was to be able to cut our own keys, hand them to the renters, and wipe the keys after their stay.  Much more convenient and secure than trying to work with conventional keys and locks.



Assembly Instructions:

  • Connect an AC power cable to the input of the transformer
  • Connect the transformer output to the AC input terminals on the power supply
  • Connect pin 6 of the Arduino to the "Control +" terminal of the power supply
  • Connect VSS of the Arduino to the "Control -" terminal of the power supply
  • Connect 5V from the Arduino to the VDD terminal of the RFID reader
  • Connect VSS of the Arduino to the VSS terminal of the RFID reader
  • Connect the RFID RX terminal to the Arduino pin 3
  • Connect the RFID TX terminal to the Arduino pin 10
  • Connect the power supply 12V DC output to the Vin pin on the Arduino

Programming Instructions:

  • Use the Arduino serial monitor to read the serial numbers of the keys you wish to use for adding/disabling new keys
  • Change the "MASTER_ADD[4]" and "MASTER_DEL[4]" to match the serial numbers of the keys you wish to use for your master keys

Adding Keys:

  • Swipe your MASTER_ADD key to place the arduino into ADD mode
  • Swipe the new key you wish to add

Deleting All Keys:

  • Swipe the MASTER_DEL key


  • Keys are stored in the EEPROM, so they persist after a power outage
  • The power supply has a built-in opto-isolator on the CONTROL +/- terminals, which is why the arduino can safely control the 12v power supply using a 5v output
  • I should have used This power supply ($21) with a built in transformer as a cheaper solution to the power supply + transformer listed above
  • I'm switching over to this Wiegand 26/34 RFID Reader ($16), which is waterproof and suitable for mounting outdoors.  However the Wiegand data format is quite different, so the code will need to be changed to support it.  I'll provide more details once my card reader arrives.
  • You can connect a rechargeable 12V battery to the BATT +/- terminals of the power supply if you want it to work when the power is out
  • The duration the door is unlocked can be adjusted with the small potentiometer on the power supply
  • I'll probably enhance my code to separate "guest" keys, which are erased with the MASTER_DEL key, from "owner" keys which remain permanently in the system.

Future Extensions:

  • Once the Spark Core is released, I'll network these door locks, using MQTT as the communication protocal, and a Raspberry PI running MySQL as the centralized key repository
  • I could also add an internet portal, which would allow me to remotely add/disable keys

Sunday, February 24, 2013

Sous-Vide Duck Breast

After seeing this video, I knew this had to be my next sous-vide attempt.

After searching 6 supermarkets, and 3 specialty meat stores, I finally found duck breast.
Pat dry with a paper towel, slice grooves in the skin (careful not to cut the meat).
Add salt, pepper and thyme.
Vacuum seal.
Into the sous-vide at 134F for at least 2 hours.
Take out of the bag, place into a hot pan over medium heat.
Fry for 5 minutes, skin side down, then flip over and cook skin side up for 1 minute.
Cut, plate and eat!
Ok, so my presentation isn't as pretty as the video...but it still tasted good nonetheless.

Tuesday, February 19, 2013

Sous-Vide Steak

In my first post, I described how I built my sous-vide cooker.
 This time I have a photo-journal of cooking my first sous-vide steak.

 I started with an angus strip, and patted it dry with a paper towel.

Season with salt and pepper.  Place in a vacuum seal bag

Place in vacuum sealer

And vacuum seal

I'm using a napkin holder to keep the steak from floating in the crock pot

Place in the crock pot, and wait

1.5 hours later take it out.  Doesn't look very appetizing

Take it out of the bag,  not much improvement

Sear for 1 minute each side on a very hot cast iron skillet.   Now it looks like a steak

Cut it open.  Bad cut angle, but you get an idea of how evenly the steak is cooked.

Here's a better cut.  Click the picture to get the full resolution to see just how juicy the steak was.

Conclusion:  The results were somewhat mixed.  
The recipe guide I had listed 
  • 120F as rare
  • 134F as medium-rare
  • 140F as medium

My sous-vide was tracking a bit high and ended up cooking at 135F.   The result was probably juiciest and most tender steak I've ever eaten, however the flavor was what I associate with "medium".  Tasted more like roast beef, than a grilled steak.

I think the next time, I will lower the temperature to 130F, add more salt/pepper seasoning, and sear a little more (or use a kitchen torch) to get more of a crispy seared crust.

Internet-of-Things-Sous-Vide Cooker Project

My latest embedded project is an "Internet-of-Things Sous-Vide Cooker".  If you love fine cuisine and embedded micro-controller projects, then read on. However, this is my first blog, so lower your expectations...

Articles on the book "Modernist Cuisine" first introduced me to the sous-vide cooking technique.  In a nutshell, it's a high-end crock pot that cooks your food to perfection, every time, with almost no effort.  The downsides being that the machine costs $500 and it takes hours to cook even the fastest items.  But I'm not going to go into the reasons why to cook sous-vide, just click the link or Google it for more details.

Instead, I'm here to describe how I built my own sous-vide machine, and how I completely over-designed it to do all the things that don't really need to be done, but are very cool and serve as a template for turning any dumb appliance into a smart, internet connected device.

Project Summary:

The Arduino hardware:

Here's a PCB that I drew up.  Probably not much point in having it fab'd unless other people want copies.

The Arduino code:

This code measures the temperature of the water using the DS18B20 temperature setting, and feeds that into the Proportional-Integral-Derivative (PID) controller, to calculate the optimal duty cycle for the pulse-width-modulated (PWM) signal being sent to the relay.  The PWM cycle is 5 seconds, so a 20% duty cycle would turn the relay on for 1 second and off for 4 seconds, every PWM cycle.
  • The LiquidCrystal.h library is to control the LCD, which allows me to see the temperature setpoint (S) the actual temperature (A) and the PWM output (O) measured in milliseconds.  In the picture, 5000 milliseconds means that the relay is on 100% of the 5 second cycle, or full-on.  I set the temperature using a 10k potentiometer wired into one of the analog pins.
  • The Ethernet.h library is used to communicate with the Ethernet shield, and is very convenient that v1.0 now supports DHCP, so obtaining an IP address is now just plug and play.
  • The OneWire.h and DallasTemperature.h libraries are to interface with the DS18B20 one-wire temperature sensor.
  • The PubSubClient.h library is used to implement the Multiple-Queue-Telemetry-Transport (MQTT) protocol, which is a lightweight protocol for machine-to-machine (M2M) communication.  MQTT is a publish/subscribe protocol, which is divided into "topics".  Any client can subscribe or publish to any topic.  Whenever a client publishes to a topic, all clients that are subscribed to that topic receive that message.
I use the following MQTT topics:
  • cmd - for sending commands to the arduino
  • stdout - for status replies from the arduino
  • data - for real-time temperature log data
#include <PID_v1.h>
#include <SPI.h>
#include <Ethernet.h>
#include <PubSubClient.h>
#include <OneWire.h>
#include <DallasTemperature.h>
#include <LiquidCrystal.h>

double logTime = 0;
int logInterval = 30000; // millis interval between mqtt reports
int pwmWindowSize = 5000; // PWM window for relay
int lcdInterval = 1000; // interval for LCD updates
unsigned long pwmWindowStartTime;
double output; // PID output for relay control
//double kp=675, ki=0.1, kd=0; // PI tuning
//double kp=943, ki=1.13, kd=1909; // PID tuning
double kp=600, ki=0.06, kd=2.4; // PID tuning
double setpoint; // desired temp
double temperature; // actual temp
unsigned long logTimer; // timer for log updates
unsigned long lcdTimer; // timer for LCD updates
String string; // temporary var for data type conversion
char buffer[256]; // temporary var for data type conversion

// Relay on pin 17
#define relayPin 7

// 10kohm pot
const int analogInPin = A0;
int potValue = 0;

// LCD pins (RS,E,D4,D5,D6,D7)
LiquidCrystal lcd(A1,A2,6,5,3,2);

// Setup a oneWire instance to communicate with any OneWire devices 
#define ONE_WIRE_BUS 9
OneWire oneWire(ONE_WIRE_BUS);
// Pass our oneWire reference to Dallas Temperature. 
DallasTemperature sensors(&oneWire);

// Setup PID
PID myPID(&temperature, &output, &setpoint, kp, ki, kd, DIRECT);

// Update these with values suitable for your network.
byte mac[]    = { 0x90, 0xA2, 0xDA, 0x0D, 0x7D, 0x3A };
byte server[] = { 192, 168, 1, 101 };

// start ethernet client
EthernetClient ethClient;
// start mqtt client
PubSubClient mqttClient(server, 1883, callback, ethClient);

#define lcdBacklight A3
boolean networked = true;

void setup()
  Serial.begin(9600); // disable Serial, to regain use of pins 0,1
  lcd.print("Requesting IP");
  lcd.print("from DHCP");
  if (networked) Ethernet.begin(mac);
  if (networked) mqttConnect();
  logTimer = millis();
  pwmWindowStartTime = millis();
  myPID.SetOutputLimits(0, pwmWindowSize);
  pinMode(relayPin,OUTPUT); // disable Serial if using pin 0,1
  mqttClient.publish("stdout","restart"); // restart the log file

void loop()
  // read temperature sensor
  sensors.requestTemperatures(); // read DS18B20
  temperature = sensors.getTempFByIndex(0);
  Serial.print("temp: ");
  // read pot
  potValue = analogRead(analogInPin);
  setpoint = map(potValue, 0 , 1023, 130, 185); // 130F min safe food temp
  Serial.print(" set: ");

  if (networked) logUpdate(); // publish temperature data to mqtt channel "data"
  lcdUpdate(); // update lcd
  relayUpdate(); // implement pwm on relay
  myPID.Compute(); // PID compute loop

  if (networked && mqttClient.loop() == false) mqttConnect();  // reconnect if I've lost connection to mqtt

void callback(char* topic, byte* payload, unsigned int length) {
  // handle message arrived
  String content="";
  char character;
  for (int num=0;num<length;num++) {
    character = payload[num];
  if (content=="getKp") dtostrf(myPID.GetKp(),2,2,buffer);
  else if (content=="getKi") dtostrf(myPID.GetKi(),2,2,buffer);
  else if (content=="getKd") dtostrf(myPID.GetKd(),2,2,buffer);
  else if (content=="stop") { myPID.SetMode(MANUAL); strncpy(buffer,"PID off",sizeof(buffer)); }
  else if (content=="start") { myPID.SetMode(AUTOMATIC); strncpy(buffer,"PID on",sizeof(buffer)); }
  else if (content.startsWith("setKp=")) {
  else if (content.startsWith("setKi=")) {
  else if (content.startsWith("setKd=")) {

// this is what to do whenever I'm disconnected from MQTT
void mqttConnect() {
  if (mqttClient.connect("arduinoClient")) {
    Serial.println("[INFO] connected to mqtt");

void logUpdate() {
  if(millis()>logTimer) {
    size_t len=strlen(buffer);
    buffer[len++] = ',';
    len = strlen(buffer);
    buffer[len++] = ',';
    len = strlen(buffer);
    buffer[len++] = ',';

void lcdUpdate() {
  if(millis()>lcdTimer) {
    lcd.print(" A:");

void relayUpdate() {
  if(millis() - pwmWindowStartTime>pwmWindowSize)
  { //time to shift the Relay Window
    pwmWindowStartTime += pwmWindowSize;
  if(myPID.GetMode()==MANUAL) output = 0;
  if(output > millis() - pwmWindowStartTime) {
  } else {

The Node.js script on the Raspberry Pi:

The node.js script is a very simple script that subscribes to the "data" and "stdout" topics on the MQTT broker (also running on the raspberry pi).  All packets on the "data" topic get logged to a file.  If the message "restart" is seen on the "stdout" topic, then the existing log is deleted to restart data logging.
var sys = require('sys');
var mqtt = require('mqttjs');
var fs = require('fs');

process.on('uncaughtException', function (error) {

mqtt.createClient(1883, '', function(err,client) {
    if (err) {

    client.connect({keepalive: 3000});

    client.on('connack', function(packet) {
        client.subscribe({topic: 'data', qos: 0})
        client.subscribe({topic: 'stdout', qos: 0})

    client.on('publish', function(packet) {
        sys.puts(packet.topic+" : "+packet.payload);
        if (packet.topic == 'data') {
            fs.appendFile('/logs/log.csv', packet.payload+"\n", function(err) {
                if (err) throw err;
        if (packet.topic == 'stdout' && packet.payload == 'restart') {
            sys.puts('deleting /logs/log.csv');
            fs.unlink('/logs/log.csv', function(err) {
                if (err) throw err;

The Google Charts web page on the Raspberry Pi:

This html utilizes the Google charts API to render in your browser, a plot of the data being sent by the arduino and logged by the node.js script.

    <meta charset="UTF-8">
    <meta http-equiv="refresh" content="30">
    <!--Load the AJAX API-->
    <script src=""></script>
    <script src=""></script>

        // Load the Visualization API and the piechart package.
        google.load('visualization', '1.0', {'packages':['controls']});

        // Set a callback to run when the Google Visualization API is loaded.

        /* getData calls an external url to populate and return a json object
        suitable for google charts as a dataTable.  Validate the incoming json
        text with a site like

        The first column should be the Time.
        The second column should be the total number of bugs
        Subsequent columns can be whatever you want (optional)

        The format should look like:
                "cols": [
                "rows": [
        function getData(myUrl) {
                var jsonString = $.ajax({url: myUrl,dataType:'json',async: false}).responseText;
                // convert string to json
                var obj = jQuery.parseJSON(jsonString);
                return obj;

        function drawChart() {
                // get dataTable
            var jsonData = getData("csv2json.php");
                var data = new google.visualization.DataTable(jsonData);

                // Create a dashboard.
                var dashboard = new google.visualization.Dashboard(

                var today = new Date();

                var control = new google.visualization.ControlWrapper({
                        'controlType': 'ChartRangeFilter',
                        'containerId': 'control_div',
                        'options': {
                                // Filter by the date axis.
                                'filterColumnIndex': 0,
                                'ui': {
                                        'chartType': 'LineChart',
                                        'chartOptions': {
                                                'chartArea': {'width': '80%'},
                                                'hAxis': {'baselineColor': 'none'}
                                        // Display a single series that shows the total bug count
                                        // Thus, this view has two columns: the date (axis) and the count (line series).
                                        'chartView': {'columns': [0, 1]},
                                        // 1 day in milliseconds = 24 * 60 * 60 * 1000 = 86,400,000
                                                     // 1 minute in milliseconds = 1000 * 60
                                        'minRangeSize': 60000
                        // Initial range: half hour ago
                        //'state': {'range': {'start': new Date(today.getTime()-1000*60*60*24), 'end': today}}

                var chart = new google.visualization.ChartWrapper({
                        'chartType': 'ComboChart',
                        'containerId': 'chart_div',
                        'options': {
                                // Use the same chart area width as the control for axis alignment.
                                'chartArea': {'height': '80%', 'width': '80%'},
                                //'hAxis': {'slantedText': false,'maxAlternation':1},
                                //'vAxis': {'viewWindow': {'min': 0}},
                                'legend': {'position': 'in'},
                                'title':'Chuck\'s Sous-Vide Temperature Log',
                                'hAxis':{'title': 'Minutes'},
                                'vAxis':{'title': 'Fahrenheit/PWM duty cycle'}

                dashboard.bind(control, chart);
        <div id="dashboard_div">
                <div id="chart_div" style="width: 800px; height: 500px;"></div>
                <div id="control_div" style="width: 800px; height: 50px;"></div>

The csv2json php script:

The log file is in a csv format, and the html uses ajax/jQuery to call the csv2json.php script to convert the csv logfile into a json javascript object notation.
"cols": [
"rows": [
$rows = array();
$handle = @fopen("/logs/log.csv","r");
if ($handle) {
   while (($buffer = fgets($handle)) !== false) {
      $buffer = chop($buffer);
      $csv = preg_split("/,/",$buffer);
      $line = '    {"c":[{"v":' . $csv[0] . '},{"v":' . $csv[1] . '},{"v":' . $csv[2] . '},{"v":' . $csv[3] . '}]}';
   echo implode (",\n",$rows) . "\n ]\n}";