First steps with Node-RED

In my ongoing quest to find a good home automation software framework to run the hardware I built, I discovered Node-Red.  Node-Red is the self-professed "visual tool for wiring the Internet of Things".  Sounds promising.

Node-Red is a graphical programming environment based on node.js.  (Node.js is basically server-side javascript). Since it's built on node.js, Node-Red can run anywhere node.js runs.  It's all open source, and was started by IBM.

The interface to Node-Red runs completely in a web-browser, so there's no need to install anything on your development machine, and it is even usable from a mobile browser. It works by connecting wires between different functional nodes, kind of like the picture above.  There are lots of different nodes that do various things, from interfacing to a WeMo light switch to accessing emails from a Gmail account.

There is no built-in UI in Node-Red, but I think that might actually be an advantage.  It has numerous methods of connecting to a custom UI, and this way, you don't restrict users to a default UI that might not suit their application.

I got interested in Node-Red because I saw several different people using it to do home automation with JeeNodes.

Installation
I mostly followed the instructions here to install Node-Red to an old laptop running Xubuntu.

First I installed the Node.js package manager and Node.js itself:
sudo apt-get install npm
sudo apt-get install node

Then I downloaded the latest release zip file from nodered.org and unzipped to ~/dev
cded into that dir and ran 'npm install --production'.

Next up I launched Node-Red it by doing 'nodejs red.js'.  Running 'node red.js' doesn't work since the debian packages install it as nodejs to avoid a conflict. Eventually I ended up symlinking node to nodejs to make things easier:
ln -s /usr/bin/nodejs /usr/bin/node

And with that Node-Red seems to be up and running.

Node-Red ships with a pretty good set of basic nodes, but there is also a git repo that has even more nodes, so I thought I'd try that out.  I installed the extra nodes by doing 'cd nodes; git clone https://github.com/node-red/node-red-nodes.git'.

It should be noted that many of these additional nodes rely on node.js libraries and until those libraries are installed, the new node will not appear in the Node-Red palette.  As an example, if I want to use the suncalc node to generate an event at sunrise every day, I would need to install the suncalc library by doing 'npm install suncalc'.

Hello World
I ran through the initial tutorial and was successful.  Programs in Node-Red are called "flows". The basic design pattern in Node-Red is: drop and config nodes, wire them together, hit deploy.  Pretty simple, and even better, it actually works.

The debugging is a little simplistic in that you can print values to a debug log.  But again, it's simple, and it works, so I can't complain too much.

Flows can be exported as a json string, and imported by the same mechanism.  For the rest of this post I'll post the exported json for the flows I've used.

Controlling  execution
The Node-Red documentation has a good "hello world" tutorial, and has a tutorial on creating new plugin nodes, but it's a little sparse beyond that.  I did some experiments to figure out how the nodes in Node-Red actually execute.  Here's what I've figured out.

First off, Node-Red is completely event based.  Nothing executes unless an external event triggered it.  The "delay" and "trigger" nodes also allow timing based events to be generated.

The wires between the nodes are there to symbolize a data connection to the previous node.  More specifically the wires are really a graphical representation of how javascript Objects get passed to and from nodes.  These are usually referred to as msg objects and their contents can be somewhat arbitrary but most often contain a data member called payload.

So how can you control whether a node executes, like you can with an "if" statement in most text based languages?  If you return a "null" for any of the outputs then any of the subsequent nodes attached to that output don't execute.  A way to demonstrate that is by creating a function node that has multiple outputs which means you return an array of msg objects.  Any element in the array that has a value of null will prevent the connected node from executing.  By the way, this is exactly how a "switch" node works.  Check out the "demo switch" node in the example flow below.

Here's the example flow I'll be using for the rest of the post:

[{"id":"f64d219e.53a208","type":"inject","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"x":439,"y":454,"z":"d5a6cdcf.6203b","wires":[["5863bf74.888a3"]]},{"id":"5863bf74.888a3","type":"function","name":"demo switch","func":"msg.payload = \"hello\";\n//return [null, msg];\nreturn [msg, msg];","outputs":"2","x":628,"y":452,"z":"d5a6cdcf.6203b","wires":[["6d0fe6ba.2d2e88"],["d6a0f7c1.e949e8"]]},{"id":"6d0fe6ba.2d2e88","type":"debug","name":"demo out 1","active":true,"console":"false","complete":"false","x":821,"y":407,"z":"d5a6cdcf.6203b","wires":[]},{"id":"d6a0f7c1.e949e8","type":"debug","name":"demo out 2","active":true,"console":"false","complete":"false","x":820,"y":486,"z":"d5a6cdcf.6203b","wires":[]},{"id":"1ec662d2.3b7c25","type":"http in","name":"","url":"/test","method":"get","x":450,"y":277,"z":"d5a6cdcf.6203b","wires":[["dd8cd3d3.17b08"]]},{"id":"9c30b932.4da1e","type":"debug","name":"test URL","active":true,"console":"false","complete":"false","x":850,"y":189,"z":"d5a6cdcf.6203b","wires":[]},{"id":"6b3281a7.0a29c8","type":"http response","name":"","x":841,"y":276,"z":"d5a6cdcf.6203b","wires":[]},{"id":"dd8cd3d3.17b08","type":"function","name":"Gen response","func":"var resp = \"Hello \" + msg.payload.name;\nmsg.payload = resp;\nreturn msg;","outputs":1,"x":665,"y":276,"z":"d5a6cdcf.6203b","wires":[["6b3281a7.0a29c8","9c30b932.4da1e"]]},{"id":"7a4aeee7.85b51","type":"inject","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"x":424.5182189941406,"y":573.5229034423828,"z":"d5a6cdcf.6203b","wires":[["fbd25db9.042da"]]},{"id":"fbd25db9.042da","type":"function","name":"count","func":"if (context.hasOwnProperty(\"counter\"))\n\tcontext.counter += 1;\nelse\n\tcontext.counter = 0;\n\ncontext.global.counter = context.counter + 1;\n\nmsg.payload = \"local counter = \" + parseInt(context.counter, 10);\n\nreturn msg;","outputs":1,"x":602.5182647705078,"y":572.522876739502,"z":"d5a6cdcf.6203b","wires":[["caaa07a5.3555f8"]]},{"id":"caaa07a5.3555f8","type":"debug","name":"","active":true,"console":false,"complete":false,"x":782.5182189941406,"y":577.5229034423828,"z":"d5a6cdcf.6203b","wires":[]},{"id":"36a0278c.c95fd8","type":"inject","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"x":424.5182189941406,"y":638.5228900909424,"z":"d5a6cdcf.6203b","wires":[["4b22fcbc.b4dd04"]]},{"id":"4b22fcbc.b4dd04","type":"function","name":"get global count","func":"msg.payload = \"global counter = \" + parseInt(context.global.counter, 10);\n\nreturn msg;","outputs":1,"x":603.5182189941406,"y":634.5229034423828,"z":"d5a6cdcf.6203b","wires":[["f51d7f6a.0ae28"]]},{"id":"f51d7f6a.0ae28","type":"debug","name":"","active":true,"console":false,"complete":false,"x":782.5182762145996,"y":633.5228900909424,"z":"d5a6cdcf.6203b","wires":[]}]

Local and Global variables
Within a function node there is a way to persist data between calls to that node that is both local to that node and global to all nodes.  It's documented pretty well here.  I created a quick demonstration of globals using a couple of function nodes.

First web service
Just to try out what I'd learned I used some techniques from here, and got a working test URL using http in and http response nodes!  Just pass a "name" parameter to the URL and it will print a Hello message.  Something like this: "http://site/test?name=Ken"

Conclusions
I really liked working with Node-Red, and I think I'll probably continue using it.

Pros

• Open source and actively developed by IBM
• Web-based development
• Graphical programming/rule development
• https and basic http authentication is supported.  See here for more info.
• Mobile-friendly interface

Cons

• No built-in UI
• No built-in database for sensor readings, though there are nodes to interface with various DB formats

Trying out HouseAgent

For the last couple of years I've been looking for some Home Automation software to run the hardware that I'm slowly building to automate my house. There are so many choices that I'm not sure where to start.  I discovered HouseAgent mostly because it's written in Python, and I really like Python, so I thought I'd give it a test drive.

HouseAgent is a python-based home automation tool, with a promising architecture.  It appears that plugins are separate processes that communicate via AMQP, which means they can be written in (almost) any programming language.  It has a web interface, and a simple rules engine.

I installed the latest daily build of House Agent on a Windows 7 machine and ran into some problems.  The main problem was that it wouldn't launch.

After poking around for a while I figured out that you could run the House Agent daemon at the command line to get more info.  Just run 'houseagent.exe debug' to display debug messages.  I quickly discovered that there were files missing, namely houseagent.conf.  I grabbed these files from the git repo and put them in the places the error messages referred to.  The next error I had was that the port it was trying to use was already in use (8080) so I changed that in houseagent.conf.

At this point House Agent successfully launched, and I could get to the main web interface.  Then I went to the git repo for the JeeLabs plugin and grabbed all of the files there.  There wasn't any documentation so I made a directory called JeeLabs in the plugins folder in the House Agent install directory and copied all the files from the git repo there.  I then restarted the HouseAgent daemon and was able to successfully add a JeeLabs device.

I stopped at this point since I'll need to actually tie in some JeeNode hardware to continue my investigation.  There is a .conf file in the JeeLabs plugin directory that can be used to set the COM port to use and various other things.

There is also a fork of the JeeLabs which appears to be a little more up-to-date here. Even with those changes, when compared to another more up-to-date plugin, like ZWave, it looks like the JeeLabs plugin has been somewhat abandoned.

The web UI is also pretty plain and if I decide to use HouseAgent I'd like to modify it to look a little more snazzy. Right now it looks like an "admin" interface, rather than one that a user would really appreciate.  There does seem to be a branch where someone is working on mobile interface here.

At this point I kind of gave up since it seemed like getting HouseAgent to run was just fixing one bug after another.  HouseAgent seems like a great architecture but would probably take too much time and effort to make usable.

Pros:

• Written in python (yeah!)
• Good plugin architecture
• Web-based UI

Cons: 

• It doesn't look like HouseAgent has been under active developement in several years.  Has the project been abandoned?
• In general, it appears to be a little rough around the edges
• Web UI is a little ugly and not mobile friendly

embeddedWorld 2014

I was lucky enough to get chosen to speak at embeddedWorld 2014 in Nuremburg, Germany.  I gave a short presentation on using Yocto to develop a commercial product.  If you're not sure what Yocto is, don't worry, most people don't; it's a tool that helps Linux run on embedded hardware (like your WiFi router, for instance).  Rather than talk about a bunch of technical stuff I'll just stick mostly to just how the trip itself, impressions of Germany, etc.

General Observations
As I was flying from Dusseldorf to Nuremberg, I could see at least 4 nuclear reactors, quite a few windmills, and lots of solar panel installations (many on the roofs of houses and businesses and some empty fields with lots of solar panels in rows).  I also noticed lots of large-scale greenhouses in the farm fields, I couldn't tell if these were permanent installations or just temporary structures built over top of farm fields.  At least from the air, Germany looks very "green".

Once I got to Germany, the language barrier was not a problem at all.  I'd been told that there were alot of English speakers in Germany, but it really seemed that everyone spoke English.  It was funny too because most people would say "my English is not very good" and then they would go on to speak nearly flawless English.

Germany just felt very comfortable to me for some reason that's hard for me to articulate.  And I guess the Germans thought so too, because they almost always would initially speak German to me, until I asked "Sprechen Sie Englisch?".  The other people that came over with me from the States would usually get English right off the bat from the Germans they met.

We had several people also attending the conference from NI's Munich Office, and they were outstanding hosts, taking us to dinner each night and generally helping us out in any way.

I also found that business etiquette was a bit different:

1. It's expected to wear a suit and tie, the US standard of a button up shirt and khakis doesn't cut it.
2. Don't be late.  You'll get some very disapproving stares if you aren't punctual.

And this should surprise no one: Germans know beer.  Every beer I had there was excellent (and I had quite a few).  Don't worry about getting something you won't like; just drink it and it will be good.

I didn't have any time for sight-seeing so this is a crappy pic I took in downtown Nuremburg

Embedded World Exhibition Floor
The Exhibition part of Embedded World was crazy-huge, with 856 exhibitors and 26,000+ visitors.   It took me a long time just to do a relatively quick walkthrough of the floor.

I kept my eyes open for any LabVIEW front panels and I saw quite a few.  One of particular interest was a cool looking demo from Intel.  It's a slightly creepy looking insectile robot that plays the Chinese mandolin (not sure what the real name of the instrument is).  You can see the LabVIEW front panel on the monitor in the picture below.  The Intel guy running the demo couldn't say enough good things about developing in LabVIEW.

Presentation
My presentation went really well.  Almost every seat was filled; probably around 50 people.  I got some good questions and some compliments after it was over.  I also talked for a while with a couple of Intel guys that are heavily involved in the Yocto project.

Flight Back
The flight back was the only negative part of the whole trip. Coming in through Customs Stateside is no fun.  'Nuf said.

First open source patch accepted

I was about to post about my first patch accepted to an upstream open source project, but then I started digging around and discovered this one was actually my first!

I'm not trying to toot my own horn (ok, I am a little), I was just proud that I actually have some small, insignificant bits of code contributed to the open source community.

If anyone is remotely interested the first patch is to the Busybox project to fix a bug whereby a linked local network interface will keep getting a different IP between reboots when it should be getting the same IP on subsequent boots.  I can see your eyes glazing over...

The second patch is to the Yocto project and has to do with the generated entropy on an embedded Linux system.  Basically, if there's not enough entropy in the system then random numbers become much less random which can be a security risk if those random numbers are used for something related to cryptography.

And I guess to add to that I have lots of commits submitted to repos here and here, but those are repos owned by the company I work for so I wasn't counting those.