A little bit of Spring and MQTT

thoughts on using Spring and MQTT from one of my favorite people. This is the best way to MQTT enable your Java app servers, though obviously you can also use JMS if your MQTT broker (for example MessageSight) supports that too.  But if your java app servers are in one location and your MQTT broker is in another, i.e. cloud then this is the way.

The lost outpost

I’ve been involved with Spring (the Java framework, not the season…) for a couple of years now – since I joined VMware in 2012 through the former SpringSource organisation in the UK – and I’ve remained “involved” with it through my transition to Pivotal and the framework’s evolution into Spring 4 / Spring.IO under Pivotal’s stewardship.

To be clear, although I’ve been a “Java guy” / hacker through my time at IBM, I have never been a hardcore JEE coder, and my knowledge of Spring itself has always been limited. My good buddy Josh (MISTER Spring, no less) has done his best to encourage me, I briefly played with Spring Shell last year with @pidster, and the brilliant work done by the Spring Boot guys has been helpful in making me look at Spring again (I spoke about Boot – very briefly, as a newcomer to it –…

View original post 435 more words


Publishing Tizen IVI CANBus Data to the Cloud

Some practical tips to have Tizen IVI publish CAN bus to cloud via MQTT

Bridging Things and Services

I was asked to look into how to easily publish CANBus data using MQTT to the cloud in the context of Tizen IVI. It turned out to be quite interesting experience to understand how different layers (AMB, dBus, WRT IVI plugin) in Tizen are working together. The goal is a quick prototype to make everything work end to end.

View original post 408 more words

thoughts about using IoT MQTT for V2V and Connected Car from CES 2014

Had a great time at CES 2014 showing my IoT tech with Continental AG in the Renaissance Hotel, Jaguar Land Rover + Intel at GENIVI Show Trump Tower, QNX + Qualcomm in North Hall LVCC, and IBM in the Venetian.  Really enjoyed CES, GENIVI Show, Tao, and especially Firefly on Paradise with my guys.  Thanks!

IMG_8029 IMG_8080 IMG_8025 IMG_8075

Spent 4 days with dozens of automakers and car system providers, fielding questions and showing live demos.  Based on those discussions I’ve pulled together information I hope is useful.  Or if not useful, at least interesting:

  • thoughts https://mobilebit.wordpress.com/?s=mqtt
  • rants https://twitter.com/search?q=%40mobilebit%20mqtt&src=typd&f=realtime
  • slides http://www.slideshare.net/JoeSpeed/ibm-connected-car-is-a-big-data-problem-for-autotech-council-dec-13-2013-joe-speed
  • monologue http://youtu.be/u69I-GLYd6I
  • MQTT vs HTTPS on mobiles – 93x faster, 11x less power to send, 170x less power to receive, 8x less network overhead.  QoS and LWT for reliable comms on unreliable networks.
  • IBM MessageSight – secure IoT appliance and core component of global IBM IoT Cloud on Softlayer.   Concurrently connects 21M vehicles & mobiles at up to 340.2M telematics messages/second per rack.  Low latency, 60μs device-to-device MQTT messaging on fast network (i.e. 10GigE) under load.
  • video of AT&T iPhone & Verizon iPad remote control of GENIVI Tizen HVAC, note the low latency  Scottsdale > Dallas cloud > car in MA changes temp > Dallas > Scottsdale https://twitter.com/MobileBit/status/423181215672188928 
  • demos, how-to examples and source code http://m2m.demos.ibm.com/ 
  • demo video using QNX TI OMAP5 car http://youtu.be/C3ebGjJ0KjM?t=1m21s
  • another demo https://www.dropbox.com/s/lfjuih5v3928cg6/JLR_IBM_Intel_GENIVI_Tizen_demo.mov or even better is this one https://www.dropbox.com/s/a4rqm7nnw8tzxgk/JLR_Tizen_IBM_IoT_720p.mov
    note the data path for what you’re seeing in the 2 side-by-side HTML5 apps is Verizon iPad Scottsdale AZ > Dallas Softlayer cloud > GENIVI Tizen car unit in Dartmouth MA > Dallas Softlayer cloud > AT&T iPhone Scottsdale AZ
  • try this MQTT HTML5 whiteboard demo on several mobiles side-by-side, and observe the latency.  http://m2m.demos.ibm.com/whiteboard/  This is running against  Softlayer’s Dallas cloud in North America.  Let me know if you need the URLs for this in Asia, Europe or elsewhere.
  • MQTT + MQTT-SN simplifies V2V / V2X. MQTT for V2V via cloud 10s-100s ms, MQTT-SN for V2V via DSRC, ZigBee, 6LoWPAN, serial, UDP, et al.  And Eclipse Mosquitto RSMB 1.3 is 75KB MQTT + MQTT-SN broker for embedded that bridges V2V mesh to cloud.  I think it wouldn’t be an exaggeration to say that RSMB is battle hardened.
  • MQTT/MQTT-SN for transport independent V2V/V2X via dynamic mesh of DSRC/WAVE, Wi-Fi, LTE-A, cloud a’la ITA’s MQTT Sensor Fabric https://twitter.com/MobileBit/status/423877657118261248
  • Yes, the cloud is  too slow to tell me the car ahead hit their brakes.  For that you need 20ms latency.  But you know what else besides DSRC can do that?  Cameras, radars, lasers, many things.  And DSRC doesn’t help  with what’s around the next bend, what’s over the next hill.  It is very localized.  Which is an issue because there is a whole class of V2V problems that require macro-level awareness and real-time decisioning for which higher latency (100ms+) is acceptable.  Continental is public example of someone that tackling those using IoT + real-time Big Data geospatial decisioning in the cloud.
  • Yes, not every car is connected nor will they be in the near term, so what about the blind spots?  There are things that help mitigate that.  For example if a car senses other cars and publishes that data to the cloud, then the cloud knows about those cars even though they’re not connected.   IoT MQTT cloud connectivity is software, just requires is a programmable TCU.  Ford already made public that they’re pushing an OTA update to 3.4M existing cars adding more connected car capabilities, so many existing models can enabled for V2V cloud, not just future ones.  Automakers are launching aftermarket OBD2 dongles such as this one w 3G/GPS/accelerometers/etc. are $100 and less at retail ($20 in volume  for an OEM?) that cloud connect via MQTT any car built after 1995.
  • Yes, it is an issue there that each OEM is having their own cloud.  Solutions include having them use 1 cloud, for example what Continental AG is doing with having IBM give them the global connected car cloud which will be used by multiple OEMs that they supply.  Can also tackle it via federated car clouds, interconnecting them and exchanging data in real-time.  While not trivial, federated clouds and federated ESBs is fairly well traveled ground.  The technology is there.  MQTT even has a bridging convention whereby an MQTT broker (server) can also be a client.  The same approach you use to have an MQTT broker like RSMB in the car bridging a local V2V mesh to the cloud can also be used to bridge one cloud to another, effectively interconnecting them.  Yes that requires agreement re data format (preferably protobuf or JSON) and topic space, but those are solvable.
  • I’ve nothing against DSRC, in fact I’m very excited to have MQTT-SN running atop DSRC/WAVE, I think the pub/sub topics and QoS adds to DSRC’s usefulness.  That said, DSRC is early 90’s RF tech that is beginning to show its age.  And many of the use cases DSRC was intended for in V2V are becoming obviated by other sensor and wireless tech.  I don’t need DSRC to tell me when the car ahead hits their brakes, the car is already getting several other sensors that can do that.  And nobody has figured out where the $3 trillion dollars is going to come from to get DSRC radios into all the cars and infrastructure.  And we’re talking about adding that DSRC cost to cars that are already getting LTE Advanced and/or WiFi.  You know what else besides DSRC can do peer 2 peer?  LTE Advanced and WiFi.  Credit for this  to Roger Lanctot @rogermud who clued me into this.  He’s also the one that pointed out to me that latency is a driver distraction issue.
  • I already knew latency is an owner satisfaction and brand perception issue.  She’s not going to buy your car if her buying experience includes making her standing in the rain for 30 to 90 seconds while pondering her request to unlock the doors.
  • MQTT client source http://eclipse.org/paho/ for C, C++, Java, JavaScript, Python, Lua
  • MQTT + MQTT-SN bridging broker for embedded systems.  http://mqtt.org/2013/12/mqtt-for-sensor-networks-mqtt-sn 
  • MQTT-SN “MQTT for Sensor Networks” is designed for WSNs and mesh networks using datagrams instead of socket.  It works over DSRC, ZigBee, 6LoWPAN, LTE Advanced p2p, UDP, et al.  Like MQTT it is open standard, open source and royalty free.  Some think it can greatly improve V2V / V2X by making it publish/subscribe based and transport independent (like the military’s).  The meshing capabilities and ability to propogate topics and messages across the mesh w QoS is really quite amazing.   I’m trying to get permission to youtube the videos form the field trials.   http://mqtt.org/2013/12/mqtt-for-sensor-networks-mqtt-sn
  • MQTT Sensor Fabric secure WSN mesh used by US and UK military uses RSMB which is now open source per above.
  • MQTT 3.1 spec http://mqtt.org/documentation
  • MQTT 3.1.1 spec draft https://www.oasis-open.org/news/announcements/30-day-public-review-for-mqtt-version-3-1-1
  • MQTT-SN 1.2 spec http://mqtt.org/new/wp-content/uploads/2009/06/MQTT-SN_spec_v1.2.pdf 
  • Q:  Is MQTT security better or worse than HTTPS?  A:  Neither.  It is the same.  They are both socket protocols that rely on TLS/SSL connection level encryption.  For example to the cloud I often suggest TLS 1.2 w mutual authentication using “signed at the foundry” keys built into the chipset.

Connected Car sessions at IOD Big Data

Here’s a partial list of sessions and things to see:

Tuesday, Nov 5

10:00 AM – 11:00 AM

South Pacific I

LMP-3613A: Connected Car is Mobile, Social, Cloud, Big Data

Joe Speed, IBM m2m Leader


Tuesday, Nov 5

3:00 PM – 4:00 PM

Jasmine G

Session IBD-3990 – Real-time big data analytics – Bringing Data Integration and Application Integration together

IBM Speaker:  Michael Curry, Vice President, WebSphere Foundation Product Management

Tuesday Nov 5

12:45 PM – 1:45 pm

Bayside Dining IM BoF Tables 15 & 15

Session number:  IBD-3957A – “Connected Car and Big Data:  Driving a Responsive Experience“

Moderator:  Michael Riley

Tuesday Nov 5 12:30 PM – 2:30 PM

Tradewinds Room – Mandalay Bay North (lower level)

By invite

Conquer the Big Data Challenge of Sensor Data – Luncheon

Steve Shoaf, IBM, IM Worldwide Sales

Joe Speed, IBM m2m Leader

Oliver Goh, CEO Shaspa

Kevin Brown, IBM, Informix Chief Architect

Wed, Nov 6

11:15 AM – 12:15 PM

South Pacific B

LAU-3836A: Best practices in collection, change and version management of Infotainment data at Jaguar Land Rover

Matt Jones, Director Infotainment Systems JLR, VP GENIVI

Every day Connected car demo featuring MessageSight, InfoSphere Streams, and Worklight in a hands-on demo area of the expo

Joint ped #31 with Geospatial in the demo room

Wednesday, Nov 6

10:00 AM – 11:00 AM

South Pacific C

LAU-3618A: Big Data & Analytics Drives Quality Improvements at General Motors

Kevin Mixer, General Motors, Global Director Vehicle Solutions for Quality, Engineering and Research; Jim Bydalek, IBM, Partner – Business Analytics & Optimization

Tuesday, Nov 5

01:45 PM – 02:45 PM South Pacific B

LAU-3315A: Solving the mystery: How BMW drives business analytics to leverage customer value

Alexander Thamm, alexanderthamm GmbH, Managing Director; Stefan Meinzer, BMW AG, Head of Customer Satisfaction Analytics; Anuj Marfatia, IBM, Program Director

Tuesday, Nov 5

11:15 AM – 12:15 PM South Pacific B

LAU-3283A: Preventing Manufacturing Problems Before They Occur

Tom Marks, Daimler Trucks North America, Manager, Business Excellence;
Dan Barrett, IBM , Business Analytics Industrial Sector Solutions

Tue Nov 5 ( 10:00 a.m. – 11:00 a.m. )

Islander H

LCI-1783A: IBM MobileFirst: Put Your Business in Motion

Phil Buckellew, IBM


Tue Nov 5 ( 3:00 p.m. – 4:00 p.m. )

Mandalay Bay C

BBD-2540A: Using Big Data to Drive Innovation and Dealer Satisfaction at BMW

Stefan Meinzer, BMW Group

Matthias Oehm, IBM Deutschland

Johann Prenninger,BMW Group

Tue Nov 5 ( 4:30 p.m. – 5:45 p.m. )

South Pacific B

LAU-1732A: Social, Mobile, Analytics, Cloud, and Beyond for the Automotive Industry

Dan Ricci

IoT Cloud enable GENIVI, Tizen, AGL in 5 minutes via MQTT

While helping one of my customers, I asked Ian Craggs @icraggs to prototype an MQTT+D-Bus bridge to cloud enable GENIVI, Tizen and Automotive Grade Linux.  Ian surprised me with a command line solution that MQTT enables them  in 5 minutes.  I expect we’ll still develop a bridge component for GENIVI but this is a trivially easy solution folks can do now.

You can test it against this MQTT MessageSight instance ip= port=1883 in the cloud and use the MQTT Client web app on http://m2m.demos.ibm.com to test your GENIVI/Tizen/AGL publishing and subscribing.

Ian’s quick and easy solution as follows:

I have been experimenting with getting information from D-Bus to MQTT and back, and have come up with the following simple methods, using the Paho C MQTT client, available from http://www.eclipse.org/paho/  source repository http://git.eclipse.org/c/paho/org.eclipse.paho.mqtt.c.git/

These methods do no transformation or extraction of information at the moment, but it is easy to change them to get the exact information we need.  Any questions, please ask.

D-Bus to MQTT

There is a Linux program called dbus-monitor which will listen for events and print them to stdout.  Using the stdinpub sample of the Paho C client (src/samples directory), raw Linux desktop notification messages can be published to an MQTT server by using a command like this:

dbus-monitor –session interface=org.freedesktop.Notifications | stdinpub Notifications –delimiter -1 –verbose –maxdatalen 1000

You can test this using the command:

notify-send summary body

The raw output looks like this (you can see where the “summary” and “body” fields end up) :

signal sender=:1.78232 -> dest=:1.273 serial=122 path=/org/freedesktop/Notifications; interface=org.freedesktop.Notifications; member=NotificationClosed
uint32 13
uint32 1
method call sender=:1.81115 -> dest=org.freedesktop.Notifications serial=5 path=/org/freedesktop/Notifications; interface=org.freedesktop.Notifications; member=GetServerInformation
method call sender=:1.81115 -> dest=org.freedesktop.Notifications serial=6 path=/org/freedesktop/Notifications; interface=org.freedesktop.Notifications; member=Notify
string “notify-send”
uint32 0
string “”
string “summary”
string “body”
array [
array [
dict entry(
string “urgency”
variant             byte 1
dict entry(
string “category”
variant             string “”
int32 -1

We can easily extract just the relevant data from the events we are interested in.

MQTT to D-Bus

Using another program called notify-send and another sample stdoutsub (available from Paho again), we can use MQTT to create desktop events via D-Bus:

stdoutsub notify –delimiter newline | xargs -I %body –delimiter \\n notify-send summary %body

We can use dbus-send instead of notify-send to create events on any D-Bus interface.