Project showcase
SmartQ Notification with Azure IOT Hub and Virtual Shields

SmartQ Notification with Azure IOT Hub and Virtual Shields © GPL3+

Waiting hours in queue is history, SmarQ will update you the current queue status and alarm you when it is almost your turn to be served.

  • 1,585 views
  • 0 comments
  • 6 respects

Components and supplies

Necessary tools and machines

09507 01
Soldering iron (generic)

Apps and online services

About this project

Waiting hours in queue for your turn to be attended is a waste of time.  SmarQ allows you to take a queue number with a press of button. You can go away for a while to do other business in the middle of the waiting. SmarQ will update you the current queue status and notify you when it is almost your turn to be served.


Current implementation allows a queue attendant to speaks to the Virtual Shields for Arduino app in order to manage the queue number.  A notification will be sent to the customers whenever the queue number changes.  Hence the customers can estimate how soon they will be served.


This project currently consists of the following 3 parts:

  • Smart Queue Universal Windows app
  • Virtual Shields for Arduino app
  • ProcessDeviceToCloudMessage and ProcessD2CInteractiveMessages apps

The project includes the use of Azure IoT Hub, Queue and Azure Storage and Notification Hub.


SmartQueueUniversalWindows app


This app notifies a customer the current queue number being served so that the customer can manage the time needed to be presented at the customer counter.  

Please follow this link Get started with Notification Hubs for Windows Store Apps to build this app. I have added some codes at the code section below or get from GitHub.  I use the Visual Studio Community 2015.

Important steps:

  • Register your app for the Windows Store
  • Configure your notification hub
  • Connect your app to the notification hub


Virtual Shields for Arduino App


Queue attendant uses this app to manage the queue.  A attendant will speak to the app to get the next queue number to be served.  This app can be installed from Windows Store.


ProcessDeviceToCloudMessages and ProcessD2CInteractiveMessages Apps


Both of the above are console apps.  Follow the instructions in this tutorial (How to process IoT Hub device-to-cloud messages) to build these apps.  

Follow the section titled "Create the event processor" to build the ProcessDeviceToCloudMessages app. and the section title "Receive interactive messages" to build the ProcessD2CInteractiveMessages app.  I have added some codes at the code section below or get from GitHub.

The ProcessDeviceToCloudMessages app processes the IoT Hub events and stores messages to a queue and the ProcessD2CInteractiveMessages app takes the messages from the queue and sends them to Notification Hub.




Useful Links


Azure IoT Hub developer guide

IoT Hub REST


Code

MKR1000SmartQueueSpeechC/C++
MKR1000 sketch
#include <ArduinoJson.h>

#include <VirtualShield.h>
#include <Text.h>
#include <Speech.h>
#include <Recognition.h>

#include <SPI.h>
#include <WiFi101.h>


VirtualShield shield;	          // identify the shield
Text screen = Text(shield);	  // connect the screen
Speech speech = Speech(shield);	  // connect text to speech
Recognition recognition = Recognition(shield);	  // connect speech to text

int queueNum = 0;

int LED_PIN = 8;

///*** WiFi Network Config ***///
char ssid[] = "yourNetwork"; //  your network SSID (name)
char pass[] = "secretPassword";    // your network password (use for WPA, or use as key for WEP)

///*** Azure IoT Hub Config ***///
//see: http://mohanp.com/  for details on getting this right if you are not sure.

char hostname[] = "YourIoTHubName.azure-devices.net";    // host name address for your Azure IoT Hub
char feeduri[] = "/devices/SmartQueueMKR1000/messages/events?api-version=2016-02-03"; //feed URI
char authSAS[] = "YourSharedAccessKey";

///*** Azure IoT Hub Config ***///

int status = WL_IDLE_STATUS;

WiFiSSLClient client;

String message;

void recognitionEvent(ShieldEvent* event) 
{
  if (event->resultId > 0) {
  	digitalWrite(LED_PIN, recognition.recognizedIndex == 1 ? HIGH : LOW);
    screen.printAt(6, "Heard " + String(recognition.recognizedIndex == 1 ? "next" : "stop"));

    if (recognition.recognizedIndex == 1) {
      queueNum++;
      message =  "Currently serving queue #" + String(queueNum);
      screen.printAt(8, message);

      azureHttpRequest(message);
      //Serial.println();
      //Serial.println(message);
    }
    
  	recognition.listenFor("next,stop", false);	    // reset up the recognition after each event
  }
}

// when Bluetooth connects, or the 'Refresh' button is pressed
void refresh(ShieldEvent* event) 
{
  // String message = "Hello Virtual Shields. Say the word 'on' or 'off' to affect the LED";
  String message = "Hello. Say the word 'next' to serve the next queue number.";
        
	screen.clear();
	screen.print(message);
  speech.speak(message); 

	recognition.listenFor("next,stop", false);	// NON-blocking instruction to recognize speech
}

void setup()
{
	pinMode(LED_PIN, OUTPUT);
	pinMode(LED_PIN, LOW);

  //check for the presence of the shield:
  if (WiFi.status() == WL_NO_SHIELD) {
    // don't continue:
    while (true);
  }

  // attempt to connect to Wifi network:
  while (status != WL_CONNECTED) {
    status = WiFi.begin(ssid, pass);
    // wait 10 seconds for connection:
    delay(10000);
  }

  // set up a function to handle recognition events (turns auto-blocking off)
	recognition.setOnEvent(recognitionEvent);	
  shield.setOnRefresh(refresh);

  // begin() communication - you may specify a baud rate here, default is 115200
	shield.begin(9600);

  randomSeed(analogRead(0));
}

void loop()
{
  String response = "";
  char c;
  ///read response if WiFi Client is available
  while (client.available()) {
    c = client.read();
    response.concat(c);
  }
  
	shield.checkSensors();		    // handles Virtual Shield events.
}

// this method makes an HTTPS connection to the Azure IOT Hub Server:
void azureHttpRequest(String content) {

  // close any connection before send a new request.
  // This will free the socket on the WiFi shield
  client.stop();

  String messageId = String(random(300)) + String(millis());
  //Serial.println();
  //Serial.println(messageId);

  String contentType = "text/plain";
  String accept = "application/json";
  
  // if there's a successful connection:
  if (client.connect(hostname, 443)) {
    //make the GET request to the Azure IOT device feed uri
    client.print("POST ");  //Do a POST
    client.print(feeduri);  // On the feedURI
    client.println(" HTTP/1.1"); 
    client.print("Host: "); 
    client.println(hostname);  //with hostname header
    // client.print("Accept: ");  // On the feedURI
    // client.println(accept); 
    client.print("Authorization: ");
    client.println(authSAS);  //Authorization SAS token obtained from Azure IoT device explorer
    client.print("IoTHub-MessageId: "); 
    client.println(messageId);  
    client.println("Connection: close");
    client.print("IoTHub-app-messageType: "); 
    client.println("interactive");  

    client.print("Content-Type: ");
    client.println(contentType);
    client.print("Content-Length: ");
    client.println(content.length());
    client.println();
    client.println(content);

    client.println();

  }
  else {
    // if you couldn't make a connection:
    Serial.println();
    Serial.println("connection failed");
  }

}
MainPage.xamlXML
MainPage.xaml of SmartQueueUniversalWindows app
<Page
    x:Class="SmartQueueUniversalWindows.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:SmartQueueUniversalWindows"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d">

    <Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
        <Image Source="Assets/StoreLogo.png" VerticalAlignment="Bottom" Margin="0,0,0,10" Height="50" Width="50"/>
        <TextBlock x:Name="textBlock" HorizontalAlignment="Left" Margin="10,10,10,10" TextWrapping="Wrap" Text="Waiting hours in queue for your turn to be attended is a waste of time.  SmartQ allows you to take a queue number with a press of button.   You can go away for a while to do other business in the middle of the waiting.  SmartQ will update you the current queue status and alarm you when it is almost your turn to be served." VerticalAlignment="Top" FontSize="20"/>
    </Grid>
</Page>
App.xaml.csC#
App.xaml.cs of SmartQueueUniversalWindows app
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices.WindowsRuntime;
using Windows.ApplicationModel;
using Windows.ApplicationModel.Activation;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;

using Windows.Networking.PushNotifications;
using Microsoft.WindowsAzure.Messaging;
using Windows.UI.Popups;

namespace SmartQueueUniversalWindows
{
    /// <summary>
    /// Provides application-specific behavior to supplement the default Application class.
    /// </summary>
    sealed partial class App : Application
    {
        /// <summary>
        /// Initializes the singleton application object.  This is the first line of authored code
        /// executed, and as such is the logical equivalent of main() or WinMain().
        /// </summary>
        public App()
        {
            Microsoft.ApplicationInsights.WindowsAppInitializer.InitializeAsync(
                Microsoft.ApplicationInsights.WindowsCollectors.Metadata |
                Microsoft.ApplicationInsights.WindowsCollectors.Session);
            this.InitializeComponent();
            this.Suspending += OnSuspending;
        }

        /// <summary>
        /// Invoked when the application is launched normally by the end user.  Other entry points
        /// will be used such as when the application is launched to open a specific file.
        /// </summary>
        /// <param name="e">Details about the launch request and process.</param>
        protected override void OnLaunched(LaunchActivatedEventArgs e)
        {

#if DEBUG
            if (System.Diagnostics.Debugger.IsAttached)
            {
                this.DebugSettings.EnableFrameRateCounter = true;
            }
#endif

            Frame rootFrame = Window.Current.Content as Frame;

            // Do not repeat app initialization when the Window already has content,
            // just ensure that the window is active
            if (rootFrame == null)
            {
                // Create a Frame to act as the navigation context and navigate to the first page
                rootFrame = new Frame();

                rootFrame.NavigationFailed += OnNavigationFailed;

                if (e.PreviousExecutionState == ApplicationExecutionState.Terminated)
                {
                    //TODO: Load state from previously suspended application
                }

                // Place the frame in the current Window
                Window.Current.Content = rootFrame;
            }

            if (rootFrame.Content == null)
            {
                // When the navigation stack isn't restored navigate to the first page,
                // configuring the new page by passing required information as a navigation
                // parameter
                rootFrame.Navigate(typeof(MainPage), e.Arguments);
            }
            // Ensure the current window is active
            Window.Current.Activate();

            InitNotificationsAsync();
        }

        /// <summary>
        /// Invoked when Navigation to a certain page fails
        /// </summary>
        /// <param name="sender">The Frame which failed navigation</param>
        /// <param name="e">Details about the navigation failure</param>
        void OnNavigationFailed(object sender, NavigationFailedEventArgs e)
        {
            throw new Exception("Failed to load Page " + e.SourcePageType.FullName);
        }

        /// <summary>
        /// Invoked when application execution is being suspended.  Application state is saved
        /// without knowing whether the application will be terminated or resumed with the contents
        /// of memory still intact.
        /// </summary>
        /// <param name="sender">The source of the suspend request.</param>
        /// <param name="e">Details about the suspend request.</param>
        private void OnSuspending(object sender, SuspendingEventArgs e)
        {
            var deferral = e.SuspendingOperation.GetDeferral();
            //TODO: Save application state and stop any background activity
            deferral.Complete();
        }

        private async void InitNotificationsAsync()
        {
            var channel = await PushNotificationChannelManager.CreatePushNotificationChannelForApplicationAsync();

            var hub = new NotificationHub("<hub name>", "<connection string with listen access>");
            var result = await hub.RegisterNativeAsync(channel.Uri);

            // Displays the registration ID so you know it was successful
            if (result.RegistrationId != null)
            {
                // var dialog = new MessageDialog("Registration successful: " + result.RegistrationId);
                var dialog = new MessageDialog("Registration successful");
                dialog.Commands.Add(new UICommand("OK"));
                await dialog.ShowAsync();
            }

        }
    }
}
StoreEventProcessor.csC#
StoreEventProcessor.cs of ProcessDeviceToCloudMessages app
using System.IO;
using System.Diagnostics;
using System.Security.Cryptography;
using Microsoft.ServiceBus.Messaging;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System.Threading.Tasks;
using System;
using System.Text;
using System.Collections.Generic;
using System.Linq;

namespace ProcessDeviceToCloudMessages
{
    class StoreEventProcessor : IEventProcessor
    {
        private const int MAX_BLOCK_SIZE = 1024;   // 4 * 1024 * 1024;
        public static string StorageConnectionString;
        public static string ServiceBusConnectionString;

        private CloudBlobClient blobClient;
        private CloudBlobContainer blobContainer;
        private QueueClient queueClient;

        private long currentBlockInitOffset;
        private MemoryStream toAppend = new MemoryStream(MAX_BLOCK_SIZE);

        private Stopwatch stopwatch;
        private TimeSpan MAX_CHECKPOINT_TIME = TimeSpan.FromSeconds(3);

        public StoreEventProcessor()
        {
            var storageAccount = CloudStorageAccount.Parse(StorageConnectionString);
            blobClient = storageAccount.CreateCloudBlobClient();
            blobContainer = blobClient.GetContainerReference("d2ctutorial");
            blobContainer.CreateIfNotExists();
            queueClient = QueueClient.CreateFromConnectionString(ServiceBusConnectionString, "d2ctutorial");
        }

        Task IEventProcessor.CloseAsync(PartitionContext context, CloseReason reason)
        {
            Console.WriteLine("Processor Shutting Down. Partition '{0}', Reason: '{1}'.", context.Lease.PartitionId, reason);
            return Task.FromResult<object>(null);
        }

        Task IEventProcessor.OpenAsync(PartitionContext context)
        {
            Console.WriteLine("StoreEventProcessor initialized.  Partition: '{0}', Offset: '{1}'", context.Lease.PartitionId, context.Lease.Offset);

            if (!long.TryParse(context.Lease.Offset, out currentBlockInitOffset))
            {
                currentBlockInitOffset = 0;
            }
            stopwatch = new Stopwatch();
            stopwatch.Start();

            return Task.FromResult<object>(null);
        }

        async Task IEventProcessor.ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> messages)
        {
            foreach (EventData eventData in messages)
            {
                byte[] data = eventData.GetBytes();

                if (eventData.Properties.ContainsKey("messageType") && (string)eventData.Properties["messageType"] == "interactive")
                {
                    var messageId = (string)eventData.SystemProperties["message-id"];

                    var queueMessage = new BrokeredMessage(new MemoryStream(data));
                    queueMessage.MessageId = messageId;
                    queueMessage.Properties["messageType"] = "interactive";
                    await queueClient.SendAsync(queueMessage);

                    WriteHighlightedMessage(string.Format("Received interactive message: {0}", messageId));
                    continue;
                }

                if (toAppend.Length + data.Length > MAX_BLOCK_SIZE || stopwatch.Elapsed > MAX_CHECKPOINT_TIME)
                {
                    await AppendAndCheckpoint(context);
                }
                await toAppend.WriteAsync(data, 0, data.Length);

                Console.WriteLine(string.Format("Message received.  Partition: '{0}', Data: '{1}'",
                  context.Lease.PartitionId, Encoding.UTF8.GetString(data)));
            }
        }

        private async Task AppendAndCheckpoint(PartitionContext context)
        {
            var blockIdString = String.Format("startSeq:{0}", currentBlockInitOffset.ToString("0000000000000000000000000"));
            var blockId = Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes(blockIdString));
            toAppend.Seek(0, SeekOrigin.Begin);
            byte[] md5 = MD5.Create().ComputeHash(toAppend);
            toAppend.Seek(0, SeekOrigin.Begin);

            var blobName = String.Format("iothubd2c_{0}", context.Lease.PartitionId);
            var currentBlob = blobContainer.GetBlockBlobReference(blobName);

            if (await currentBlob.ExistsAsync())
            {
                await currentBlob.PutBlockAsync(blockId, toAppend, Convert.ToBase64String(md5));
                var blockList = await currentBlob.DownloadBlockListAsync();
                var newBlockList = new List<string>(blockList.Select(b => b.Name));

                if (newBlockList.Count() > 0 && newBlockList.Last() != blockId)
                {
                    newBlockList.Add(blockId);
                    WriteHighlightedMessage(String.Format("Appending block id: {0} to blob: {1}", blockIdString, currentBlob.Name));
                }
                else
                {
                    WriteHighlightedMessage(String.Format("Overwriting block id: {0}", blockIdString));
                }
                await currentBlob.PutBlockListAsync(newBlockList);
            }
            else
            {
                await currentBlob.PutBlockAsync(blockId, toAppend, Convert.ToBase64String(md5));
                var newBlockList = new List<string>();
                newBlockList.Add(blockId);
                await currentBlob.PutBlockListAsync(newBlockList);

                WriteHighlightedMessage(String.Format("Created new blob", currentBlob.Name));
            }

            toAppend.Dispose();
            toAppend = new MemoryStream(MAX_BLOCK_SIZE);

            // checkpoint.
            await context.CheckpointAsync();
            WriteHighlightedMessage(String.Format("Checkpointed partition: {0}", context.Lease.PartitionId));

            currentBlockInitOffset = long.Parse(context.Lease.Offset);
            stopwatch.Restart();
        }

        private void WriteHighlightedMessage(string message)
        {
            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine(message);
            Console.ResetColor();
        }
    }
}
Program.csC#
program.cs of ProcessDeviceToCloudMessages app
using Microsoft.ServiceBus.Messaging;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ProcessDeviceToCloudMessages
{
    class Program
    {
        static void Main(string[] args)
        {
            string iotHubConnectionString = "{iot hub connection string}";
            string iotHubD2cEndpoint = "messages/events";
            StoreEventProcessor.StorageConnectionString = "{storage connection string}";
            StoreEventProcessor.ServiceBusConnectionString = "{service bus send connection string}";

            string eventProcessorHostName = Guid.NewGuid().ToString();
            EventProcessorHost eventProcessorHost = new EventProcessorHost(eventProcessorHostName, iotHubD2cEndpoint, EventHubConsumerGroup.DefaultGroupName, iotHubConnectionString, StoreEventProcessor.StorageConnectionString, "messages-events");
            Console.WriteLine("Registering EventProcessor...");
            eventProcessorHost.RegisterEventProcessorAsync<StoreEventProcessor>().Wait();

            Console.WriteLine("Receiving. Press enter key to stop worker.");
            Console.ReadLine();
            eventProcessorHost.UnregisterEventProcessorAsync().Wait();
        }
    }
}
Program.csC#
Program.cs of ProcessD2CInteractiveMessages app
using Microsoft.Azure.NotificationHubs;
using Microsoft.ServiceBus.Messaging;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ProcessD2CInteractiveMessages
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Process D2C Interactive Messages app\n");

            string connectionString = "{service bus listen connection string}";
            QueueClient Client = QueueClient.CreateFromConnectionString(connectionString, "d2ctutorial");

            OnMessageOptions options = new OnMessageOptions();
            options.AutoComplete = false;
            options.AutoRenewTimeout = TimeSpan.FromMinutes(1);

            Client.OnMessage((message) =>
            {
                try
                {
                    var bodyStream = message.GetBody<Stream>();
                    bodyStream.Position = 0;
                    var bodyAsString = new StreamReader(bodyStream, Encoding.ASCII).ReadToEnd();

                    Console.WriteLine("Received message: {0} messageId: {1}", bodyAsString, message.MessageId);
                    SendNotificationAsync(bodyAsString);

                    message.Complete();
                }
                catch (Exception)
                {
                    Console.WriteLine("Message abandon!");
                    message.Abandon();
                }
            }, options);

            Console.WriteLine("Receiving interactive messages from SB queue...");
            Console.WriteLine("Press any key to exit.");
            Console.ReadLine();
        }

        private static async void SendNotificationAsync(String notification)
        {
            NotificationHubClient hub = NotificationHubClient
                .CreateClientFromConnectionString("<connection string with full access>", "<hub name>");
            var toast = @"<toast><visual><binding template=""ToastText01""><text id=""1"">" + notification + "</text></binding></visual></toast>";
            await hub.SendWindowsNativeNotificationAsync(toast);
        }
    }
}

Schematics

Smart Queue Breadboard
Smartqueue bb
Smart Queue Schematics
Smartqueue schem

Comments

Similar projects you might like

IoT for coins

Project tutorial by Erik Moran

  • 8,373 views
  • 1 comment
  • 42 respects

GPS Datalogger, Spatial Analysis, and Azure IoT Hub.

Project tutorial by Shawn Cruise

  • 18,275 views
  • 4 comments
  • 70 respects

IoT PCR: Low Cost DNA Replication Connected to the Internet.

Project showcase by kemfic

  • 8,590 views
  • 0 comments
  • 29 respects

IoT Pull-Up Bar

Project showcase by Mark Dreyband

  • 8,065 views
  • 8 comments
  • 30 respects

Azure Stream Analytics saving lives!

Project tutorial by Asad Zia

  • 6,413 views
  • 3 comments
  • 26 respects

A fall detection system based on Arduino, Windows and Azure

Project tutorial by Jiong Shi

  • 21,584 views
  • 7 comments
  • 46 respects
Add projectSign up / Login