Project in progress
The Smart Table Assistant

The Smart Table Assistant © CC BY-NC-ND

A voice-powered table assistant for makers and hackers to help manage inventory information and control power to maker/ home appliances.

  • 2,988 views
  • 0 comments
  • 6 respects

Components and supplies

Necessary tools and machines

3drag
3D Printer (generic)

Apps and online services

About this project

The Problem Statement

  • Imagine you are busy soldering something at your table and you need to turn your solder iron OFF/ the lights ON.
  • Lets assume that's a basic solder you have. Also, you need to quickly know where you have placed the 220 ohm resistor. Let's assume you have not labelled your inventory.

This is where the Smart Table Assistant come into picture. It helps to manage inventory information and control power to appliances hands-free and through voice. You can ask the assistant to turn off the solder and ask for the place where you have placed the resistor (provided you have not changed it from the place where you had told the assistant to remember)!

Introduction

A voice-powered smart table assistant for makers and hackers to help manage inventory information and control power to appliances like a table lamp, solder iron and more, when their hands are busy developing. Being a developer myself, I am aware of the difficulty of managing inventory items and that too, when you are developing at a rapid pace. In such situations imagine an assistant to whom you can tell,

"Hey, remember that I have taken 220 ohm resistor and a 4 channel 5V DC relay module"

and you can keep adding items to the list whenever you take them from your inventory.

At the end of the session, the assistant will have the list of the items you have taken out and will remind you to place them back at their respective places. Say,

"John you have taken out these items and they are on your table. You need to place the 220 ohm resistor in the white box inside your second table drawer and so on".

Also imagine a scenario where you are soldering something and without taking your hands off, you need to turn ON/ OFF a solder iron you are using or a table lamp. This is one more scenario where the smart table assistant can come in handy and useful when you are the only person available to both develop your project and need to record a video of what you are making. You can say to the assistant,

"Hey, turn the focus lights off"

or

"Hey, turn the monitor* on"

*monitor - may be used for the purpose of showing the camera feed or to display the dialogue script.

It is useful for makers and hackers who are all by their own and who need some extra pair of imaginary hands to help!

Plan

As explained earlier the main features are

  • to help makers and hackers manage their inventory and
  • to control appliances through voice

when both their hands are busy building something.

The solution will have the Voice Interaction Base Kit as the central brain of the system. User will ask it what he/she needs, to the kit and based on the voice command received it will take the respective action tagged under it.

For controlling appliances it will instruct the wirelessly connected smart relay system to turn on/ off the appliance with respect to the voice commands received. This system consists of the following components,

  • Arduino MKR1000 - to receive POST requests from IFTTT through thinger.io and then based on it, instruct the serially connected Arduino UNO to turn ON/ OFF relay respective to the voice command.
  • Arduino UNO - it receives serial information from Arduino MKR1000 to turn ON/ OFF relay respective to the voice command.
  • Relay Module - it receives signals to close or open the AC power circuit connected to it.

For smart inventory management, the kit will receive information on where the items are placed. This will be stored in Google Sheets which will act as a database which can be retrieved or altered when the users requests so. IFTTT can act as a bridge between the voice interaction base kit and Google Sheets.

Unboxing, Assembling and OOB Demo of the Voice Interaction Base Kit

In the video below, I'll show the unboxing of the voice interaction base kit and a timelapse of how to assemble the parts of the kit together. After which I'll show the out of the box (OOB) demo of the kit which has the below conversation,

[Me] - "Hey Snips, what's the currenttemperature?"

[Assistant] - "The current temperature is 31.2 degree"

[Me] - "Hey Snips, what's the humidity?"

[Assistant] - "The current humidity is 66.7 percent"

[Me] - "Hey Snips, turn on the relay"

[Assistant] -

***Plays a sound and turns the relay on but this is not shown in the video!***

[Me] - "Hey Snips, turn off the relay"

[Assistant] -

***Plays a sound and turns the relay off but this is not shown in the video!***

Unboxing, Assembling and OOB Demo | Voice Interaction Base Kit

For a more detailed explanation and code of the out of the box demo project - Temperature & Relay, please visit the snips official github repository link - https://github.com/snipsco/snips-demo-dev-kit

The Grove Relay is a 5V DC relay which will turn ON when we instruct it through voice and it will be indicated by a Red LED (D3). However, it receives, only 3.3V from the Raspberry Pi 3B+ and hence the indicator Red LED will not glow brightly and also it will not properly close/ open the AC circuit connected to it. Therefore connect it to a 5V signal to make it to work.

Product Page - http://wiki.seeedstudio.com/Grove-Relay/

Smart Relay

The process goes like this when you instruct Snips, the instructions are understood by Snips and it calls out the respective applet in IFTTT. Then IFTTT triggers the respective action for the obtained input from Snips. The action would be to pass a POST request to thinger.io with JSON body as below.

{
 "in": {
   "state": 1
 }
}

This has the input variable 'state' passed to the Arduino MKR1000 through its API in thinger.io. Based on the state passed Arduino MKR1000 serially communicates the corresponding relay to be turned ON/ OFF to the Arduino UNOconnected to the Relay module. For instance, 1 - Turns the Relay1 ON and 2 - Turns OFF the Relay1, 3 & 4 for the next relay ON & OFF and 5 & 6 for the third relay. You can connect your home appliances to the extension box and control using voice commands for the respective relays.

I have modeled and 3D printed a case for the Smart Relay system, find below the gallery of images of the finished case containing the system.

Below is the Smart Relay System in action without Snips.ai.

Conclusion

This is a challenge where I have learned so much in a little span of time. I was recently introduced to the snips ai platform through matrix creator. Immediately after which I was selected in this competition for free hardware kit and it was so much fun with the voice interaction base kit. Thanks once again for sponsoring! This is also when I got introduced to 3D modeling and printing. So I was able to 3D print a case and light bulb holder for the smart relay system. However the one part which put me back and where I struggled the most was the code actions and this error,

The session was ended because one of the component didn't respond in a timely manner

There are so many discussions for this error in the Snips forum - https://forum.snips.ai/search?q=The%20session%20was%20ended%20because%20one%20of%20the%20component%20didn%27t%20respond%20in%20a%20timely%20manner

and few useful documentations regarding code actions (official and user blogs) - https://docs.snips.ai/additional-resources/troubleshooting and https://www.smb-tec.com/blog/5-quick-tips-about-snips-skill-actions

However I could not resolve it and progress further. If I had found a video demo or programming documentation of code actions end to end implementation and basics (which is not available atleast in English), this would have been complete by now. But still I will continue to build and try to complete this project and blog here. Stay tuned!

1. Challenge page - https://www.hackster.io/contests/building-for-voice

2. Snips console - https://console.snips.ai/

3. Snips documentation - https://docs.snips.ai/the-maker-kit/dev-kit

4. Snips forum - https://forum.snips.ai/

5. Voice Interaction Base Kit product page - http://wiki.seeedstudio.com/Snips_Voice_Interaction_Base_Kit/

6. Snips quick start YouTube playlist - https://www.youtube.com/playlist?list=PLmOf6ALFW9aY_EZOYtTM27TqKSasHqZr_

7. Code action reference - https://www.smb-tec.com/blog/5-quick-tips-about-snips-skill-actions

8. Troubleshooting - https://docs.snips.ai/additional-resources/troubleshooting

9. Snips GitHub page - https://github.com/snipsco

Code

Smart Relay - Arduino UNO CodeC/C++
This is the code for the Arduino UNO used as part of the Smart Relay which controls the 4 channel 5V relay module.
/* This is the code for Arduino UNO used in the 'Smart Relay' part of the 'Smart Table Assistant project' built for the 
 *  'Smart Homes on the Edge' contest conducted by Hackster.io and sponsored by Seeed and Snips.
 * 
 * Contest Page - https://www.hackster.io/contests/building-for-voice
 * 
 * Blog Link - 
 * 
 * Author : Dixon Selvan
 * Date   : April 11, 2019
 * Project: Smart Table Assitant
 * Website: 
 * 
 * Hardware components Required
 * ----------------------------------------
 * 1. Arduino UNO board
 * 2. Four Channel Relay Module
 * 3. Arduino MKR1000
 * 4. Few Jumper wires
 * 
 * Connections
 * -----------
 * Arduino UNO      |   Relay Module
 * ---------------------------------------------
 *      5V          |         VCC
 *      Gnd         |         Gnd
 *      IN1         |         D10     
 *      IN2         |         D11
 *      IN3         |         D5
 *      IN4         |         D6
 * Arduino UNO      |   Arduino MKR1000
 * ---------------------------------------------
 *      Vin         |         +5V
 *      Gnd         |         Gnd
 *      RX          |         TX     
 *      TX          |         RX
 * 
 */
 
int Relay1 = 10;
int Relay2 = 11;
int Relay3 = 5;
int Relay4 = 6;
int serialData = 0;

void setup() {
  // put your setup code here, to run once:
  pinMode(Relay1, OUTPUT);
  pinMode(Relay2, OUTPUT);
  pinMode(Relay3, OUTPUT);
  pinMode(Relay4, OUTPUT);
  Serial.begin(38400);
  digitalWrite(Relay1, HIGH);
  digitalWrite(Relay2, HIGH);
  digitalWrite(Relay3, HIGH);
  digitalWrite(Relay4, HIGH);
}

void loop() {
  // put your main code here, to run repeatedly:
  if(Serial.available()>0){
     serialData = Serial.read();
     Serial.println(serialData);
     if (serialData == 49){
      digitalWrite(Relay1, LOW);
     }
     else if (serialData == 50){
      digitalWrite(Relay1, HIGH);
     }
     else if (serialData == 51){
      digitalWrite(Relay2, LOW);  
     }
     else if (serialData == 52){
      digitalWrite(Relay2, HIGH); 
     }
     else if (serialData == 53){
      digitalWrite(Relay3, LOW);
     }
     else if (serialData == 54){
      digitalWrite(Relay3, HIGH);
     }
     else if (serialData == 55){
      digitalWrite(Relay4, LOW);  
     }
     else if (serialData == 56){
      digitalWrite(Relay4, HIGH); 
     }
  }
}
Smart Relay - Arduino MKR1000 Code - Secrets.hC/C++
This is the secrets.h code for the Arduino MKR1000 used as part of the Smart Relay which controls the Arduino UNO connected to the 4 channel 5V relay module.
#define SECRET_SSID ""
#define SECRET_PASS ""
#define SECRET_USERNAME ""
#define SECRET_DEVICEID ""
#define SECRET_DEVICECREDENTIAL ""
Smart Relay - Arduino MKR1000 CodeC/C++
This is the code for the Arduino MKR1000 used as part of the Smart Relay which controls the Arduino UNO connected to the 4 channel 5V relay module.
/* This is the code for Arduino MKR1000 used in the 'Smart Relay' part of the 'Smart Table Assistant project' built for the 
 *  'Smart Homes on the Edge' contest conducted by Hackster.io and sponsored by Seeed and Snips.
 * 
 * Contest Page - https://www.hackster.io/contests/building-for-voice
 * 
 * Blog Link - 
 * 
 * Author : Dixon Selvan
 * Date   : April 11, 2019
 * Project: Smart Table Assitant
 * Website: 
 * 
 * Hardware components Required
 * ----------------------------
 * 1. Arduino MKR1000 board
 * 2. Few Jumper wires
 * 3. [Optional] 3.7V Lithium Battery
 * 
 * Connecting Arduino MKR1000 with thinger.io, getting started guide - http://docs.thinger.io/arduino/
 */

/*Disabling the secure TLS/SSL connection*/
#define _DISABLE_TLS_

#include <WiFi101.h>
#include <ThingerWifi101.h>
#include "arduino_secrets.h"

/*Declaring Variables*/
int relayState = 0;

/*Create an account in thinger.io and replace username below with that username. 
Create a new device and replace deviceId, deviceCredential below with the one you had created.*/
ThingerWifi101 thing(SECRET_USERNAME, SECRET_DEVICEID, SECRET_DEVICECREDENTIAL);

void setup() {
  pinMode(LED_BUILTIN, OUTPUT);
    
  /*Serial initialisation*/
  Serial1.begin(38400);
  
  /*Replace the below accrodingly with your WiFi SSID and password*/
  thing.add_wifi(SECRET_SSID, SECRET_PASS);

  /*The relay state is obtained as an input resource (integer/ number) from thinger.io*/
  thing["Relay"] << [](pson& in){
      relayState = in["state"];
      changeRelayState();
  };
}

void loop() {
  thing.handle();
}

void changeRelayState(){
  if(relayState == 1)
  {
    digitalWrite(LED_BUILTIN, HIGH);
    Serial1.print(1);
  }
  else if (relayState == 2){
    digitalWrite(LED_BUILTIN, LOW);
    Serial1.print(2);
  }
  else if(relayState == 3)
  {
    digitalWrite(LED_BUILTIN, HIGH);
    Serial1.print(3);
  }
  else if (relayState == 4){
    digitalWrite(LED_BUILTIN, LOW);
    Serial1.print(4);
  }
  else  if(relayState == 5)
  {
    digitalWrite(LED_BUILTIN, HIGH);
    Serial1.print(5);
  }
  else if (relayState == 6){
    digitalWrite(LED_BUILTIN, LOW);
    Serial1.print(6);
  }
}
Smart Table Assistant - GitHub Repository
This is the GitHub Repository where you can find the entire code for the 'Smart Table Assistant' project.

Custom parts and enclosures

Smart Relay Case
This is a smart relay case built to contain Arduino MKR1000, Arduino UNO, 4 channel 5V DC Relay module. The connections can be extended through jumper wires which can go through the slot inside each compartment and the top lid.

Schematics

Smart Relay
Circuit Diagram of the 'Smart Relay' part of the 'Smart Table Assistant'
Smart relay bb b2kwkxcfjp

Comments

Similar projects you might like

Smart Blind Assistant

Project in progress by Chandru S

  • 3,698 views
  • 1 comment
  • 5 respects

Health Band - A Smart Assistant for the Elderly

Project tutorial by Technovation

  • 24,581 views
  • 13 comments
  • 68 respects

Candy Dispenser with Google Assistant

Project tutorial by Arduino “having11” Guy

  • 16,214 views
  • 1 comment
  • 60 respects

Smart Pool: Alexa Controlled Pool Manager

Project tutorial by Benjamin Winiarski

  • 2,007 views
  • 2 comments
  • 8 respects

Smart Personal Money Vault Monitoring System Based on IoT

Project tutorial by Salah Uddin

  • 1,256 views
  • 0 comments
  • 8 respects

Alzheimer's Assistant

Project tutorial by Abdullah Sadiq

  • 26,267 views
  • 19 comments
  • 101 respects
Add projectSign up / Login