Project tutorial
eMotion - Towards a Better Future

eMotion - Towards a Better Future © GPL3+

We believe we can use biometric sensors, the security of the Helium platform and strength of Google Cloud to surface possible anxiety states

  • 4,982 views
  • 2 comments
  • 14 respects

Components and supplies

Apps and online services

About this project

We believe that we can use biometric sensors to surface possible anxiety states.

Levels of anxiety are increasing worldwide. Between 2005 and 2015 the amount of people living with anxiety on daily basis has increased by 15%. Even more so, anxiety is particularly problematic in autistic patients. They can have severe anxiety attacks in many cases leading to self-harm.

What are we doing about it?

Inspired by the desire of helping a close friend, as well as millions of people across the globe, we've decided to create a concept that can surface possible anxiety states in early stages of manifestation. We are using a blend of biometric measurements combined with the potential of machine learning to correlate and display relevant events.

We believe that a combination of skin resistance, pulse data, brainwave patterns and machine learning provides a more reliable indicator of anxiety states that any other commercially available product. By taking this concept on a product journey we would be able to provide direct benefits to the individuals and carers and system benefits for government and health organisations.

How does the Device work?

The Device measures several parameters and sends the combinations securely via our Helium Channel to Google Cloud for analysis. We can then visualise the interpreted results and flag patterns that we believe match a state of Anxiety.

What are we Measuring?

Galvanic skin response (GSR), is a method of measuring the electrical conductance (Actually resistance!) of the skin. Strong emotion may cause stimulus to your nervous system, resulting more sweat being secreted by the sweat glands. We measure this using a Grove GSR Sensor

Heart rate is a critical indicator of many physical and mental conditions. Anxiety is generally likely to result in faster, or irregular Heart rates. In some cases, Highly fit / Athletic Individuals, or individuals that have experienced extended bed rest can be experiencing an Anxiety state, but also seeing lower heart rates than generally acknowledged as "normal". Knowing your own Heart rate patterns is an important part of interpreting an Anxiety state. We measure heart rate using a Pulse sensor

Brain Emissions

The Human Brain is an amazing and complex organ that has been studied and marvelled for centuries. Significant Electroencephalography (EEG) research has repeatedly shown that the brain produces low frequency emissions that vary in frequency and amplitude according to activities and mental state. A very high level overview generally categorises these frequencies into five distinct bands.

0.5-4 Hz: Delta waves are associated with deep sleep and are known for triggering healing and growth hormones, hence the reason why sleep is so important!

4-8 Hz: Theta waves apparently occur mainly in children during early sleep stages, in emotionally stressed adults, or are somehow involved with learning and navigation

8-13 Hz: Alpha waves occur when a person is relaxed, but alert

14-30 Hz: Beta waves occur when a person is focused, alert and engaged

above 30 Hz: Gamma waves are associated with sensory perception or perception of consciousness. Considerable research is still being done on all of these.

We measure Brain emissions using the innovative Neurosky Mindwave Mobile 2 Headset, which uses a TGAT Application Specific Integrated Circuit (ASIC) to perform Network edge calculations prior to our Helium Channel Transport. We found Neurosky incredibly helpful to us, and they have an excellent set of Developer tools to enable Arduino connectivity.

Headset connectivity is acheived via Bluetooth at 57600 baud. This is accomplished by pairing a BlueSmirf Module with the Headset, restarting, and then piping all RX / TX direct between the Mindwave Mobile 2 and the Arduino/Atom.

The Build

The Build consists of Hardware and Software components. Let's tackle the Hardware first. As in all great projects, Build up in small sections and test, test, Test!

Hardware - Helium Atom

The Helium Atom is a dual-band, long range, low-power, hardware-secured IEEE-standards compliant wireless module that can be purchased in a starter kit. We are very conscious that our device handles health specific data, and the Helium platform bakes in everything you need for highly secure cloud connectivity. In the Kit is an adaptor shield that fits directly over your Arduino, the kit also contains the Helium Element access point. The Atom connects to the nearest Element out of the box with little interference and requiring no device-level configuration.

You can plug the adaptor directly over your Arduino UNO, and there is extensive and excellent documentation on the Helium Developer site to get you started. However If you normally plug and play without reading the documentation, several items are worthy of note. Here is our Atom on the adaptor with points of interest circled in Red:

We are using an Arduino UNO, so the default interrupt settings of 8 and 9 worked fine. Depending on the specific Arduino board, you will have to configure the RX and TX jumpers on the Adapter to get the board to talk to the Helium Atom using the right pins. Take care to get the antenna orientation correct - follow the labels! FInally you will need the MAC address in order to activate the Atom and activate your Element in your Helium Dashboard. It is worthwhile now going through the activation sequence to ensure your Atom and Element are functioning correctly and sending sample data to the Helium Dashboard.

Hardware - Headset Connectivity

In this stage we initiate basic Headset connectivity. It's worthwhile to start with a fresh sketch and remove the Helium adaptor while getting the Headset connected and paired through our BlueSmirf pipe. The pairing process is documented in a comprehensive worked example here. Our Headset MAC address is "9807 2D80 140B", PIN is "0000", Mode is set to "AutoConnect".

Remember that the Uno has one Serial Port - Once the BlueSmirf is connected to D0 and D1, it's not possible to upload further sketches without errors. You will need to Power down, disconnect the BlueSmirf, Power up, upload new sketch, power down - reconnect the BlueSmirf and power back up! Make sure you also connect and test the Helium Atom adaptor shield at each stage to ensure no IO conflicts exist.

Once everything has been checked and tested, Power up the Headset.

Hardware - Pulse sensor

The Pulse sensor is straightforward after the Headset! We recommend that you disconnect everything and start with a new plain sketch. Read the documentation for the Pulse Sensor and install the Libraries. Connect up the red sensor wire to 5v (VCC) and the black sensor wire to 0v (GND). We connected the Purple Sensor wire to the A0 Analog input for our device. Run the sample sketches and You should be able to get an Arduino plot output easily using this method.

Hardware - GSR Sensor

The GSR sensor connection and testing is also straightforward. The finger electrodes plug directly into the Grove board, then the Grove connector has three wires we need to connect. Connect a red wire from Grove to 5v (VCC) and a black wire from Grove to 0v (GND). We connected the yellow Grove wire to the A1 Analog input for our device. Take care to follow manufacturer instructions to calibrate the Midpoint reference for the GSR - it should be adjusted to 512 (1/2 of the Arduino 1024 value ADC). A sample sketch to check basic operation is also available here.

Combine and Build....

After we had confirmed the operation of each sensor and sketch individually, it was time to combine everything together and ensure the whole device worked as expected. Rather than a Breadboard, we find that using a screw connector shield helps to keep everything tightly connected..... but not necessarily neat! Here you can see our Helium Atom adaptor shield stacked under the Terminal shield.

Hardware - Final steps

The Hardware now needs to be minimised and integrated with all sensors into a single cap that can be worn with dignity in an everyday situation.

Software- Overview

After initializing the sensors and connecting to the Helium cloud, our code connects to each of the three sensors, collects the current data and formats it as a JSON string. This is then sent to Google cloud IOT core via a Helium MQTT channel. It repeats every 5 seconds.

Software - Google Cloud IOT core

To kick off, We registered with Google Cloud and created a Firebase project. We then built a Firestore Database using the Google Documentation and began to add our JSON data. You can see the function to subscribe to the IoT Pub/Sub to store the data here on our Github.

Once the Firebase project, was created, We uploaded our website to Firebase Hosting (which has a inbuilt connection to Firestore) then subscribed to the Google cloud IOT publisher to listen for real time changes and store them in Firestore. Our website listens to new data points added and charts them on an interactive graph for us to visualize our health state. With the Firebase Server set up, you can see the output below

Here is a Demonstration of the system in action!

The complete system

Software- Finalsteps

With the Data flowing into our site we can now begin to correlate individual data against several large sample data sets, then produce and send meaningful alerts to individuals and their Carers.

At a personal level this information may allow individuals to self recognise and choose to manage their emotions differently. It may also allow their Carers to respond more rapidly, armed with more detailed specific knowledge of an often dynamic and misunderstood emotional state.

At a de-personalised Macro or System level, this information may provide Government and Health care professionals signficant insights into endemic issues within our society at a local, specific event or discrete community level that may previously have been invisible. This may contribute to more responsive and inclusive policies and more effective community health management.

Our outputs can be seen in our Youtube Video or direct on the Firebase site, All of our code for this project is also available on our Github.

Code

Source CodeArduino
The program reads heart rate, galvanic skin response, and mindfulness from the connected sensors every 5 seconds and sends them to Google cloud via the Helium cloud.
After initializing the sensors and connecting to the Helium cloud, the program connects to each of the three sensors, collects the current data and formats it as a json string. This is then sent to Google cloud via Helium. It repeats every 5 seconds.
/****************************************************************************************************************
* Descdiption: Program to measure heart rate, GSR, and brain activity and send to Google cloud for analysis
*              to determine onset of Anxiety.
*              Uses ???? for heart rate monitoring
*              Uses ???? for GSR
*              Uses Neurosky mobile headset for measuring brain activity
*****************************************************************************************************************/
#define DEBUG 1                        // Will output messages to the serial monitor
#define USE_ARDUINO_INTERRUPTS true    // Set-up low-level interrupts for most acurate BPM math.
#define BAUDRATE 57600                 // The default for the bluesmirf

#include "Arduino.h"
#include "Board.h"
#include "Helium.h"
#include "HeliumUtil.h"
#include <PulseSensorPlayground.h>     // Includes the PulseSensorPlayground Library.  

#define CHANNEL_NAME "NeuroG"          // The name of the Helium channel configured to send data to Google cloud

//  Variables
const int PulseWire = 0;               // PulseSensor PURPLE WIRE connected to ANALOG PIN 0
const int GSRPin=A1;                   // GSR connected to Analog Pin 1
const int LED13 = 13;                  // The on-board Arduino LED, close to PIN 13.
int Threshold = 550;                   // Determine which Signal to "count as a beat" and which to ignore.
                             
PulseSensorPlayground pulseSensor;     // Creates an instance of the PulseSensorPlayground object called "pulseSensor"

Helium  helium(&atom_serial);          // Standard Helium objects
Channel channel(&helium);


void channel_send(void const * data, size_t len)  // used to send data to Helium channel
{
    int    status;
    int8_t result;

    status = channel.send(data, len, &result);
}

byte ReadOneByte()                     // Returns a byte from the Neurosky headset which is connected via bluesmirf bluetooth adaptor
{
  int ByteRead;

  while(!Serial.available());          // wait until there's some data then 
  ByteRead = Serial.read();            // get a single byte of data 

  #if DEBUG  
    Serial.print((char)ByteRead);      // echo the same byte out the USB serial (for debug purposes)
  #endif

  return ByteRead;
}


String getMindwaveData()               // read the data stream coming from the Mindwave and split into components
{                                      // adapted from http://developer.neurosky.com/docs/doku.php?id=mindwave_mobile_and_arduino
  // checksum variables
  byte generatedChecksum = 0;
  byte checksum = 0; 
  int payloadLength = 0;
  byte payloadData[64] = {0};
  
  // system variables
  long lastReceivedPacket = 0;
  boolean bigPacket = false;
  String MindwaveData;                 // a json formatted string of the values returned from the mindwave headset

  
  byte poorQuality = 0;                // indicates quality of signal from mindwave headset
  byte attention = 0;                  // level of attention reading as determined by mindwave headset
  byte meditation = 0;                 // level of meditation reading as determined by mindwave headset

  // Look for sync bytes
  while(ReadOneByte() != 170);

    if(ReadOneByte() == 170) {

      payloadLength = ReadOneByte();
      if(payloadLength > 169)                      //Payload length can not be greater than 169
          return;

      generatedChecksum = 0;        
      for(int i = 0; i < payloadLength; i++) {  
        payloadData[i] = ReadOneByte();            //Read payload into memory
        generatedChecksum += payloadData[i];
      }   

      checksum = ReadOneByte();                      //Read checksum byte from stream      
      generatedChecksum = 255 - generatedChecksum;   //Take one's compliment of generated checksum

        if(checksum == generatedChecksum) {    

        poorQuality = 200;
        attention = 0;
        meditation = 0;

        for(int i = 0; i < payloadLength; i++) {    // Parse the payload
          switch (payloadData[i]) {
          case 2:
            i++;            
            poorQuality = payloadData[i];
            bigPacket = true;            
            break;
          case 4:
            i++;
            attention = payloadData[i];                        
            break;
          case 5:
            i++;
            meditation = payloadData[i];
            break;
          case 0x80:
            i = i + 3;
            break;
          case 0x83:
            i = i + 25;      
            break;
          default:
            break;
          } // switch
        } // for loop

        // save the data into a json format string
        if(bigPacket) {
          MindwaveData += "\"poorQuality\":" + String(poorQuality) + ",";
          MindwaveData += "\"attention\":" + String(attention) + ",";
          MindwaveData += "\"meditation\":" + String(meditation);
        }
      }
    }

  return(MindwaveData);
}

String getHeartRate()                                     // read the heart rate from the pulse sensor
{                                                         // adapted from https://pulsesensor.com/pages/getting-advanced
  String HeartRate="";
 
  int myBPM = pulseSensor.getBeatsPerMinute();            // Calls function on our pulseSensor object that returns BPM as an "int".
                                                          // "myBPM" hold this BPM value now.
  if (pulseSensor.sawStartOfBeat()) {                     // Constantly test to see if "a beat happened".
    HeartRate = "\"heartRate\":" + String(myBPM) + ",";   // Save the heart rate as a json string
  }
  return(HeartRate);

}



String getGSR()                                           // read the galvanic skin response from the sensor on analogue pins
{
  int sensorValue=0;                                      // used for a single reading
  int gsr_average=0;                                      // to average 10 readings
  long sum=0;
  String GSR;                                             // json formatted string to return
  
  for(int i=0;i<10;i++)                                   // Average the 10 measurements to remove the glitch
     {
     sensorValue=analogRead(GSRPin);                      // GSR sensor connected to analogue pin
     sum += sensorValue;
     delay(5);
     }
  gsr_average = sum/10;                                   // calculate the average of the 10 readings
  GSR = "\"GSR\":" + String(gsr_average) + ",";

  return(GSR);
}

String readSensorMeasurements()                           // read from the 3 sensors and concatenate into a single string
{  
  String totalMessage;
 
  totalMessage = "{" ;
  totalMessage += getHeartRate();
  totalMessage += getGSR();
  totalMessage += getMindwaveData();
  totalMessage += "}";
  
  return(totalMessage);
}


void setup()
{
    Serial.begin(BAUDRATE);
    DBG_PRINTLN(F("Starting"));

    // Begin communication with the Helium Atom The baud rate differs
    // per supported board and is configured in Board.h
    helium.begin(HELIUM_BAUD_RATE);

    // Connect the Atom to the Helium Network
    DBG_PRINTLN(F("Connecting"));
    helium_connect(&helium);
    // and do a channel connect
    DBG_PRINTLN(F("Creating Channel"));
    channel_create(&channel, CHANNEL_NAME);
    DBG_PRINTLN(F("Channel created"));

   // Configure the PulseSensor object, by assigning our variables to it.
   pulseSensor.analogInput(PulseWire);  
   pulseSensor.blinkOnPulse(LED13);       //auto-magically blink Arduino's LED with heartbeat.
   pulseSensor.setThreshold(Threshold);  
  
   // Double-check the "pulseSensor" object was created and "began" seeing a signal.
    if (pulseSensor.begin()) {
     Serial.println("Checking......");  //This prints one time at Arduino power-up,  or on Arduino reset.  
    }
}

void loop()
{

    // Send some data to the configured channel
    int8_t result;
    String sensorMeasurements;
    size_t used;

    DBG_PRINTLN(F("Reading Sensors"));
    sensorMeasurements = readSensorMeasurements();
    used = sensorMeasurements.length();
    
    DBG_PRINT(F("Measurements - "));
    DBG_PRINTLN(sensorMeasurements);
    
    channel_send(sensorMeasurements.c_str(), used);
    DBG_PRINTLN(F("Sent to Helium"));

    // Wait a while till the next time
    delay(5000);
}
IoT Code
The program reads heart rate, galvanic skin response, and mindfulness from the connected sensors every 5 seconds and sends them to Google cloud via the Helium cloud. After initializing the sensors and connecting to the Helium cloud, the program connects to each of the three sensors, collects the current data and formats it as a json string. This is then sent to Google cloud via Helium. It repeats every 5 seconds.
Firebase Code
Code to demo the graphs on Firebase: https://neurog-206604.firebaseapp.com/ This program creates a function to subscribe to Google IoT Core and push the data to FireStore for further analysis. We then use the FireStore data to show live readings on a graph.

Schematics

Circuit schematic
This is the representative schematic file (Helium Atom omitted for clarity)
Schematicv3 iymorjjtpx
Circuit diagram
This is our circuit as built, omitting the Helium Atom for clarity
Breadboardv3 i0iactohga

Comments

Similar projects you might like

SPCPM (Solar Powered City Pollution Monitor)

Project tutorial by 5 developers

  • 13,090 views
  • 2 comments
  • 40 respects

MushroomBot

Project tutorial by Jeff Gensler

  • 2,211 views
  • 0 comments
  • 8 respects

Smartap Future of Faucets

Project tutorial by Ferdinand Thomas

  • 1,755 views
  • 1 comment
  • 10 respects

The Train of the Future

Project showcase by Team STT

  • 17,510 views
  • 5 comments
  • 22 respects

Octopod: Smart IoT Home/Industry Automation Project

Project tutorial by Saksham Bhutani

  • 13,406 views
  • 11 comments
  • 44 respects
Add projectSign up / Login