Project tutorial
Alexa Humour DJ

Alexa Humour DJ © GPL3+

Chooses the best song and the best illumination type based on your current mood.

  • 1,607 views
  • 0 comments
  • 11 respects

Components and supplies

Necessary tools and machines

3drag
3D Printer (generic)

Apps and online services

About this project

A smart integration of Amazon Alexa services with Raspberry Pi 3 and Arduino UNO.

The purpose of the project is to build a device that is able to play music and light up your room according to your current mood.

In order to achieve this, we turned a Raspberry into an Alexa Echo using Alexa Voice Service kit and we built a custom skill with Alexa Skills Kit in order to understand the user's mood.

After this, we developed a python server that is linked to our custom skill via endpoint and is running on the Raspberry. This server handles the conversation and gathers useful informations to calculate the user's mood.

Done this, the raspberry is ready to play the chosen song and to send a signal to Arduino that will finally light up the room.

In the following chapters we are going to show you how we did it!

1. Alexa Skills Kit

The first step is to build the custom skill for Alexa in order to manage the mood comprehension. So, we move to https://www.developer.amazon.com and then:

Goto Alexa --> Alexa skill kit --> Add new skill.

After this, we land to the following page:

The first thing to do is to select the "Skill Type" "Custom Interaction Model", then we choose a name for the custom skill and the select the invocation name; this part is important because what we write here will be what we'll have to say in order to invoke our skill.

To build the interaction model we use the skill builder. In this section we are going to define the four moods that Alexa can understand:

  • 1. Angriness
  • 2. Happiness
  • 3. Neutral
  • 4. Sadness

These moods are declared as intents, and every intent has a list of utterances that represent the corresponding mood. Below you can see screenshots about every intent we developed:

Once we've finished to declare the four intents we save the model and we build it with the two related buttons:

In the "Configuration" section we create the endpoint to which the requests made to Alexa have to be sent in order to be processed. Since we decided to use the Respberry as server, we need to select HTTPS as "Service Endpoint Type" and, in the "Default" field we set the url. We obtain this url using ngrok (in the next section we will explain how we did it).

The last section to fill is the "SSL Certificate": since we use ngrok we have to choose the second box ("My development endpoint is a sub-domani that has a wildcard certificate from a certificate authority").

Now that the custom skill is ready to be used, we go for the Raspberry setup.

2. Raspberry configuration

Alexa Echo is not available in Italy, so we used the Alexa Voice Service. We installed it in the Raspberry turning it in a sort of homemade Echo.

To do this, we followed this step-by-step tutorial:

https://github.com/alexa/alexa-avs-sample-app/wiki/Raspberry-Pi

Now that AVS has been installed we need to develop the server that processes the informations received from the custom skill.

We decide to use Python as programming language and Flask as web framework. We also use flask_ask (specific Python library for Alexa) in order to easily handle every intent declared in the custom skill.

from flask import Flask
from flask_ask import Ask, statement,question
from flask import request

To turn the Raspberry into a music player we need to use the VLC library. To correctly import it, we followed this tutorial:

https://blog.computerbacon.com/playing-audio-in-python-with-libvlc.html

To communicate with Arduino we use the serial library: in this way we can create a serial channel between Arduino and Raspberry.

import serial  
import vlc  

The server listens on localhost:5000 but, in order to connect the custom skill with our local web server, we have to expose it using ngrok ( https://ngrok.com/ ): this will result in a URL that has to be inserted in the endpoint field mentioned above during the custom skill creation.

app = Flask(__name__)
ask = Ask(app, '/')
.
.
.
if __name__ == '__main__':
    app.run(host='localhost', port = 5000, debug = True)

When someone invokes the custom skill, the server intercepts the "Launch" event and returns a welcome message followed by the first question chosen to understand the user's mood:

"How are you?"

 @ask.launch
def handler():
 global counter
 global player 
   
 if player.is_playing():
     return statement("Humour dj is already playing a song, wait!")
 else:
     return question("Hi! Welcome to the Alexa Humour DJ ! I can play a song for you based on your current mood. First of all, how are you?")

The answer to this question is classified from the custom skill in one of the moods mentioned above (angriness, happiness, neutral, sadness).

After that, Alexa sends an event containing the classification result to the server. This information leads the conversation to the second question:

"How was your day?"

@ask.intent('Happy')
def happy():
 global counter
 global state
 counter += 1
 if counter == 1:
        state["Happy"]+= 1
        return question("And now, tell me, how was your day?")
 elif counter == 2:
        counter = 0
        state["Happy"]+= 1
        chooseSong()

Again, the user's answer will be classified and the server will handle the event fired from the custom skill.

Now that every information has been gathered, we can calculate the result, play some music and send an event to Arduino in order to light up the room!

 def chooseSong():
 global state
 global ser
 global player 
 if state["Happy"] == 2 or ( state["Happy"] ==1 and state["Angry"]==1  ) or (state["Happy"] ==1 and state["Sad"]==1 ):
        player = vlc.MediaPlayer("/home/pi/Desktop/Happy/PharrellWilliams-Happy.mp3")
        player.play()
        Arduino.write(b'H')
 if state["Neutral"] == 2 or (state["Happy"] ==1 and state["Neutral"]==1  ):
        player = vlc.MediaPlayer("/home/pi/Desktop/Neutral/Riders_on_the_Storm.mp3")
        player.play()
        Arduino.write(b'N')
 if state["Angry"] == 2 or (state["Angry"]==1 and state["Sad"]==1  ) or (state["Angr   y"]==1 and state["Neutral"]==1 ):
        player = vlc.MediaPlayer("/home/pi/Desktop/Angry/Enter_Sandman.mp3")
        player.play()
        Arduino.write(b'A')
 if state["Sad"] == 2 or (state["Sad"]==1 and state["Neutral"]==1 ):
        player = vlc.MediaPlayer("/home/pi/Desktop/Sad/Wish_You_Were_Here.mp3")
        player.play()
        Arduino.write(b'S')
 state = {"Happy": 0, "Neutral": 0,"Angry": 0, "Sad" : 0}

The logic that we decided to use to calculate the mood is the following one:

Happiness > Angriness > Sadness > Neutral > Happiness

The entire script with comments has been uploaded in the "Code" section.

3. Arduino

Arduino manages two type of lighting:

  • the first one is relating to the house lights and we decided to simulate it using a led strip directly connected to a breadbord
  • the second one is relating to a 3D printed box that contains some leds that light up following the music rythm

Developing the first one is very simple: in the sketch running on Arduino, you just have to catch the message sent by the Raspberry and turn on the LED stripe using the color related to the mood.

//save the message received from the raspberry in the mood variable
mood=Serial.read();  
switch (mood){
 case 'A': //angry
 {
//set the color Red for Angriness
leds1.setColorRGB(0,255,0,0);
 break;
 }
 case 'S': //sadness
 {
 //set the color Blue for Sadness
 leds1.setColorRGB(0,0,0,255);
 break;
 }
case 'N': //neutral
{
 //set the color White for Neutral
 leds1.setColorRGB(0,255,255,255);
 break;
}
 case 'H': //happiness
 {
//set the color Green for happiness
leds1.setColorRGB(0,0,255,0);
break;
 }
default:
Serial.flush();
}

We decided to use a touch sensor to turn off Arduino ( leds and sound sensor )!

The second part is a bit tricky: in order to implement that, we used a the KY-038 Sound Sensor that listens to the music played by the Raspberry, understands the rythm and coordinates the LEDs in order to turn them on along with the music.

We used 8 LEDs: when the sensor hears a weak sound, only two LEDs turn on; when the sound is strong, the whole set of LEDs light up.

valore_A0 = analogRead(pin_A0);
 Serial.println(valore_A0);
 //Low sounds
 if(valore_A0 <= 160){
   digitalWrite(pin_led_A, LOW);
   digitalWrite(pin_led_B, LOW);
   digitalWrite(pin_led_C, LOW);
   digitalWrite(pin_led_D, LOW);
   digitalWrite(pin_led_E, LOW);
   digitalWrite(pin_led_F, LOW);
   digitalWrite(pin_led_G, LOW);
 }
 if (valore_A0 > 160 && valore_A0 < 162)
 {
   digitalWrite(pin_led_A, HIGH);
   digitalWrite(pin_led_B, LOW);
   digitalWrite(pin_led_C, LOW);
   digitalWrite(pin_led_D, LOW);
   digitalWrite(pin_led_E, LOW);
   digitalWrite(pin_led_F, LOW);
   digitalWrite(pin_led_G, LOW);
 }
 //Middle sounds
 if (valore_A0 >= 162 && valore_A0 < 165)
 {
   digitalWrite(pin_led_A, HIGH);
   digitalWrite(pin_led_B, HIGH);
   digitalWrite(pin_led_C, LOW);
   digitalWrite(pin_led_D, LOW);
   digitalWrite(pin_led_E, LOW);
   digitalWrite(pin_led_F, LOW);
   digitalWrite(pin_led_G, LOW);
 }
 if (valore_A0 >= 165 && valore_A0 <170)
 {
   digitalWrite(pin_led_A, HIGH);
   digitalWrite(pin_led_B, HIGH);
   digitalWrite(pin_led_C, HIGH);
   digitalWrite(pin_led_D, LOW);
   digitalWrite(pin_led_E, LOW);
   digitalWrite(pin_led_F, LOW);
   digitalWrite(pin_led_G, LOW);
 }
 if (valore_A0 >= 170 && valore_A0 < 180)
 {
   digitalWrite(pin_led_A, HIGH);
   digitalWrite(pin_led_B, HIGH);
   digitalWrite(pin_led_C, HIGH);
   digitalWrite(pin_led_D, HIGH);
   digitalWrite(pin_led_E, LOW);
   digitalWrite(pin_led_F, LOW);
   digitalWrite(pin_led_G, LOW);
 }
 if (valore_A0 >= 180 && valore_A0 < 190)
 {
   digitalWrite(pin_led_A, HIGH);
   digitalWrite(pin_led_B, HIGH);
   digitalWrite(pin_led_C, HIGH);
   digitalWrite(pin_led_D, HIGH);
   digitalWrite(pin_led_E, HIGH);
   digitalWrite(pin_led_F, LOW);
   digitalWrite(pin_led_G, LOW);
 }
 if (valore_A0 >= 190 && valore_A0 < 220)
 {
   digitalWrite(pin_led_A, HIGH);
   digitalWrite(pin_led_B, HIGH);
   digitalWrite(pin_led_C, HIGH);
   digitalWrite(pin_led_D, HIGH);
   digitalWrite(pin_led_E, HIGH);
   digitalWrite(pin_led_F, HIGH);
   digitalWrite(pin_led_G, LOW);
 }
  //High sounds
  if (valore_A0 >= 270)
 {
    digitalWrite(pin_led_A, HIGH);
   digitalWrite(pin_led_B, HIGH);
   digitalWrite(pin_led_C, HIGH);
   digitalWrite(pin_led_D, HIGH);
   digitalWrite(pin_led_E, HIGH);
   digitalWrite(pin_led_F, HIGH);
   digitalWrite(pin_led_G, HIGH);
 }
 delay(50);

We decided to print a case with a 3D printer to build a little "home" the LEDs.

Below you'll find a video that show our completely working project. Hope you'll enjoy it!

Humour DJ in action!

Code

Server.pyPython
Python script
from flask import Flask
from flask_ask import Ask, statement,question
from flask import request
from mutagen.mp3 import MP3

app = Flask(__name__)
ask = Ask(app, '/')


import serial
import time  
import vlc   

player = vlc.MediaPlayer()
#counter needed to know if we are at the first or the second question
counter = 0

#data strcuture that stores the informations that will be used to calculate the resultant mood
state = {
  "Happy": 0, 
  "Neutral": 0,
  "Angry": 0, 
  "Sad" : 0
  }

#create the serial channel for the communication wih Arduino  
Arduino = serial.Serial("/dev/ttyACM0",9600)


def calculateState( mood ):
    global counter
    global state
   
    counter += 1
    if counter == 1:
        state[mood]= 1
        return question("How was your day?")
    elif counter == 2:
        counter = 0
        state[mood]= 2
        return statement("Trying to calculate your state...")

#here we intercept the event fired from Alexa when someone trigger our custom skill
@ask.launch
def handler():
  #we use global variable in order to change values for every events and functions
    global counter
    global player
    #here we check if another song is currently playing
    if player.is_playing():
        return statement("Humour dj is already playing a song, wait!")
    #if not, we send the "Welcome message" and we ask the first question
    else:
        return question("Hi! Welcome to the Alexa Humour DJ ! I can play a song for you based on your current mood. First of all, how are you?")

#here we handle the classified response for the four moods
@ask.intent('Happy')
def happy():
    global counter
    global state
    counter += 1
    #if we have asked only the first question, counter==1 and we ask the second
    if counter == 1:
        state["Happy"]+= 1
        return question("And now, tell me, how was your day?")
    #else we go and calculate the mood with the chooseSong() function
    elif counter == 2:
        counter = 0
        state["Happy"]+= 1
	#decisione della canzone in base allo stato d'animo
        chooseSong()
        
        return statement("This is the song  i chose for you, i hope you like it!")
        
        
        
#same as above, for every mood    
@ask.intent('Neutral')
def neutral():
    global counter
    global state
    counter += 1
    if counter == 1:
        state["Neutral"]+= 1
        return question("And now, tell me, how was your day?")
    elif counter == 2:
        counter = 0
        state["Neutral"]+= 1

        chooseSong()
        
        return statement("This is the song  i chose for you, i hope you like it!")

@ask.intent('Angry')
def angry():
    global counter
    global state
    counter += 1
    if counter == 1:
        state["Angry"]+= 1
        return question("And now, tell me, how was your day?")
    elif counter == 2:
        counter = 0
        state["Angry"]+= 1
        chooseSong()
        
        return statement("This is the song  i chose for you, i hope you like it!")

@ask.intent('Sad')
def sad():
    global counter
    global state
    global player 
    
    counter += 1
    if counter == 1:
        state["Sad"]+= 1
        return question("And now, tell me, how was your day?")
    elif counter == 2:
        counter = 0
        state["Sad"]+= 1
        chooseSong()
        return statement("This is the song  i chose for you, i hope you like it!")
        
        
#here is the function that we use to calculate the mood, launch the related song and send a signal to Ardunino
def chooseSong():
    global ser
    global player 
    #if Happy
    if state["Happy"] == 2 or ( state["Happy"] ==1 and state["Angry"]==1  ) or (state["Happy"] ==1 and state["Sad"]==1 ):
        player = vlc.MediaPlayer("/home/pi/Desktop/Happy/PharrellWilliams-Happy.mp3")
        player.play()
        Arduino.write(b'H')
      
    #if Neutral    
    if state["Neutral"] == 2 or (state["Happy"] ==1 and state["Neutral"]==1  ):
        player = vlc.MediaPlayer("/home/pi/Desktop/Neutral/Riders_on_the_Storm.mp3")
        player.play()
        Arduino.write(b'N')
        
    #if Angry
    if state["Angry"] == 2 or (state["Angry"]==1 and state["Sad"]==1  ) or (state["Angry"]==1 and state["Neutral"]==1 ):
        player = vlc.MediaPlayer("/home/pi/Desktop/Angry/Enter_Sandman.mp3")
        player.play()
        Arduino.write(b'A')
        
    #if Sad
    if state["Sad"] == 2 or (state["Sad"]==1 and state["Neutral"]==1 ):
        player = vlc.MediaPlayer("/home/pi/Desktop/Sad/Wish_You_Were_Here.mp3")
        player.play()
        Arduino.write(b'S')
    state = {"Happy": 0, "Neutral": 0,"Angry": 0, "Sad" : 0}
    
    return;

#the server listens on localhost:5000
if __name__ == '__main__':
    app.run(host='localhost', port = 5000, debug = True)
ArduinoSketchArduino
#include <ChainableLED.h>
ChainableLED leds1(2,3,1);
// Led and sensors
int pin_A0 = A5;
int pin_led_A = 5;
int pin_led_B = 6;
int pin_led_C = 7;
int pin_led_D = 8;
int pin_led_E = 9;
int pin_led_F = 10;
int pin_led_G = 11;
int pin_TouchPin=4;

//Variables
char mood=0;
int TouchValue = 0; 
int valore_A0 = 0;
int on = 1;

void setup()
{
  Serial.begin(9600);
  //Inputs
  pinMode(pin_A0, INPUT);
  pinMode(pin_TouchPin, INPUT); 
  //Outputs
  pinMode(pin_led_A, OUTPUT);
  pinMode(pin_led_B, OUTPUT);
  pinMode(pin_led_C, OUTPUT);
  pinMode(pin_led_D, OUTPUT);
  pinMode(pin_led_E, OUTPUT);
  pinMode(pin_led_F, OUTPUT);
  pinMode(pin_led_G, OUTPUT);
  leds1.init();
  
  
}
void loop() {
  // put your main code here, to run repeatedly:

  if ( on == 1 ) {
  valore_A0 = analogRead(pin_A0);
  }
 
  
 
  //Low sounds
  if(valore_A0 <= 168){
    digitalWrite(pin_led_A, LOW);
    digitalWrite(pin_led_B, LOW);
    digitalWrite(pin_led_C, LOW);
    digitalWrite(pin_led_D, LOW);
    digitalWrite(pin_led_E, LOW);
    digitalWrite(pin_led_F, LOW);
    digitalWrite(pin_led_G, LOW);
    
  }
  if (valore_A0 > 168 && valore_A0 < 171)
  {
    digitalWrite(pin_led_A, HIGH);
    digitalWrite(pin_led_B, LOW);
    digitalWrite(pin_led_C, LOW);
    digitalWrite(pin_led_D, LOW);
    digitalWrite(pin_led_E, LOW);
    digitalWrite(pin_led_F, LOW);
    digitalWrite(pin_led_G, LOW);
  }
  //Middle sounds
  if (valore_A0 >= 171 && valore_A0 < 174)
  {
    digitalWrite(pin_led_A, HIGH);
    digitalWrite(pin_led_B, HIGH);
    digitalWrite(pin_led_C, LOW);
    digitalWrite(pin_led_D, LOW);
    digitalWrite(pin_led_E, LOW);
    digitalWrite(pin_led_F, LOW);
    digitalWrite(pin_led_G, LOW);
  }
 
  if (valore_A0 >= 174 && valore_A0 <176)
  {
    digitalWrite(pin_led_A, HIGH);
    digitalWrite(pin_led_B, HIGH);
    digitalWrite(pin_led_C, HIGH);
    digitalWrite(pin_led_D, LOW);
    digitalWrite(pin_led_E, LOW);
    digitalWrite(pin_led_F, LOW);
    digitalWrite(pin_led_G, LOW);
  }
  if (valore_A0 >= 176 && valore_A0 < 180)
  {
    digitalWrite(pin_led_A, HIGH);
    digitalWrite(pin_led_B, HIGH);
    digitalWrite(pin_led_C, HIGH);
    digitalWrite(pin_led_D, HIGH);
    digitalWrite(pin_led_E, LOW);
    digitalWrite(pin_led_F, LOW);
    digitalWrite(pin_led_G, LOW);
  }
  if (valore_A0 >= 180 && valore_A0 < 190)
  {
    digitalWrite(pin_led_A, HIGH);
    digitalWrite(pin_led_B, HIGH);
    digitalWrite(pin_led_C, HIGH);
    digitalWrite(pin_led_D, HIGH);
    digitalWrite(pin_led_E, HIGH);
    digitalWrite(pin_led_F, LOW);
    digitalWrite(pin_led_G, LOW);
  }
  //High sounds
  if (valore_A0 >= 190 && valore_A0 < 220)
  {
    digitalWrite(pin_led_A, HIGH);
    digitalWrite(pin_led_B, HIGH);
    digitalWrite(pin_led_C, HIGH);
    digitalWrite(pin_led_D, HIGH);
    digitalWrite(pin_led_E, HIGH);
    digitalWrite(pin_led_F, HIGH);
    digitalWrite(pin_led_G, LOW);
  }
   
   if (valore_A0 >= 270)
  {
     digitalWrite(pin_led_A, HIGH);
    digitalWrite(pin_led_B, HIGH);
    digitalWrite(pin_led_C, HIGH);
    digitalWrite(pin_led_D, HIGH);
    digitalWrite(pin_led_E, HIGH);
    digitalWrite(pin_led_F, HIGH);
    digitalWrite(pin_led_G, HIGH);
  }
  delay(50);


tastiera=Serial.read();  
 switch (mood){
  case 'A': //angry
  {
 leds1.setColorRGB(0,255,0,0);
  break;
  }

  case 'S': //sadness
  {
  leds1.setColorRGB(0,0,0,255);
  break;
  }

case 'N': //neutral
{
  leds1.setColorRGB(0,255,255,255);
  break;
}

  case 'H': //happiness
  {
 leds1.setColorRGB(0,0,255,0);
 break;
  }

 default:
 Serial.flush();
 }

//"Turn off" Arduino
TouchValue=digitalRead(pin_TouchPin);
  if (TouchValue == 1)
  {
    leds1.setColorRGB(0,0,0,0);
    mood='Z';
    valore_A0 = 0;
    on = 0;
    digitalWrite(pin_led_A, LOW);
    digitalWrite(pin_led_B, LOW);
    digitalWrite(pin_led_C, LOW);
    digitalWrite(pin_led_D, LOW);
    digitalWrite(pin_led_E, LOW);
    digitalWrite(pin_led_F, LOW);
    digitalWrite(pin_led_G, LOW);
    
    
  }



}

Schematics

HW Schema
Project schema
Schema tkdblyzi18
High-Level Schema
Schema2 rku7x6jpll

Comments

Similar projects you might like

Light Control Using Arduino and Amazon Alexa

Project tutorial by Rajesh

  • 15,326 views
  • 1 comment
  • 17 respects

Enable Alexa Control to your Ceiling Fan

Project tutorial by Jithin Thulase

  • 3,124 views
  • 5 comments
  • 9 respects

Eye Lock

Project tutorial by Team Nsci

  • 1,953 views
  • 0 comments
  • 17 respects

Animated Smart Light with Alexa and Arduino

Project tutorial by Bruno Portaluri

  • 3,671 views
  • 9 comments
  • 23 respects

Alexa Controlled LEDs Through Raspberry Pi

Project tutorial by vincent wong

  • 5,271 views
  • 5 comments
  • 23 respects

Anna: Alexa Powered Interactive Light

Project tutorial by vincent wong

  • 1,815 views
  • 0 comments
  • 7 respects
Add projectSign up / Login