Project tutorial
Voice-Controlled Robot - Google Cloud Services

Voice-Controlled Robot - Google Cloud Services © Apache-2.0

Build a voice-controlled Arduino robot. You will use your own Android phone (powered by Google Cloud Services) to control it. Lots of fun!

  • 10,957 views
  • 4 comments
  • 33 respects

Components and supplies

Apps and online services

About this project

A couple of weeks ago I published a tutorial explaining how to build an Arduino robot controlled by Alexa Echo. I got thousands of views, tens of respects so thanks for this! For those who haven’t seen it yet, here’s the link: https://www.hackster.io/lukaszbudnik/voice-controlled-robot-ebbacf

Alexa Echo is not as popular as for example Android phone and it was a little bit difficult to build replicas. Also, as I was already using Android for bluetooth and cloud - phone communication, I thought about switching completely to Google so that it would be easier to build a complete solution/replica.

The Control Flow

Here is how all the pieces are working together:

  • Launch Android application
  • Sign in into your Google account
  • Device will store phone’s settings in Firebase Database
  • You talk to Android phone
  • Android application:

5.1 Records your voice

5.2 Uploads audio file to Google Storage

5.3 Invokes node.js application (hosted on Google App Engine) which will stream the recorded audio directly to Google Cloud Speech API

  • Node.js application parses transcripts returned by Google Cloud Speech API and translates them to proper digital messages. Messages are sent to Google Cloud Messaging service
  • Android application receives an incoming remote message and relays it over Bluetooth to Arduino.
  • Arduino using HC-06 bluetooth serial module is reading the messages and moving the robot accordingly

Simple :)

Google Cloud Services

In this section I will show you how I extended my original application and integrated it with a lot of additional Google Services. The changes include:

  • Google Authentication
  • Google Firebase Database - for storing phone’s settings. For example in the previous Alexa Skill we had to copy phone’s Firebase registration token and provide it to Robot skill. This is now stored in Firebase Database and handled automatically by the application.
  • Google Cloud Speech API - for voice recognition
  • Google Firebase Storage - for storing audio recorded by Android
  • Google App Engine - hosts new node.js application which fetches phone’s settings from Firebase Database and then streams audio from Firebase Storage to Cloud Speech API.

The application is complete. No changes are required. You only have to download google-services.json from Google web console and you’re done (just like in the Alexa Skill version). Watch the video where I talk a little bit more about Google Services.

Google Cloud Speech API

In this section you will learn how to:

  • use Google Cloud Speech API node.js client
  • how to set it up for streaming
  • how to use different languages and improve accuracy of voice recognition
  • finally how to stream audio and respond to voice recognition events

Google Cloud Speech API service is in Beta and its Node.js client is in Alpha stage. Keep that in mind that some things may change.

Still in Beta but the completeness (68 languages supported out of the box) and quality (while playing around with it had only 1 error when streaming from Firebase Storage to Cloud Speech API I got an error complaining about too fast streaming and asking to slow down) is pretty impressive. Watch the video for more details.

Implementing new language support

Google Cloud Speech API (while still in beta) already provides support for 68 languages. Watch this video to learn how to add a new language support to the node.js application.

Arduino Robot

Arduino is exactly the same as in the Alexa Robot Skill version. No changes at all. Below is exactly the same video I used in my previous tutorial.

Show time

Here is my robot in action:

Summary

My Robot understands both English and Polish. By now you should be able to add support for any other language (supported by Google Speech API of course).

The whole solution is open-source and available on github (see code section). I’m open to all pull requests so if you want to contribute, fire one my way.

Code

Speech Recognition App
A node.js application running on Google App Engine using Google Cloud Speech API
Arduino code to move the robot
My Arduino library. It contains the main ino files together with a set of very useful libraries which I used in my project.
Android application for voice recording and bluetooth communication
Application records your voice and invokes Speech Recognition application. Recognised commands and recieved asynchronously from Google Cloud Messaging. Android phone receives such messages and relays them to Arduino. Communication between Android and Arduino is done over bluetooth.

Schematics

Arduino & Robot circuits
Arduino and Robot wiring. The schematics does not show HC-06 device. For a better schematics please see a twin project which is using IR controls available at circuits.io: https://circuits.io/circuits/3591193-ir-robot-control. Please also watch the video which describes the robot design in a little bit more detail.
Arduino circuit gnw6lrehmm

Comments

Similar projects you might like

Voice-Controlled Robot

Project tutorial by Łukasz Budnik

  • 14,588 views
  • 2 comments
  • 39 respects

Voice-Controlled Pick-and-Place Robot

by Sagar pawar

  • 2,126 views
  • 0 comments
  • 8 respects

Line Follower Robot - PID Control - Android Setup

Project tutorial by MJRoBot

  • 28,042 views
  • 7 comments
  • 33 respects

2WD Voice Controlled Robot with Arduino and BitVoicer Server

Project tutorial by Marcio Tamagushi

  • 10,208 views
  • 11 comments
  • 32 respects

Line Follower Robot Using Arduino & 1Sheeld

Project tutorial by Ashraf Nabil

  • 2,161 views
  • 0 comments
  • 5 respects
Add projectSign up / Login