Project in progress

Walaarm: Walabot-Powered Robotic Arm © GPL3+

A robotic arm which can pick an object from any random location.

  • 4,593 views
  • 0 comments
  • 33 respects

Components and supplies

Apps and online services

About this project

Introduction

Walaarm is a robotic arm powered by Walabot. Walaarm can be controlled by voice command using Amazon Alexa. It can pick an object automatically from any location within its range, Walabot helps it to find the location of the object. Walabot is working as an eye for the arm.

How it works

The logic behind it is very simple. A Raspberry Pi is working as the main controlling unit for the robotic arm. A Walabot is connected to the raspberry pi which calculate the position of any object within its range and sent the location information to Raspberry Pi. An Arduino controlled robotic arm is also connected to raspberry pi through USB port.

A custom Alexa skill was developed to control the arm. When user ask Alexa like "Alexa, ask Walaarm to pick the object," Alexa sends a MQTT message to raspberry pi. Raspberry pi then ask walabot for the location of the object and after getting the location information it sends the location to Arduino using serial protocol. Arduino drives the motors of the arm accordingly to pick the object from the specific location.

The Alexa skill 'Walaarm' is live at the store (Skill ID: amzn1.ask.skill.3afea6bd-1769-4dfb-83d8-fd3cccb76a05)

Detailed Instructions

For the demonstration I used an open source 3D-printed small size robotic arm designed by bentommye. The arm is very small and inexpensive and only required five 9g micro servo. The design is available here.

As this is a very small size robotic arm and not very flexible I wrote an Arduino sketch to move the arm in four specific location only. So the arm can pick an object from four fixed locations and each location can be identified by Walabot. This is a proof of concept project and practically an object from any random location within the range can be picked using a more flexible and accurate robotic arm because Walabot can locate an object very precisely within it's range.

For interfacing Walabot with Raspberry Pi, Walabot official SDK for Raspberry Pi was used. The program was developed using Python. The complete Python program is attached in the code section. For receiving MQTT message paho-mqtt client module for Python was used. Python serial module was used to transmit data to Arduino Nano board.

JSON file for the Alexa skill is also attached to the code section. I am assuming you have some previous experience on developing custom Alexa skill. If not read the tutorial https://developer.amazon.com/alexa-skills-kit/alexa-skill-quick-start-tutorial.

How to use the Alexa Skill

If you don't like to build your own skill you can used my published skill Walaarm. Just search for walaarm or walabot from Alexa skills store and enable the skill. No account linking will be required.

After enabling the skill, launch the skill by asking

Alexa, open wala arm.

Then ask Alexa for help by the word

help

Alexa will speak a unique topic to which you should subscribe to receive command from AWS Lambda. Note down the topic and use it in the python code for Raspberry Pi provided in code section. The topic will start as:

WALABOT/ROBOT/ARM/..............

First three word is common for all but after that Alexa will spell 10 characters from your unique user id which will be unique for you. Unique topic mean that only you will receive your command.

For the connection of the motors with Arduino please see the schematic.

Code

lambda_functionPython
Python code for AWS Lambda
"""
This sample demonstrates a simple skill built with the Amazon Alexa Skills Kit.
The Intent Schema, and Sample Utterances for this skill, as well
as testing instructions are located at http://amzn.to/1LzFrj6
 
The code is developed by:
	Md. Khairul Alam
	February, 2018
For additional samples, visit the Alexa Skills Kit Getting Started guide at
http://amzn.to/1LGWsLG
"""
 
from __future__ import print_function
import urllib2
import xml.etree.ElementTree as etree
from datetime import datetime as dt
import time
import paho.mqtt.client as mqtt

received_message = ""
user = ""
mqtt_topic = ""
# Define event callbacks
def on_connect(client, userdata, flags, rc):
    print("rc: " + str(rc))

def on_message(client, obj, msg):
    received_message = str(msg.payload)
    print(msg.topic + " " + str(msg.qos) + " " + received_message)


def on_subscribe(client, obj, mid, granted_qos):
    print("Subscribed: " + str(mid) + " " + str(granted_qos))
	
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.on_subscribe = on_subscribe

#client.username_pw_set("cnpemclt", "ccZRVtdCTqID")
client.connect("iot.eclipse.org", "1883")

#client.connect("m12.cloudmqtt.com", 15708, 60)

# Start subscribe, with QoS level 0
client.subscribe("test/robot", 0)



def lambda_handler(event, context):
    """ Route the incoming request based on type (LaunchRequest, IntentRequest,
    etc.) The JSON body of the request is provided in the event parameter.
    """
    print("event.session.application.applicationId=" +
          event['session']['application']['applicationId'])
    print("event.session.application.userId=" +
          event['session']['user']['userId'])      
    #char = event['session']['user']['userId']
    #first 10 char from used id will be used to generate an unique topic
    #common part of the user id is deducted
    print(event['session']['user']['userId'][18:28])
    global user
    global mqtt_topic
    user = event['session']['user']['userId'][18:28]
    topic = "WALABOT/ROBOT/ARM/"
    mqtt_topic = topic + user
    print(mqtt_topic)
    """
    Uncomment this if statement and populate with your skill's application ID to
    prevent someone else from configuring a skill that sends requests to this
    function.
    """
    # if (event['session']['application']['applicationId'] !=
    #         "amzn1.echo-sdk-ams.app.[unique-value-here]"):
    #     raise ValueError("Invalid Application ID")
 
    if event['session']['new']:
        on_session_started({'requestId': event['request']['requestId']},
                           event['session'])
 
    if event['request']['type'] == "LaunchRequest":
        return on_launch(event['request'], event['session'])
    elif event['request']['type'] == "IntentRequest":
        return on_intent(event['request'], event['session'])
    elif event['request']['type'] == "SessionEndedRequest":
        return on_session_ended(event['request'], event['session'])
 
 
def on_session_started(session_started_request, session):
    """ Called when the session starts """
 
    print("on_session_started requestId=" + session_started_request['requestId']
          + ", sessionId=" + session['sessionId'])
 
 
def on_launch(launch_request, session):
    """ Called when the user launches the skill without specifying what they
    want
    """
 
    print("on_launch requestId=" + launch_request['requestId'] +
          ", sessionId=" + session['sessionId'])
    # Dispatch to your skill's launch
    return get_welcome_response()
 
 
def on_intent(intent_request, session):
    """ Called when the user specifies an intent for this skill """
 
    print("on_intent requestId=" + intent_request['requestId'] +
          ", sessionId=" + session['sessionId'])
 
    intent = intent_request['intent']
    intent_name = intent_request['intent']['name']
 
    # Dispatch to your skill's intent handlers
    if intent_name == "PickObjectIntent":
        return pick_object(intent, session)
    elif intent_name == "AMAZON.HelpIntent":
        return get_help()
    elif intent_name == "AMAZON.StopIntent" or intent_name == "AMAZON.CancelIntent":
        return session_end(intent, session)
    else:
        raise ValueError("Invalid intent")

 
def on_session_ended(session_ended_request, session):
    """ Called when the user ends the session.
 
    Is not called when the skill returns should_end_session=true
    """
    print("on_session_ended requestId=" + session_ended_request['requestId'] +
          ", sessionId=" + session['sessionId'])
    # add cleanup logic here
 
# --------------- Functions that control the skill's behavior ------------------
 
 
def get_welcome_response():
    """ If we wanted to initialize the session to have some attributes we could
    add those here
    """
 
    session_attributes = {}
    card_title = "Welcome"
    speech_output = "Welcome to the wala-arm application. I can pick an object from any random location " \
                    "by the help of Wala-bot. To know how to configure Wala-bot just say help, "\
                    "or to pick an object you can ask me, 'pick the object.'" 
                
    # If the user either does not reply to the welcome message or says something
    # that is not understood, they will be prompted again with this text.
    reprompt_text = "To pick an object just ask me, pick the object." 
                  
    should_end_session = False
    #myMQTTClient.publish("test/door", "welcome", 0)
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))
		
def get_help():
    """ If we wanted to initialize the session to have some attributes we could
    add those here
    """
    global user
    temp = user
    count = 0
    while count<20:
        temp = temp[:count] + ' ' + temp[count:] #added a space after every character
        count+=2
    session_attributes = {}
    card_title = "Help"
    speech_output = "I am sending an MQTT message to the Raspberry Pi, connected with Wala-bot. " \
                    " Use wala-bot/ robot/ arm/ " + temp + ", as your MQTT topic. "\
                    " I am spelling the last word again, " + temp + " . Use block letters for the topic name."\
                    " For details please visit hackster.io and search for Wala-arm." \
                    
  
    # If the user either does not reply to the welcome message or says something
    # that is not understood, they will be prompted again with this text.
    reprompt_text = "I am waiting for your command."
    should_end_session = False
    #myMQTTClient.publish("test/door", "welcome", 0)
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))

def pick_object(intent, session):
    """ If we wanted to initialize the session to have some attributes we could
    add those here
    """
    #client.publish("walabot/robot/arm/...", "pick")
    global mqtt_topic
    client.publish(mqtt_topic, "pick")
    time.sleep(2)
    session_attributes = {}
    card_title = "Picking Object"
    speech_output = "OK."
    # If the user either does not reply to the welcome message or says something
    # that is not understood, they will be prompted again with this text.
    reprompt_text = ""
    should_end_session = True
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))
 

def session_end(intent, session):
    """ If we wanted to initialize the session to have some attributes we could
    add those here
    """
    session_attributes = {}
    card_title = "End"
    speech_output = "Thank you for calling me. Have a nice day!"
    
    # If the user either does not reply to the welcome message or says something
    # that is not understood, they will be prompted again with this text.
    reprompt_text = ""
    should_end_session = True
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))        
 
# --------------- Helpers that build all of the responses ----------------------
 
 
def build_speechlet_response(title, output, reprompt_text, should_end_session):
    return {
        'outputSpeech': {
            'type': 'PlainText',
            'text': output
        },
        'card': {
            'type': 'Simple',
            'title': title,
            'content': output
        },
        'reprompt': {
            'outputSpeech': {
                'type': 'PlainText',
                'text': reprompt_text
            }
        },
        'shouldEndSession': should_end_session
    }
 
 
def build_response(session_attributes, speechlet_response):
    return {
        'version': '1.0',
        'sessionAttributes': session_attributes,
        'response': speechlet_response
    }
walaarmPython
Code for raspberry pi
from __future__ import print_function
from sys import platform
from os import system
import time
import serial
from imp import load_source
import paho.mqtt.client as mqtt

#ser = serial.Serial('/dev/ttyACM0',9600)
ser = serial.Serial('/dev/ttyAMA0',9600)

received_message = ""

# Define event callbacks
def on_connect(client, userdata, flags, rc):
    print("rc: " + str(rc))

def on_message(client, obj, msg):
    received_message = str(msg.payload)
    print(received_message)
    if received_message == 'pick':
        WalabotAPI.Start()
        WalabotAPI.StartCalibration()
        WalabotAPI.Trigger()  # initiates a scan and records signals
        targets = WalabotAPI.GetImagingTargets() # provides a list of identified targetsc
        system('cls' if platform == 'win32' else 'clear')  # clear the terminal
        for i, t in enumerate(targets):
            print('Target #{}\nx = {}\ny = {}\nz = {}\n'.format(
                i+1, t.xPosCm, t.yPosCm, t.zPosCm))
            if t.yPosCm <-1 and t.zPosCm <1: #i am taking three random position
                print('pos1')
                position = 'pos1'
            elif t.yPosCm >1 and t.zPosCm >1:
                print('pos2')
                position = 'pos2'
            elif t.yPosCm >3 and t.zPosCm >2:
                print('pos3')
                position = 'pos3'
            ser.write(position)
            ser.write('\n')
        WalabotAPI.Stop()  # stops Walabot when finished scanning
            
        

def on_subscribe(client, obj, mid, granted_qos):
    print("Subscribed: " + str(mid) + " " + str(granted_qos))
	
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.on_subscribe = on_subscribe

client.connect("iot.eclipse.org", "1883")
client.subscribe("WALABOT/ROBOT/ARM/XXXXXXXXXX", 0)
#replace XXXXXXXXXX by the character alexa spells from user id
#run the Wala-arm skill and ask for help to get the character sequence
#this 10 character will be unique for every user
WalabotAPI = load_source('WalabotAPI',
 '/usr/share/walabot/python/WalabotAPI.py')
 
R_MIN, R_MAX, R_RES = 2, 50, 5  # SetArenaR values
THETA_MIN, THETA_MAX, THETA_RES = -25, 5, 2  # SetArenaTheta values
PHI_MIN, PHI_MAX, PHI_RES = -75, 75, 2  # SetArenaPhi values
TSHLD = 5  # SetThreshold value
VELOCITY_THRESHOLD = 0.7  # captured vel bigger than that counts as key-press

WalabotAPI.Init()  # load the WalabotSDK to the Python wrapper
WalabotAPI.SetSettingsFolder()
WalabotAPI.ConnectAny()  # establishes communication with the Walabot
WalabotAPI.SetProfile(WalabotAPI.PROF_SENSOR)
WalabotAPI.SetArenaR(R_MIN, R_MAX, R_RES)
WalabotAPI.SetArenaTheta(THETA_MIN, THETA_MAX, THETA_RES)
WalabotAPI.SetArenaPhi(PHI_MIN, PHI_MAX, PHI_RES)
WalabotAPI.SetThreshold(TSHLD)
WalabotAPI.SetDynamicImageFilter(WalabotAPI.FILTER_TYPE_MTI)
#WalabotAPI.Start()
#WalabotAPI.StartCalibration()

#client.loop_forever()
rc = 0
while rc == 0:
    rc = client.loop()
print("rc: " + str(rc))

#WalabotAPI.Stop()  # stops Walabot when finished scanning
#WalabotAPI.Disconnect() # stops communication with Walabot
Code for ArduinoArduino
This sketch if for the Arduino connected with the Robotic Arm
/*************************************
 * Author: Md. Khairul Alam
 * This code is for controlling a robotic arm containing 5 servos
 */

#include <Servo.h>

Servo servo1, servo2, servo3, servo4, servo5;
String inputString = "";         // a string to hold incoming data
boolean stringComplete = false;  // whether the string is complete

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  servo1.attach(2);
  servo2.attach(3);
  servo3.attach(4);
  servo4.attach(5);
  servo5.attach(6);
  inputString.reserve(200);
  nutral();
  unhold(); 
}

void loop() {
  
  if (stringComplete) {
    if(inputString == "pos1"){//if received message = pos1
        pick_pos1(); //pick the object from location 1
        delay(20);
      }
    else if(inputString == "pos2"){
        pick_pos2();
        delay(20);
      } 
    else if(inputString == "pos3"){
        pick_pos3();
        delay(20);
      }  
    // clear the string:
    inputString = "";
    stringComplete = false;
  }

}

void servo_rotate(int num, int value){
  switch(num){
    case 1:
    servo1.write(value);
    break;

    case 2:
    servo2.write(value);
    break;

    case 3:
    servo3.write(value);
    break;

    case 4:
    servo4.write(value);
    break;

    case 5:
    servo5.write(value);
    break;
    }
 }
 
//this function grip an object
void hold(){  
  for(int i=180; i>110; i--)
    {
      servo_rotate(1, i);
      delay(15);
      }  
  }
  
//this function unhold an object
void unhold(){
  for(int i=95; i<180; i++)
    {
      servo_rotate(1, i);
      delay(15);
      }
  }
  
//this function drive servo 5 to set x position in angle
void set_xpos(int pos){
  int previous_pos = servo5.read();
  if(pos>previous_pos){
    for(int i=previous_pos; i<pos; i+=2){
      servo_rotate(5, i);
      delay(25);
      }
    }
  if(pos<previous_pos){
    for(int i=previous_pos; i>pos; i-=2){
      servo_rotate(5, i);
      delay(25);
      }
    } 
 }
//set sevo 3 position
void set_3pos(int pos){
  int previous_pos = servo3.read();
  if(pos>previous_pos){
    for(int i=previous_pos; i<pos; i+=2){
      servo_rotate(3, i);
      delay(25);
      }
    }
  if(pos<previous_pos){
    for(int i=previous_pos; i>pos; i-=2){
      servo_rotate(3, i);
      delay(25);
      }
    } 
 }
//set sevo 2 position
void set_2pos(int pos){
  int previous_pos = servo2.read();
  if(pos>previous_pos){
    for(int i=previous_pos; i<pos; i+=2){
      servo_rotate(2, i);
      delay(25);
      }
    }
  if(pos<previous_pos){
    for(int i=previous_pos; i>pos; i-=2){
      servo_rotate(2, i);
      delay(25);
      }
    } 
 }
//set sevo 4 position
void set_4pos(int pos){
  int previous_pos = servo4.read();
  if(pos>previous_pos){
    for(int i=previous_pos; i<pos; i+=2){
      servo_rotate(4, i);
      delay(25);
      }
    }
  if(pos<previous_pos){
    for(int i=previous_pos; i>pos; i-=2){
      servo_rotate(4, i);
      delay(25);
      }
    } 
 }

//go to middle point
void nutral(){
  set_2pos(90);
  delay(300);
  set_3pos(60);
  delay(300);
  set_4pos(90);
  delay(300); 
  set_xpos(90); 
  delay(300);   
}
//pick an object from position 1
void pick_pos1(){
  delay(1000);
  set_xpos(140);
  delay(1000);
  set_3pos(80);
  delay(1000);
  set_4pos(140);  
  delay(1000);
  set_2pos(90);
  delay(1000);
  hold();
  delay(1000);
  set_3pos(60);
  delay(1000);
  delay(300);
  set_2pos(90);
  delay(300);
  set_4pos(90);
  delay(300);
  set_xpos(10);
  delay(300);
  set_3pos(100);
  delay(300);
  set_4pos(130);
  delay(300);
  unhold();
  delay(300);
  nutral(); 
}
//pick an object from position 2
void pick_pos2(){
  delay(300);
  set_xpos(120);
  delay(300);
  set_3pos(105);
  delay(300);
  set_2pos(140);  
  delay(300);
  set_4pos(100);
  delay(300);
  set_2pos(160); 
  delay(300);
  hold();
  delay(500);
  set_2pos(90);
  delay(500);
  set_3pos(80);
  delay(500);
  set_3pos(60);
  delay(500);
  set_4pos(90);
  delay(300);
  set_xpos(10);
  delay(300);
  set_3pos(100);
  delay(300);
  set_4pos(130);
  delay(300);
  unhold();
  delay(300);
  nutral(); 
 }
//pick an object from position 3
void pick_pos3(){
  delay(300);
  set_xpos(70);
  delay(300);
  set_3pos(110);
  delay(300);
  set_4pos(120);
  delay(300);
  set_2pos(100); 
  delay(300);
  hold();
  delay(500);
  set_2pos(90);
  delay(500);
  set_3pos(80);
  delay(500);
  set_3pos(60);
  delay(500);
  set_4pos(90);
  delay(300);
  set_xpos(10);
  delay(300);
  set_3pos(100);
  delay(300);
  set_4pos(130);
  delay(300);
  unhold();
  delay(300);
  nutral(); 
}

void serialEvent() {
  while (Serial.available()) {    
    // get the new byte:
    char inChar = (char)Serial.read();     
    // if the incoming character is a newline, set a flag
    // so the main loop can do something about it:
    if (inChar == '\n') {
      stringComplete = true;
    }
    else
    // add it to the inputString:  
      inputString += inChar;
  }
}
Alexa skillJSON
JSON code for the skill Walaarm
{
    "interactionModel": {
        "languageModel": {
            "invocationName": "wala arm",
            "intents": [
                {
                    "name": "AMAZON.CancelIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.HelpIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.StopIntent",
                    "samples": []
                },
                {
                    "name": "PickObjectIntent",
                    "slots": [],
                    "samples": [
                        "hold the thing",
                        "hold the object",
                        "pick thing",
                        "pick boject",
                        "pick the thing",
                        "pick the object"
                    ]
                }
            ],
            "types": []
        }
    }
}

Schematics

Block diagram of the System
Walaarm a93aaovguk
Schematic
The arm contains 5 micro servo motor. The motors are connected sequentially from the top (gripper) to bottom.
Arm schematic wct0kns5ru

Comments

Similar projects you might like

PC Controlled Robotic Arm

Project tutorial by AhmedAzouz

  • 7,422 views
  • 10 comments
  • 64 respects

Intelligent Door Lock

Project in progress by Md. Khairul Alam

  • 14,020 views
  • 19 comments
  • 86 respects

Chatbot controlled Robotic Arm

Project tutorial by Keval Doshi

  • 2,815 views
  • 0 comments
  • 22 respects

Home Monitoring And Alerts For The Blind

Project in progress by Team Penteon

  • 2,467 views
  • 0 comments
  • 17 respects

Walabot Security Robot with Alexa Command and Control

Project in progress by Tom Minnich

  • 1,661 views
  • 0 comments
  • 3 respects

WalaBeer Tank

Project tutorial by Balázs Simon

  • 22,089 views
  • 4 comments
  • 147 respects
Add projectSign up / Login