Project in progress
Arduino WiFi Robot Controller

Arduino WiFi Robot Controller © GPL3+

Control any WiFi robot or WiFi sensor with an Arduino WiFi controller, with single protocol to rule them all!

  • 266 views
  • 0 comments
  • 0 respects

Components and supplies

Necessary tools and machines

09507 01
Soldering iron (generic)

Apps and online services

About this project

In the Arduino world, there are many projects showing how to control a robot arm or a robot car. Most of these projects are with a direct connected controller with a cable or Bluetooth dongle. Examples are XBOX or WII game controllers. There also is a generation of projects that are controlled through the internet. The Arduino runs a full webserver and can be controlled from a website or app.

I want to do this a little bit different; both the controller and the robot are build on an Arduino, and communication should be over WiFi. As they need to understand each other, an agreed protocol is needed. And I don’t want to limit myself to controlling a robot, I also want to be able to control for example a Hue light or read sensor values. Maybe also in combination with a Raspberry Pi or other platforms. (something about raising the bar high…)

I’ve reviewed many websites about (industrial) communication protocols. Although there is a lot of choices on protocols, not many of them are based on TCP or internet standards. There were 2 options (Modbus TCP and BACnet), but over all the years these exist, they are not completed for use on Arduino. Mainly because for Modbus a realtime master is difficult to setup and thus only a slave library (Mudbus) is available. And for BACnet there are only shields made to connect the Arduino, where I want to do this without shields!

So something new is needed… Lets find an Arduino based controller to work this out for.

Remote Controller: Arduino Esplora

That Remote Controller is an Arduino Esplora with a 1.8” TFT screen.

But the Esplora is based on the 2013 Arduino Leonardo board without Wifi, and it lacks the pins for connecting shields. Especially when the headers are used for the TFT screen.

No worries, I’ve got you covered! There are 2 TinkerKit output pins on the top, and that is exactly what is needed for connecting an ESP8266 board to do TX and RX for WiFi connection over AT commands. (thanks Mike Barela for the idea of using these!)

One problem left, is that the ESP8266 is 3.3v, and the Esplora is 5v. So I’ve used the ESP8266 12-E board, that contains level shifting logic back from 5v to 3.3v.

Actually, there is one more problem left… The ESP8266 communicates by default at a baud rate of 115000. The Esplora can talk at that speed, but cannot listen at a higher speed then 9600 baud without messing the data up. Luckily, the ESP board comes with an AT command to turn down the baud rate until next power cycle or reset. I'm using that to switch to 9600 baud, but also had to change the WiFiEsp library as that does an automatic reset after initialisation.

The Esplora is not made to run with a battery. It only has a 5v micro USB socket for input. But hey, there are the TinkerKit pins again! If you just supply 5v to these pins, it works perfect. But make very sure you have the polarity correct as there is no protection by connecting this way! I took an Adafruit PowerBoost 1000c to use for the LiPo battery and taking care of charging it. Added a switch to be able to turn the power on/off.

Robot controller: Arduino MKR WIFI 1010

Latest released Arduino board with an onboard WiFi chip is the MKR WIFI 1010.

Hardware part is completed with an Adafruit PWM/Servo board, an 1.3”OLED display and a MeArm that is controlled by 4 servo motors.

The MKR series of boards is 3.3v instead of the 5.5v in older iterations of Arduino boards. The PWM/Servo board can handle both 3.3v and 5v so we are good to go. To power the servo’s, we still need 5v. I used the 5v pin on the MKR to the V+ at the PW/Servo board, but while being connected to USB of my computer the board started rebooting and I lost USB connection. Probably because I didn't use the high power USB port, or the MKR is limiting output. I have to check the real reason. Solutions I can think of: use the high power USB ports, or power the servo’s with an external adapter (I took the last option at the moment).

Robot arm: MeArm

Meet the MeArm; designed by Benjamin Gray, with open source laser cut templates. I’ve ordered mine in 2015 just after the Kickstarter was funded successful and send out. Since then there were some more iterations of the MeArm featuring easier construction, you can buy the official MeArm here. They are now sold all over the world at cheap websites, and only some do proper attribution back to Benjamin.

The MeArm has 4 servo motors: Base, Elbow, Shoulder and Gripper. While playing with it, I burned some of the cheap plastic motors and put it aside. To get it working, I now changed some of the motors by metal gear motors.

To move the arm around, you can just change the PWM signal to the servo. But for the arm, I want to send coordinates. This is also known as Inverse Kinematics (IK), and there exists a terrific library for that (thanks Bob Stone!)

As the arm moves around a base, you could also use the Cylinder version of this.

Maybe you recognize the robot arm? Simone Giertz' "Toothbrush machine" is what made her famous as "Queen of Shitty Robots"!

The communication protocol

For setting up communication on a WiFi network, there are 2 main choices: TCP and UPD. What it comes down to is reliability (TCP) vs speed (UDP). Difference between TCP and UPD are very well described at this page.

Then why use UDP as it's not reliable? As UPD has less overhead, it’s better suitable for fast message delivery. Translating that to this project, to still let our source know that the message was received OK at the target, there will be send back an ACK to the source. And yes of course that ACK can go wrong too. To avoid this blocks our stream of messages, if the ACK did not come in after an interval of 30, we assume it lost. And then what… send the message again? Or just continue? I chose the last option, as it might be that the target received well, but the ACK did go wrong. But we need some safeguards to be build in.

Esplora sensors and output

The Esplora has a range of onboard sensors and output options. That is why this is such a great board. No need to search for components and after soldering together, ending up with something bulky. The Esplora is just 1 easy board and library.

As the library will be build around the Esplora, I took this as the base for the protocol.

SENSORS:

  • Pushbuttons (S1,S2,S3,S4)
  • Joystick (JX,JY)
  • Joystick push (S5)
  • Slider (SL)
  • Light (LU)
  • Microphone (MI)
  • Accellerator (AX,AY,AZ)
  • Thermometer (TC)

OUTPUT:

  • RGB (LR,LG,LB)
  • Spreaker (SP)
  • Screen (needs an advanced message)

First ideas on the protocol

The robot is always in WiFi Access Point (AP) mode. The Esplora Controller is in WiFi Device mode and scans for networks. Selected network SSID is reversed as password and connection is made. Once connection is created, request for identification is send as a WEB protocol. If this is indeed a robot that has this protocol logic, a welcome message is returned (HTML of XML page) that identifies the robot and says what the robot wants to receive (see list of sensors). SD card on Esplora has pre-configured set of images to show once robot is selected.

At the controller, a loop runs with time interval of 10 to send out sensor data. It then waits for ACK message (takes 2) and at 10 sends anew message. If no ACK is received, after waiting 30 it assumes the message was lost and continues with a new command. Command only contains the predefined message format.

At the robot, a loop runs where after waiting for 15, the current action ends (if this is for example a robot car driving forward, to avoid slamming into a wall). But that is up to the design of the robot developer. In case of the robot arm, there is no need to stop as we move the arm to coordinates and after that it stops moving.

Status of the project

DONE: Setting up the Arduino Esplora with the TFT screen and the ESP8266 and the PowerBoost. Tested the WiFi and internet capabilities, it works as a charm! Unsoldered the TinkerKit pins from the top and moved them to the bottom, facing inwards. With an acrylic baseplate below, I now have a very compact wireless controller.

DONE: Setting up the Arduino MKR WiFi 1010 with the PWM/Servo board and the MeArm. As a POC setup a webserver on the MKR and I'm able to control the MeArm by clicking links on the webinterface.

CURRENT: Working out the Protocol for UDP communications on both the Esplora and MKR, to control the Robot Arm.

TODO: making sure the protocol works on other Robots (a car?) and sensors (Raspberry Pi?)

The code uploaded is far from complete. It functions as a basic to show that I have something done already ;)

Will continue filling this project page with updates, pictures and videos in the coming weeks!

Code

Arduino Esplora controller codeArduino
This code is far from complete, it's just a start!!!
/*
  This code is far from complete, it's just a start!!!
  Code is partly based on Esplora Test from Mike Barela http://21stdigitalhome.blogspot.com/
*/

#include <Esplora.h>         // Arduino Esplora specific library
#include <TFT.h>             // Hardware-specific library
#include <SPI.h>             // SPI communications library


// Emulate Serial on pins 3/11 if not present
#include <SoftwareSerial.h>
SoftwareSerial mySerial(11,3); // RX, TX

// WiFiEsp-2.2.2 - Version: Latest 
#include <WiFiEsp.h>
//#include <WiFiEspClient.h>
#include <WiFiEspUdp.h>

char ssid[] = "Arduino_MeArm";            // your network SSID (name)
char pass[] = "mrAeM_oniudrA";        // your network password
int status = WL_IDLE_STATUS;     // the Wifi radio's status

//IPAddress remoteServer = 192.1.1.100;
unsigned int remotePort = 10002;

char packetBufferIn[255];          // buffer to hold incoming packet

const int UDP_TIMEOUT = 2000;    // timeout in miliseconds to wait for an UDP packet to arrive

byte packetBufferOut[NTP_PACKET_SIZE]; // buffer to hold incoming and outgoing packets

// A UDP instance to let us send and receive packets over UDP
WiFiEspUDP Udp;


/*
 Color definitions
*/
#define	COLOR_BLACK   0x0000 //0x0000
#define	COLOR_BLUE    0xF800 //0x001F
#define	COLOR_RED     0x001F //0xF800
#define	COLOR_GREEN   0x07E0 //0x07E0
#define COLOR_CYAN    0xFFE0 //0x07FF
#define COLOR_MAGENTA 0xF81F //0xF81F
#define COLOR_YELLOW  0x07FF //0xFFE0
#define COLOR_WHITE   0xFFFF //0xFFFF

/*
  This array holds the last sensed state of each of the buttons read.
  Later in the code, you'll read the button states, and compare them
  to the previous states that are stored in this array. If the two
  states are different, it means that the button was either pressed or released.
 */
boolean buttonStates[8];

/*
  This array holds the names of the buttons being read.
  Later in the sketch, you'll use these names with
  the method Esplora.readButton(x), where x is one of these buttons.
 */
const byte buttons[] = {
  JOYSTICK_DOWN,
  JOYSTICK_LEFT,
  JOYSTICK_UP,
  JOYSTICK_RIGHT,
  SWITCH_RIGHT, 
  SWITCH_LEFT, 
  SWITCH_UP, 
  SWITCH_DOWN, 
};




void setup() {
  Serial.begin(9600); // serial port used for debugging

  // Start the TFT Screen
  EsploraTFT.begin();  
  EsploraTFT.background(0,0,0);  // clear the screen with black
  EsploraTFT.setTextSize(2);
  displayString(0, 0,"Welcome to", COLOR_GREEN);
  displayString(0,16," Arduino", COLOR_GREEN); 
  displayString(0,32,"  Esplora", COLOR_GREEN);
  EsploraTFT.setTextSize(1);


  // initialize serial for ESP module
  mySerial.begin(115200);                         // default ESP's baud rate for AT commands
  mySerial.println("AT+UART_CUR=9600,8,1,0,0");   // update the baud rate to 9600
  mySerial.end();                                 // end the serial
  mySerial.begin(9600);                           // use the new ESP's baud rate
  WiFi.init(&mySerial);                           // initialize ESP module

  // check for the presence of the shield
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial.println("WiFi shield not present");
  } else {
    // attempt to connect to WiFi network
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network
    status = WiFi.begin(ssid, pass);
    if ( status == WL_CONNECTED) {
      // you're connected now, so print out the data
      Serial.println("You're connected to the network");
      Udp.begin(localPort);
    }
  }

  EsploraTFT.background(0,0,0);  // clear the screen with black

}

 
void loop() {
  int x_axis, y_axis, z_axis, g_axis;
  x_axis=0, y_axis=0, z_axis=0, g_axis=0;

  int S1, S2, S3, S4;   // holds values of switches 1 to 3
  S1=HIGH; S2=HIGH; S3=HIGH; S4=HIGH;
  S1 = Esplora.readButton(SWITCH_1);
  S2 = Esplora.readButton(SWITCH_2);  
  S3 = Esplora.readButton(SWITCH_3);
  S4 = Esplora.readButton(SWITCH_4);

  if(S1==LOW) { // Gripper changes
    if(Esplora.readJoystickY() > 20)   // if the joystick is moved significantly down
      g_axis = -3;
    if(Esplora.readJoystickY() < -20)  // if the joystick is moved significantly up
      g_axis = 1;
  }
  else if(S2==LOW) {  // ???
  }
  else if(S3==LOW) { // Use Accelerometer
    if(Esplora.readAccelerometer(X_AXIS) > 75)   // if the joystick is moved significantly left
      x_axis = -10;  // Map to 0-10 range
    else if(Esplora.readAccelerometer(X_AXIS) > 20)   // if the joystick is moved significantly left
      x_axis = map(Esplora.readAccelerometer(X_AXIS), 0, 75, 0, -10);  // Map to 0-10 range
    if(Esplora.readAccelerometer(X_AXIS) < -75)   // if the joystick is moved significantly right
      x_axis = 10;  // Map to 0-10 range
    else if(Esplora.readAccelerometer(X_AXIS) < -20)   // if the joystick is moved significantly right
      x_axis = map(Esplora.readAccelerometer(X_AXIS), 0, -75, 0, 10);  // Map to 0-10 range

    if(Esplora.readAccelerometer(Y_AXIS) > 150)   // if the joystick is moved significantly down
      y_axis = -10;  // Map to 0-10 range
    else if(Esplora.readAccelerometer(Y_AXIS) > 70)   // if the joystick is moved significantly down
      y_axis = map(Esplora.readAccelerometer(Y_AXIS), 25, 150, 0, -10);  // Map to 0-10 range
    if(Esplora.readAccelerometer(Y_AXIS) < -60)   // if the joystick is moved significantly down
      y_axis = 10;  // Map to 0-10 range
    else if(Esplora.readAccelerometer(Y_AXIS) < -5)   // if the joystick is moved significantly down
      y_axis = map(Esplora.readAccelerometer(Y_AXIS), 25, -60, 0, 10);  // Map to 0-10 range

    z_axis = map(Esplora.readAccelerometer(Z_AXIS), 50, 150, -5, 5);  // Map to 0-10 range
  }
  else if(S4==LOW) { // Z-axis changes
    if(Esplora.readJoystickY() > 20)   // if the joystick is moved significantly down
      z_axis = map(Esplora.readJoystickY(), 0, 512, 0, -10);  // Map to 0-10 range
    if(Esplora.readJoystickY() < -20)  // if the joystick is moved significantly up
      z_axis = map(Esplora.readJoystickY(), 0, -512, 0, 10);  // Map to 0-10 range
  }
  else { // No button pressed
    if(Esplora.readJoystickY() > 20)   // if the joystick is moved significantly down
      y_axis = map(Esplora.readJoystickY(), 0, 512, 0, -10);  // Map to 0-10 range
    if(Esplora.readJoystickY() < -20)  // if the joystick is moved significantly up
      y_axis = map(Esplora.readJoystickY(), 0, -512, 0, 10);  // Map to 0-10 range
    if(Esplora.readJoystickX() > 20)   // if the joystick is moved significantly left
      x_axis = map(Esplora.readJoystickX(), 0, 512, 0, -10);  // Map to 0-10 range
    if(Esplora.readJoystickX() < -20)  // if the joystick is moved significantly right
      x_axis = map(Esplora.readJoystickX(), 0, -512, 0, 10);  // Map to 0-10 range
  }  

  displayString(10, 10,"x_axis", COLOR_WHITE);
  displayString(10, 20,"y_axis", COLOR_WHITE);
  displayString(10, 30,"z_axis", COLOR_WHITE);
  displayString(10, 40,"g_axis", COLOR_WHITE);

  displayInt2(x_axis, 70, 10, COLOR_GREEN, COLOR_BLACK);
  displayInt2(y_axis, 70, 20, COLOR_GREEN, COLOR_BLACK);
  displayInt2(z_axis, 70, 30, COLOR_GREEN, COLOR_BLACK);
  displayInt2(g_axis, 70, 40, COLOR_GREEN, COLOR_BLACK);

  // UDP SEND LOGIC GOES HERE
  delay(500);

}



void displayString(byte x, byte y, char *text, uint16_t color) { // write string to LCD
  EsploraTFT.stroke(color);
  EsploraTFT.text(text,x,y);
}

void displayChar(byte x, byte y, char text, uint16_t color) {  // write character to LCD
  EsploraTFT.stroke(color);
  EsploraTFT.text(text,x,y);
}

// display an unsigned character on LCD (if you have a signed value, use displayInt2)
void displayInt(unsigned int num, byte nx, byte ny, unsigned int color, unsigned int backcolor) {
  displayInt2(num, nx, ny, color, backcolor);
}

void displayInt2(int num, byte nx, byte ny, unsigned int color, unsigned int backcolor) {
  char text[5];
  String str;
  str=String(num);
  str.toCharArray(text,5);

  EsploraTFT.noStroke(); // don't draw a line around the next rectangle
  EsploraTFT.fill(backcolor); // set the fill color to green
  EsploraTFT.rect(nx, ny, 29, 7); //draw a rectangle across the screen
  EsploraTFT.stroke(color);
  EsploraTFT.text(text,nx,ny);  // print handles signs, leading space, etc. in this version
}
Arduino MKR WiFi 1010 robot codeArduino
This code is far from complete, it's just a start!!!
/*
  This code is far from complete, it's just a start!!!
*/


// Adafruit PWM Servo Driver Library - Version: 1.0.2
#include <Adafruit_PWMServoDriver.h>

#include <SPI.h>
#include <WiFiNINA.h>
#include <WiFiUdp.h>

int led =  LED_BUILTIN;
int status = WL_IDLE_STATUS;

char ssid[] = "Arduino_MeArm";            // your network SSID (name)
char pass[] = "mrAeM_oniudrA";        // your network password
int keyIndex = 0;            // your network key Index number (needed only for WEP)

unsigned int localPort = 10002;      // local port to listen on

char packetBuffer[255]; //buffer to hold incoming packet
char replyBuffer[2];       // a string to send back

WiFiUDP Udp;




/* meArm IK joysticks - York Hackspace May 2014
 * Using inverse kinematics with joysticks
 */
#include "meArm.h"
#include <Servo.h>

int basePin = 6;
int shoulderPin = 4;
int elbowPin = 5;
int gripperPin = 7;

meArm arm;






void setup() {
  //Initialize serial and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

  arm.begin(basePin, shoulderPin, elbowPin, gripperPin);


  Serial.println("Access Point Web Server");

  pinMode(led, OUTPUT);      // set the LED pin mode

  // check for the WiFi module:
  if (WiFi.status() == WL_NO_MODULE) {
    Serial.println("Communication with WiFi module failed!");
    // don't continue
    while (true);
  }

  String fv = WiFi.firmwareVersion();
  if (fv != "1.0.0") {
    Serial.println("Please upgrade the firmware");
  }

  // by default the local IP address of will be 192.168.4.1
  // you can override it with the following:
  // WiFi.config(IPAddress(10, 0, 0, 1));

  // print the network name (SSID);
  Serial.print("Creating access point named: ");
  Serial.println(ssid);

  // Create open network. Change this line if you want to create an WEP network:
  status = WiFi.beginAP(ssid, pass);
  if (status != WL_AP_LISTENING) {
    Serial.println("Creating access point failed");
    // don't continue
    while (true);
  }

  // wait 10 seconds for connection:
  delay(10000);

  // start the udp server on port 80
  Udp.begin(localPort);

  // you're connected now, so print out the status
  printWiFiStatus();
  
}


void loop() {

  // compare the previous status to the current status
  if (status != WiFi.status()) {
    // it has changed update the variable
    status = WiFi.status();

    if (status == WL_AP_CONNECTED) {
      // a device has connected to the AP
      Serial.println("Device connected to AP");
    } else {
      // a device has disconnected from the AP, and we are back in listening mode
      Serial.println("Device disconnected from AP");
    }
  }


  // if there's data available, read a packet
  int packetSize = Udp.parsePacket();
  if (packetSize) {
    Serial.print("Received packet of size ");
    Serial.println(packetSize);
    Serial.print("From ");
    IPAddress remoteIp = Udp.remoteIP();
    Serial.print(remoteIp);
    Serial.print(", port ");
    Serial.println(Udp.remotePort());

    // read the packet into packetBufffer
    int len = Udp.read(packetBuffer, 255);
    if (len > 0) {
      packetBuffer[len] = 0;
    } else {
      packetID = packetBuffer[0];
      packetCMD = packetBuffer[1];
      packetVAL = packetBuffer[2];
      
      blinkLED();
      if (packetCMD == "GC"){
        arm.closeGripper();
      }
      if (packetCMD == "GO"){
        arm.openGripper();  
      }
    }
    Serial.println("Contents:");
    Serial.println(packetBuffer);


    if (len > 0) {
      packetBuffer[len] = 0;
    } else {
      packetID = packetBuffer[0];
      packetCMD = packetBuffer[1];
      packetVAL = packetBuffer[2];
      packetVALX = packetBuffer[3];
      packetVALY = packetBuffer[4];
      packetVALZ = packetBuffer[5];
    }
    

    // send a reply, to the IP address and port that sent us the packet we received
    Udp.beginPacket(Udp.remoteIP(), Udp.remotePort());
    replyBuffer[0] = packetID;
    replyBuffer[1] = "ACK";
    Udp.write(replyBuffer);
    Udp.endPacket();


    if (len <= 0) {
      blinkLED();

      if (packetCMD == "GRC"){
        arm.openGripper();
      }
      if (packetCMD == "GRO"){
        if (( arm.getG()-(packetVAL/10) ) > 0) {
          arm.closeGripper( arm.getG()-(packetVAL/10) );  
        } else {
          arm.closeGripper(0);  
        }
      }

      if (packetCMD == "ARM"){
        //if (arm.isReachableCylinder(arm.getX() + packetVALX, arm.getY() + packetVALY, arm.getZ() + packetVALZ) {
          arm.gotoPointCylinder(arm.getX() + packetVALX, arm.getY() + packetVALY, arm.getZ() + packetVALZ);
      //}
      }
    }

  }
}

void blinkLED() {
  digitalWrite(led, HIGH); // Turn the LED on
  delay(50);
  digitalWrite(led, LOW);  // Turn the LED off

}

Schematics

Arduino Esplora + ESP8266 + PowerBoost
Fritzing esplora d0jmnztymx
Arduino MKR WiFi 1010 + PWM/Servo board + MeArm + OLED display
Fritzing mkr1010 mtk3fvrvrb

Comments

Add projectSign up / Login