Project showcase

BLE Haptic Dual Joystick Controller © LGPL

Create custom Bluetooth low energy controller with haptic feedback for VR immersion using Arduino 101

  • 1,526 views
  • 0 comments
  • 0 respects

Components and supplies

Ardgen 101
Arduino 101 & Genuino 101
×1
Analog joystick (Generic)
PC analog joystick. Other controller requires calibration
×2
Mfr 25frf52 10k sml
Resistor 10k ohm
Pull up resistor for joystick buttons
×4
Mfr 25frf52 100k sml
Resistor 100k ohm
Resistors for analog joystick
×4
Relay (generic)
5V 4-Channel Relay Module for Arduino
×1
10097action
SparkFun Serial Enabled LCD Kit
×1
Gear%20vr
Oculus Gear VR
×1
11026 02
Jumper wires (generic)
×1
1434509556 android phone color
Android device
×1
Adafruit industries ada64 image 75px
Solderless Breadboard Half Size
×1
826 04
Male/Female Jumper Wires
×1

Apps and online services

Unity logo
Unity
Download personal edition to Build the Unity project

About this project

Imaging playing a VR racer game where can feel the rush of wind, vibration of the seat, and shock of pain upon crashing. In this project I'll show you how I build a customized dual joystick controller and environmental feedback gadgets to enhance VR immersion in games and experiences.

The system components are:

  • User controller device
  • Android Bluetooth serial plugins for data exchange,
  • Unity Game Engine for interaction and event output.
  • Micro controller relay input data and activate power switches for feedback devices.

Arduino 101 with low latency Bluetooth LE is perfect application for exchanging wireless controller data with Smartphone-powered VR headsets such as Samsung Gear VR or Google cardboard.

Joystick Controller

For controller input, I recycled old pair of PC joysticks with DA-15 game port connector. The exposed male connector pin can easily connect to Arduino analog and digital pins with pull up resisters without wire cutting. I wired in X and Y axis for each joystick and used only 2 buttons on each joystick.

I followed this guide for pin assignments and diagrams. Additional buttons assignments is possible by using multiplexer or shift register if running out out input pins on Arduino.

Haptic Feedback Devices

Unity Game Engine sends feedback commands via Android BLE drivers to Arduino. It will trigger output signal to a 4 channel relay that activates following AC power devices for my setup.

  • Piezoelectric Massage pad attached to the chair. Full vibration setting with analog power switch to on for all regions.
  • 2 fans with different speed settings to simulate wind and direction.
  • Massage motor attached to a wooden board holding both joystick for control feedback.

AC Socket Switch

I built the generic AC socket to create a 4 gang socket for relay control. Cut the connecting tab in the silver screws to separate out socket. Connect white neutral wire to relay.

Vibrating Motor

Provides mild buzz vibrating feedback to joystick controller. Build from a battery powered mini massager that lights up when turned on. For interesting visual effect, place a stress ball between the tripod. The vibration will slowly rotated it.

  • Removed the switching cap and short out the switch. (Save round switch for powering the Arduino.)
  • Attached power wires to positive and negative battery terminal. Connect a 5v DC female socket connector. Used a 4.5V DC adaptor to power the device from AC socket.
  • For holder, drill and screw in the plastic cap from the spray can into the board.
  • Attach to the board holding the joysticks to provide haptic feedback.

Unity Demo Project

I created a sample project in Unity to test out the controller. It includes paddle puck controlled via joysticks. As virtual object moves around different zones it'll activate vibrating pad, motor, and fans.

Joystick moves the player in x and y direction. 2nd joystick will rotate puck in the direction of joystick to hit virtual objects.

I can feel the rumble of the marble and rush of vortex wind in VR. This opens new possibilities for making future interactive VR art installations.

You can try out the Ble Controller with Arduino 101 and LCD with the game engine. Build in Accelerometer will act as single joystick. without the Relay switch for haptic feedback.

Controller needs to be on and Bluetooth enabled on the phone before launching app.

DOWNLOAD DEMO Android APK (non-VR)

Complete sample project source is available to check out.

Code

Arduino codeC/C++
Arduino code to read analog input from joysticks. Readings are displayed in the optional LCD display.
When connection is established via bluetooth LE to VR game in head set, send joystick control data and receive haptic feedback signal from app.
/*
 * Arduino 101 Sketch sends controller data from 2 analog joystick over bluetooth LE as UART serial data.
 * It also received commands from connected app to activate relays.
 * Used for custom VR haptic controller with Gear VR.
 * 
 * Created by  Leon Hong Chu   @chuartdo
 * See the bottom of this file for the licence and credits 
 */

#include <CurieBLE.h>
#include "CurieIMU.h"

#define lcd_display 1
#ifdef lcd_display 
 #include <Wire.h>
 #include "rgb_lcd.h"
 rgb_lcd lcd;
#endif 


byte txData[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
const int ledPin = 15;  // Led indicator for BLE connection

/* Joystick configuration based on 15 pin */
const int joyStick1XPin = A0;
const int joyStick1YPin = A1;
const int joyStick2XPin = A2;
const int joyStick2YPin = A3;
const int button1Pin = 0;
const int button2Pin = 1;
const int button3Pin = 2;
const int button4Pin = 4;

/* ===== Relay pin assignment for activating feedback devices */
int relayPins[] = { 8,9,10,11 };
int relayPinCount = 4;

BLEPeripheral blePeripheral;   

// Configure Nordic Semiconductor UART service 
BLEService UartService = BLEService("6E400001B5A3F393E0A9E50E24DCCA9E");
BLECharacteristic TransmitCharacteristic = BLECharacteristic("6E400003B5A3F393E0A9E50E24DCCA9E", BLENotify , 20); 
BLECharacteristic ReceiveCharacteristic = BLECharacteristic("6E400002B5A3F393E0A9E50E24DCCA9E", BLEWriteWithoutResponse, 20); 

void setup() {
  Serial.begin(9600);
  pinMode( ledPin, OUTPUT); 
  pinMode( button1Pin, INPUT); 
  pinMode( button2Pin, INPUT); 
  pinMode( button3Pin, INPUT); 
  pinMode( button4Pin, INPUT); 
  pinMode( joyStick1XPin, INPUT); 
  pinMode( joyStick1YPin, INPUT);
  pinMode( joyStick2XPin, INPUT);
  pinMode( joyStick2YPin, INPUT);

  for( int i = 0; i < relayPinCount; i++ ) {
    pinMode( relayPins[i], OUTPUT); 
  }

  #ifdef lcd_display
    lcd.begin(16, 2);
    lcd.print("Calibrate JOYSTX");
    lcd.setCursor(0,1);
    lcd.print("Rotate ");
    lcd.setRGB(100, 100, 100);
  #endif
  
  // ===== Set advertised device name:
  blePeripheral.setLocalName("DualJoy");

  // set characteristics and event handler
  blePeripheral.setAdvertisedServiceUuid(UartService.uuid());
  blePeripheral.addAttribute(UartService);
  blePeripheral.addAttribute(TransmitCharacteristic);
  blePeripheral.addAttribute(ReceiveCharacteristic);

  blePeripheral.setEventHandler(BLEConnected, blePeripheralConnectHandler);
  blePeripheral.setEventHandler(BLEDisconnected, blePeripheralDisconnectHandler);
  ReceiveCharacteristic.setEventHandler(BLEWritten, ReceiveCharacteristicWritten);

  blePeripheral.begin();

  // Onboard accelerometer setup
  CurieIMU.begin();
  CurieIMU.setAccelerometerRange(2);
  CurieIMU.setGyroRange(250);

  detectController();
}

 
int mode = 1 ;  // Fall back to use internal accelerometer for testing without connected joystick and buttons

void detectController() {
  if (digitalRead(button1Pin)> 0 &&
      digitalRead(button2Pin)> 0)
      mode =21;
}

void loop() {
  
  blePeripheral.poll();  
  
  switch (mode) {
    case 1:  
         AccelerometerLoop(); 
         break;
    case 2:
         GyroLoop();
         break;
    default:
       readJoystickLoop();
  }
}


#ifdef lcd_display
  void writeLCD(String  prefix, float x, float y, int row) {
      String outputMsg;
      outputMsg = String(prefix);
      outputMsg += String(x);
      outputMsg += " ";
      outputMsg += String(y);
      outputMsg += "\0";
  
      for (int i = outputMsg.length(); i<16 ; i++);
        outputMsg += " ";
      lcd.setCursor(0,row);
      Serial.print(outputMsg);
      lcd.print(outputMsg);
  }
  
  int colorCycle = 1;
  
  void changeColor() {
 
    int r = (0x04 & colorCycle) > 0?50:10;
    int g = (0x02 & colorCycle) > 0?50:10;
    int b = (0x01 & colorCycle) > 0?50:10;
    lcd.setRGB(r, g, b);
    colorCycle += 1;
    if (colorCycle >= 8)
       colorCycle = 0;
  }

#endif


bool sendData = false;

void blePeripheralConnectHandler(BLECentral& central) {
  Serial.print("Connected to ");
  Serial.println(central.address());
  sendData = true;
  #ifdef lcd_display
   lcd.setRGB(0, 0, 20); // Dim LCD
  #endif
  digitalWrite(ledPin, HIGH);
}

void blePeripheralDisconnectHandler(BLECentral& central) {
  Serial.print("Disconnected ");
  Serial.println(central.address());
  sendData = false;
  turnOffAllRelay();
  digitalWrite(ledPin, LOW);
}

void activateRelay(int relayNum, bool state) {
  if (relayNum >= 0 && relayNum < relayPinCount ) {
    
    #ifdef lcd_display
      colorCycle = relayNum + 1;
      if (state > 0)
        changeColor();
      
    else
      Serial.print("Activate relay ");
      Serial.println(relayNum);   
      Serial.print("  ");
      Serial.println(state?"ON":"OFF");
    #endif
    digitalWrite(relayPins[relayNum] ,state);

 }
}
/* Obtain 3 byte command from ble/ Andorid and turn on relay based on number */
/* 3 byte command, relayNum, value */

void ReceiveCharacteristicWritten(BLECentral& central, BLECharacteristic& characteristic) {

  int relayNum=-1;  
  if (characteristic.value()) {      
    int len = characteristic.valueLength(); 
    Serial.print(len);   //print out the character to the serial monitor
    Serial.println(" Received");

    byte* command = (byte*) characteristic.value();
   
    if (len == 3) {
        relayNum = (int) command[1];
        if (relayNum >= 48)  // Map Ascii character for testing via command line
          relayNum -=48;
          
        bool state = command[2] > 0?HIGH:LOW;
        activateRelay(relayNum, state);
    }
  }
}

byte* floatToByteArray(float f, byte* ret) {
    unsigned int asInt = *((int*)&f);
    int i;
    for (i = 0; i < 4; i++) {
        ret[i] = (asInt >> 8 * i) & 0xFF;
    }
    return ret;
}

byte* longToByteArray(long f, byte* ret) {
    unsigned int asInt = *((long*)&f);
    int i;
    for (i = 0; i < 4; i++) {
        ret[3-i] = (asInt >> 8 * i) & 0xFF;
    }
    return ret;
}

void GyroLoop () {
  float gx, gy, gz;
  CurieIMU.readGyroScaled(gx, gy, gz);
  sendControlData(gx, gy, gz, 0, 0);
}

void AccelerometerLoop () {
  float ax, ay, az;
  CurieIMU.readAccelerometerScaled(ax, ay, az);
  #ifdef lcd_display
   writeLCD("Acc: ",ax,ay,0);
   writeLCD("z: ",az,0,1);
  #endif
  sendControlData(ax, ay, az, 0, 0);
}

// Control data fit within 20 bytes. 2 joystick axis data + 4 button states
// ===== Modify to place custom controller data

void sendControlData(float x1, float y1, float x2, float y2, long buttons) {
   floatToByteArray(x1, &txData[0]);
   floatToByteArray(y1, &txData[4]);
   floatToByteArray(x2, &txData[8]);
   floatToByteArray(y2, &txData[12]);
   longToByteArray(buttons, &txData[16]);

/*
for (int i=0; i<20; i++) {
  Serial.print(txData[i],HEX);
}
*/
    Serial.print(x1); Serial.print(" ");
    Serial.print(y1); Serial.print(" ");
    Serial.print(x2); Serial.print(" ");
    Serial.print(y2); Serial.print(" ");
    Serial.println(buttons); 
   
    TransmitCharacteristic.setValue(txData, 20);  
}


float mapfloat(float val, float in_min, float in_max, float out_min, float out_max)
{
  return out_min + (out_max - out_min) * (( val - in_min) / ( in_max - in_min));
}

float normalize(float val, float minimum, float center, float maximum) {
  if (val > center) {
    val =  mapfloat(val,center, maximum, 0, 1.1);
    if (val > 1.0)
      val = 1.0;
  }
  else {
     val =  mapfloat(val,minimum, center, -1.0, 0);
  }
  return val;
}
 
// Read data store in following encoded bytes  
// Convert to nomalized value -1 to 1  O is near center


float getMinMax(int source, int* min, int* max) {
  if (source < *min) 
     *min = source;
  else if (source > *max)
     *max = source;
  return source;
}

void turnOffAllRelay() {
  for (int i=0; i< relayPinCount; i++) {
    digitalWrite(relayPins[i] ,LOW);
  }  
}

bool buttonState[4];
int buttonStates() {
  int state = 0;
  
  if (!buttonState[0] ) {
    state += 1;
  }
  if (!buttonState[1]) {
    state += 2;
  }
  if (!buttonState[2]) {
    state += 4;
  }
  if (!buttonState[3]) {
    state += 8;
  }
  return state;
}

int max_j1x= -1, min_j1x= 9999, max_j1y=-1, min_j1y=9999;
int max_j2x= -1, min_j2x= 9999, max_j2y=-1, min_j2y=9999;
int ctr_x1=600, ctr_y1=600, ctr_x2=600, ctr_y2=600;

bool calibration = true;
bool recordCenter = true;


void readJoystickLoop () {
  int x1a, y1a, x2a, y2a;
  
  x1a = analogRead(joyStick1XPin);
  y1a = analogRead(joyStick1YPin);
  x2a = analogRead(joyStick2XPin);
  y2a = analogRead(joyStick2YPin);

  if (recordCenter) {  // Get initial center state of joysticks on power on
    #ifdef lcd_display
      lcd.setRGB(50, 50, 50);
      lcd.print("Calibrate JOYSTX");
      lcd.setCursor(0,1);
      lcd.print("Record Center POS");
      delay(1500);   
    #endif
    
    ctr_x1 = x1a;
    ctr_y1 = y1a;
    ctr_x2 = x2a;
    ctr_y2 = y2a;
    recordCenter = false;

    #ifdef lcd_display
     lcd.setRGB(0, 100, 0);
    #endif
  }

  buttonState[0]=digitalRead(button1Pin)==0;
  buttonState[1]=digitalRead(button2Pin)==0;
  buttonState[2]=digitalRead(button3Pin)==0;
  buttonState[3]=digitalRead(button4Pin)==0;
  float buttons =  buttonStates();

 
  String outputMsg = "";
   
  // Get max and min range of joysticks
   getMinMax(x1a,&min_j1x,&max_j1x);
   getMinMax(y1a,&min_j1y,&max_j1y);
   getMinMax(x2a,&min_j2x,&max_j2x);
   getMinMax(y2a,&min_j2y,&max_j2y);

  // normalize stick movement range to -1 and 1 
  float fx1 = normalize(x1a,min_j1x, ctr_x1, max_j1x);
  float fy1 = normalize(y1a,min_j1y, ctr_y1, max_j1y);
  float fx2 = normalize(x2a,min_j2x, ctr_x2, max_j2x);
  float fy2 = normalize(y2a,min_j2y, ctr_y2, max_j2y);
  
  if (sendData) {    
    #ifdef lcd_display
      writeLCD(" 1: ",fx1,fy1,0);
      writeLCD(" 2: ",fx2,fy2,1);
    #endif
    sendControlData(  fx1,  fy1, fx2, fy2, buttons);
    
  } else {

     #ifdef lcd_display
        writeLCD("J1: ",x1a,y1a,0);
        writeLCD("J2: ",x2a,y2a,1);
     #endif

    // Turn on Relay switch with button press
    for (int i=0; i< 3; i++) {
      activateRelay(i ,buttonState[i]?HIGH:LOW);
    }
     
    // Recenter joystick when all button pressed down
    if (buttons == 0) {
      recordCenter = true;
    }
  }
 
}

/*
  The Nordic Semiconductor UART profile for Bluetooth Low Energy
  
  Copyright (c) 2015 Intel Corporation. All rights reserved. 
  
  This library is free software; you can redistribute it and/or
  modify it under the terms of the GNU Lesser General Public
  License as published by the Free Software Foundation; either
  version 2.1 of the License, or (at your option) any later version.

  This library is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
  Lesser General Public License for more details.

  You should have received a copy of the GNU Lesser General Public
  License along with this library; if not, write to the Free Software
  Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-
  1301 USA
*/
Ble Controller Android Unity Plugin
The compiled Plugin is included in the Ble Controller Unity Project file. It exposes Androids BLe serial connection methods for use inside Unity Csharp. The plugin's source code is modified based on Adafruit Android BLE UART project. To build the plugin. Download Android Studio. Clone the repository from the link. Run "exportJar" gradle task in Adafruit_Android_BLE_UART/unity_libs/build.gradle. The build plugin UARTPlugin.jar will be inside release directory.
Unity Demo Projects
Include sample source in Unity 3D for the Haptic Controller. To build the project. Download Free Personal Edition from https://store.unity.com/. Import the Project. Build for Android.

Schematics

Wiring diagram
Connections used in the project
Dualjoy2 bb 1bsqqi95xa

Comments

Similar projects you might like

Arduino 101 BLE App

Project in progress by Alexis Santiago Allende

  • 11,316 views
  • 26 comments
  • 51 respects

Arduino 101 Home BLE System

Project tutorial by Alexis Santiago Allende

  • 2,508 views
  • 0 comments
  • 11 respects

Joystick Controller for MeArm Robot - Recording Coordinates

Project tutorial by utilstudio

  • 1,367 views
  • 2 comments
  • 18 respects

Arduino101 / tinyTILE BLE: Match-Making Sunglasses

Project tutorial by Kitty Yeung

  • 11,522 views
  • 4 comments
  • 36 respects

Arduino/Genuino 101 BLE Thermometer With TMP102 and Blynk

Project tutorial by Konstantin Dimitrov

  • 8,030 views
  • 1 comment
  • 26 respects
Add projectSign up / Login