Project tutorial

Intangible Surface © GPL3+

Intangible Surface is a gesture and IoT based touchless interface to interact with the digital world.

  • 14 respects

Components and supplies

Apps and online services

About this project

1. What Problem I am Trying to Solve?

A button is the most common interface to interact with the digital world. It could be as simple as a light switch or at a pedestrian crossing or it could be as complex as in the elevator or a kiosk. These buttons can act as a catalyst in the spread of viruses. The problem I am trying to solve is to reduce the touch-based interfaces a.k.a the catalysts by replacing it with a gesture and IoT based interface that is intangible, simple, and intuitive to operate.

2. How I am Trying to Solve It?

My solution is divided into two parts

Method 1: Replace the tangible buttons with an intangible gesture-based interface. A person will then interact with the digital world (light, fan, elevator, etc.) using gesture(s) instead of touch.

Method 2: No matter how much you try you will always touch your smartphone. So why not enable your smartphone to complement the gesture-based interface. It will make some complex interfaces like an elevator easy to use (to see the elevator control using a smartphone please go to the elevator application section i.e. end of the page).

A person will swipe right to turn ON a light/fan and swipe left to turn it OFF. Rotate hand clockwise or anticlockwise to increase or decrease a fan's speed or change a light's intensity. The same device can also be used for some complex interfaces like an elevator. Swipe up or down to call the elevator based on the direction you are going. Swipe or rotate your hand to select the floor. And the magical part is that all of these actions are possible using a single interface. No need for a separate knob to control the fan speed. No need for multiple buttons to select the floor numbers.

Selecting 2 digit floor on an elevator using gestures could be a multi-step process, therefore, you also have the option of controlling an elevator using your smartphone. Not only the elevator but you can control almost anything using your smartphone.

3. Circuit Diagram

The main elements of my design are

  • Gesture sensor
  • Arduino Wi-Fi 1010
  • LCD
  • Smartphone

The electronic modules I used for prototyping are grove based. Connecting them is a child's play. The sensor and LCD both use I2C protocol and the Arduino I am using has only one I2C port so I have used an I2C hub to connect all three modules. Please refer to the below schematics for connecting the modules.

4. Lets Assemble It

I have used a product box of a charger to assemble my project. Ideally, I would love to design and print a custom enclosure but I don't have access to a 3D printer.

I have slightly modified the box and it does a fantastic job of housing the circuit. It also hides the cluttered wiring and makes it look clean.

5. How Does It Work?

A person can control any device using intangible surface via two methods

  • Gesture
  • IoT (I don't want to call it IoT, please suggest me a better name)

5.1 Gesture

The gesture sensor I am using can detect 9 different gestures

  • Left
  • Right
  • Up
  • Down
  • Clockwise
  • Anti-clockwise
  • Forward
  • Backward
  • Wave

The sensor detects the gesture and sends it to the Arduino. Arduino then translates the gesture into specific meaning based on the device we want to control. Also, the number of gestures used depends on the device we want to control. The interface at pedestrian crossing needs only 1 gesture and light needs 2 whereas a fan needs 4 gestures.

5.2 IoT (Someone Else's Computer)

In this method, your smartphone acts as a remote control. The app on your phone uses the MQTT protocol over WiFi to communicate with the intangible surface. When you click a button on your mobile app, it publishes a message to a topic. The Intangible Surface reads that message by subscribing to that topic. After processing the message it sends a control signal to the device it is controlling.

Intangible Surface then publishes the feedback message to the topic which is then reflected by the mobile app.

To know more about MQTT protocol please click here. I used the Mosquitto broker installed on a RaspberryPi (acting as a server) to test this design. I have also tested it with CloudMQTT. You can use any MQTT broker of your choice.

The clear benefit of this method is that multiple people can control the same device at the same time. So you and your friend can select different floor numbers in an elevator at the same time using your smartphones.

The smartphone app I am using to talk to the Intangible Surface is called MQTT Dashboard. I have designed various control interfaces in it according to the different applications.

6. Applications

I have developed some application interface demonstrating the capability of the Intangible Surface.

6.1 Pedestrian Crossing

You might have seen and used a switch at a pedestrian crossing which looks like this

It's a very simple interface. You press the button and wait for a while until the signal turns green. Once the signal is green, it tells you to walk. Let us replicate this functionality but this time without touching a button.

Pedestrian Crossing Interface

In the new interface just swipe LEFT or RIGHT to activate the signal. Voila!

6.2Light Switch

You must have used an on/off switch to control a light or any other electrical appliance.

Lets replace it with our touch-less interface.

On Off Switch Interface

Simply swipe RIGHT to turn ON the light and swipe LEFT to turn it OFF.

6.3 Fan Control

You might have seen a fan switch and a knob to control its speed(Sorry! I can't find a royalty-free image). Lets replicate that interface using Intangible Surface.

Fan Control Interface

Swipe RIGHT to turn ON and swipe LEFT to turn OFF. Rotate your hand CLOCKWISE to increase the speed and ANTICLOCKWISE to decrease the speed.

6.4 Elevator

A common elevator has two interfaces, one to summon it (outside), and second to select the floor (inside).

The elevator summon interface look like this.

To call the elevator using Intangible Switch, simply swipe UP or DOWN based on the direction you want to go.

Elevator Summon Interface

The floor selection interface inside the elevator is generally quite complex. You will find lots of buttons like this.

These buttons are divided into 3 main category

1. Floor selection buttons.

2. Door control buttons.

3. Alarm/Emergency button.

We can replace these 3 functionalities using our Intangible Surface.

1. Using the touch-less interface we can select the floor by rotating our hand CLOCKWISE or ANTICLOCKWISE.

Elevator Interface: Floor Selection

If a building has too many floors then you can swipe LEFT or RIGHT to decrease or increase the floor number by a factor of 5, respectively.

Elevator Interface: Floor Selection

2. For door control, simply swipe UP to open the door and swipe DOWN to close.

Elevator Interface: Door Control

3. To activate the alarm, wave your hand.

Elevator Interface: Emergency

I have implemented the Elevator control using IoT also. Using this method, one can interact with the elevator using a smartphone.

7. Future Work

I want to prototype one more feature and that is to "auto-detect Intangible Surface devices around you in the mobile App".

  • If Intangible Surface is deployed indoors then use BLE's advertise feature to inform users around it about the device's identity.
  • If Intangible Surface is deployed outdoors then use GPS in your smartphone to complement the above method.

Unfortunately in Arduino MKR 1010, you can use either WiFi or BLE. I am using WiFi to communicate with the MQTT server therefore I will need a separate BLE radio which I don't have at the moment.

8. Summary

Intangible Surface is an extremely versatile and cost-effective design that can be used to replace a number of touch-based interfaces, simple as well as complex. This device makes you touch less stuff and do more things. Replace the touch-based interface using this and Protect yourself and the people around you from COVID19.

Which interface you would like to replace?

Open Source is LOVE. Happy Making and Take Care:-)


Intangible Surface Repository
The master branch contains the stable code and the dev branch contains the latest code.


Arduino to Gesture Sensor
Arduino to gesture sensor 7hmatx4czt
Arduino to LCD
Arduino to lcd kd52vi3git


Similar projects you might like

Helmet Sterilization Box

Project tutorial by RoboticsBangladesh

  • 1 comment
  • 4 respects

Con-tactless Helmet Sterilization Box

Project tutorial by Team RoboticsBangladesh

  • 0 respects

COVID19 Realtime Monitor

Project tutorial by hwhardsoft

  • 61 respects

Connected Oximeter with SMS Alert for Covid-19

Project tutorial by Martin Cornu

  • 9 respects

Ultimate Cough Recognition: How Dangerous is Coughing?

Project tutorial by corey yin

  • 17 respects

Customer Limiting Shop Door Assistant V3 - IoT version

Project tutorial by Harry Anderson

  • 15 respects
Add projectSign up / Login