Project tutorial
Cimini

Cimini © CC BY

Chimes that everyone can operate using their own smartphone.

  • 64 views
  • 0 comments
  • 0 respects

Components and supplies

Necessary tools and machines

Apps and online services

About this project

Abstract

User sees a beautiful instrument in a corner in a public place. A sign with a QR-code invites them to a web page and to play the instrument using their mobile phone. An image of chimes appears.

User sweeps over the image, mimicing how chimes are played by dragging one's finger through the row of tiny metal bars. The page records the pattern and sends it to the Cimini, which replicates the movement, making the chimes sound. User smiles. Everybody nearby smiles. Everybody takes turns in creating some beautiful ambient music just using their phone..

Cimini in action

The microphone exaggerates the noise from the servos.

Basic concepts explored

  • People operating public IoT devices by using their own smartphones with just a web page as the interface, not having to download anything.
  • Record touch screen events on a smart device through Javascript.
  • Send a huge block of coordinates from the touch screen device to a MySQL database on a web site
  • Have the Arduino MKR WiFi 1010 (hereafter referred to as 1010) read the database on the web site and control the robot arm operating the chimes accordingly
  • Tuning 30 copper tube bars into a 15 tone chromatic range (2 bars per tone) of beautifully sounding chimes
  • Studying how people react to this kind of publically playable instrument

Chimes

Chimes are an instrument belonging to a percussionist's set of sound effect kind of instruments. It consists of a lot of tiny metal bars hanging from a wooden bar. Dragging one's finger through the row of bars makes them hit each other. The sound could be described as glass scattering in a good way.

The Cimini is simply a chimes instrument controlled by a robot. Instead of solid metal bars, the Cimini has pieces of 10 mm chrome plated copper tube.

Perché il nome?

Well, I tried to make it Italian, while still keeping the English. So chime became cime (chee-meh). And a small chime would be cimino, si? And since chimes is plural, so is Cimini. Ecco!

The tuning process

I cut one test piece from the chrome plated copper tube I'll be using. The length of the piece is 164 mm. I drilled a hole at 22.4% of the total length and hung the piece from that point. The frequency of the piece was 1658.2 Hz. The relation between length and frequency is as follows:

𝑙 = √(K/f)

𝑙 is the length of the tube piece, f is the frequency. K is a constant, which can be calculated from the density of the material of the tube and its inner and outer diameter. But K can also be calculated if we measure the frequency and length. All tube pieces will be of same material and same inner and outer diameter.

So with a 164 mm piece and 1658.2 Hz, we get a value for K about 44598000.

Using a tuning based on A4 = 442 Hz (concert pitch), I thought of having 30 pieces ranging from A♭6 to B♭7, that's 2 bars per tone. That's from 1669 Hz to 3746 Hz. Each tone is ¹²√2 times higher than the previous. After 12 steps we have doubled the frequency. That's in a nutshell the principle of equal temperament, the most used tuning system in Western music.

The 12 steps, which take you up one octave, or to the doubled frequency, are actually called half steps. If you want to go up say 7 half steps from a frequency, you multiply the frequency by 2 to the power of 7/12. Or (¹²√2)⁷. If you go the other way, if you have two frequencies, f₁ and f₂, and want to know the number of half steps, the equations go as follows:
(¹²√2)ˣ = f₁ / f₂x * ln (¹²√2) = ln (f₁ / f₂)x = ln (f₁ / f₂) / ln (¹²√2)
You end up in x having an integer part and a decimal part. If the decimal part is zero, the two tones are in tune wich each other (the integer part is the number of half steps). But they might still be out of the standard A4 = 442 Hz tuning, so check the frequencies with 442 Hz, too (if 442 Hz is your concert pitch).

My test piece happened to be 164 mm with a frequency of 1658.2 Hz. This will be my lowest note, if I just tune it up to 1669 Hz. That means filing some 0.5 mm off the bar (well, an electronic tuner is better than a high precision caliper at this point). The list of bars would be as follows:

Tone    Frequency (Hz)  Length (mm)    Hole at 22.4% (mm)
==== ========= ====== =============
Ab6 1 668.77 163.5 36.6
A6 1 768.00 158.8 35.6
Bb6 1 873.13 154.3 34.6
B6 1 984.51 149.9 33.6
C7 2 102.52 145.6 32.6
C#7 2 227.54 141.5 31.7
D7 2 360.00 137.5 30.8
D#7 2 500.33 133.6 29.9
E7 2 649.01 129.8 29.1
F7 2 806.53 126.1 28.2
F#7 2 973.41 122.5 27.4
G7 3 150.22 119.0 26.7
Ab7 3 337.54 115.6 25.9
A7 3 536.00 112.3 25.2
Bb7 3 746.26 109.1 24.4

Orchestral chimes may have other tunings, they might have tones not belonging to the Western music tradition at all, but I decided to go the safe way.

The design of the instrument

The 30 bars hang in a 150° arc, radius 160 mm. In the center point there's the tilt-pan device, which has an arm attached. The arm should reach to each chime piece. The bars hang about 5 mm apart from each other. If they are too close, they damp each other instead of letting each piece vibrate. If they are too far from each other, they stop hitting each other too soon while still swinging back and forth.

As mentioned before, each bar should have the hole drilled at 22.4% of the total length, a node, which doesn't vibrate when the bar is hit. Orchestral chimes on the market don't have the holes drilled at the critical node, they simply rely on the solid metal bar vibrating strong enough. But I use cheap copper tube and have to optimise everything. The holes should be perpendicular to the row of bars, to let the pieces freely hit each other while swinging. Or in this case, when the bars form a semi circle, the hole direction should be radial. The length of the supporting thread should be short enough not to steal the kinetic energy from the swinging bars. I'm aiming for having each bar hang about 5 mm from the ceiling plate. But the arm of the tilt-pan device must still reach to each chime, which might lead to the shortest pieces hanging a bit lower. So it's about considering a lot of things, when doing the design.

The actual arc of the hanging bars will be 150°. This will give 15° of free space at each side for the arm to operate. At this point I checked that my horizontal servo actually does 180°. It goes even a bit beyond that. The bars will hang from a rectangular plate like this (top-down view):

The thick black arc radius is 160 mm. My servo has enough torque to drag through the bars at that radius. 150° with a radius of 160 mm gives an arc length of 419 mm, or 5.17° and 14.4 mm per bar. Each bar is 10 mm, so there's some 4 mm space between them, when hanging in rest. I hope it is enough for the sustaining sound, when the swinging has diminished so much that they don't hit each other anymore.

No protractor, no compass

The radius of the bar arc is 160 mm. Each bar hangs in a string, which needs two holes in the rectangular ceiling plate. The holes form their own arcs at radii at 150 mm and 170 mm (the bar diameter is 10 mm and the supporting string hangs from holes 20 mm apart from each other). These holes I'm going to mark in the plate by using just a ruler. The school example in geometry would be to use a protractor, a compass and a ruler, but I thought a ruler and some trigonometry will do. I will mark spots along the side of the ceiling plate. A line from the center to the spot will show where the drill holes are to be made. The distance from the center to the left edge is 165 mm. The distance from the upper left corner (in the image) is 175 mm. The first spot is at 15°. Its distance from the upper left corner is hence 175 mm - 165 mm * tan(15°). In my case, I can place seven bars, marking the spots along the left side. And of course seven bars will be placed symmetrically at the right side. The rest of the bars (16 bars) get their points along the top side in a similar way. Mind that for this project I have just used pieces of wood and plywood I happened to have lying around. If you want to replicate this project, I advise you to use your own materials and recalculate everything. But if you want to use my measurements (including finding chrome plated copper tube of 10 mm), here are the measurements of the markings along the sides of the ceiling plate:

Bar	degr	mm from upper left corner
=== ==== =========================
1 15.0 130.8 along the left side
2 20.2 114.4 along the left side
3 25.3 96.8 along the left side
4 30.5 77.7 along the left side
5 35.7 56.5 along the left side
6 40.9 32.3 along the left side
7 46.0 3.9 along the left side
8 51.2 24.3 along the top side
9 56.4 48.6 along the top side
10 61.6 70.2 along the top side
11 66.7 89.7 along the top side
12 71.9 107.8 along the top side
13 77.1 124.8 along the top side
14 82.2 141.2 along the top side
15 87.4 157.1 along the top side
16 92.6 172.9 along the top side
17 97.8 188.8 along the top side
18 102.9 205.2 along the top side
19 108.1 222.2 along the top side
20 113.3 240.3 along the top side
21 118.4 259.8 along the top side
22 123.6 281.4 along the top side
23 128.8 305.7 along the top side
24 134.0 3.9 from upper right corner along the right side
25 139.1 32.3 from upper right corner along the right side
26 144.3 56.5 from upper right corner along the right side
27 149.5 77.7 from upper right corner along the right side
28 154.7 96.8 from upper right corner along the right side
29 159.8 114.4 from upper right corner along the right side
30 165.0 130.8 from upper right corner along the right side

The center is at 165 mm from the left side and 50 mm from the bottom side. Mark this center point with a pencil, mark all the 30 points along the sides, place your ruler with zero at the center point and at each of the 30 points, mark the places for the holes for the supporting string at 150 mm and 170 mm. No need to draw lines, which you would later want to rub out anyhow. I actually used a sheet of paper (A4) and made marks at the long side at 0 mm, 150 mm and 170 mm, then I used the sheet as a ruler.

For the supporting thread I'm using thin copper wire. I'm hoping to catch some of the vibrations to transfer to the ceiling plate. If nylon or cotton thread is used, my guess is that more vibration energy would dissipate in the thread itself.

The robot and the arm

When you play chimes by dragging your finger through them, you realize it's a delicate movement. You would need at least four degrees of freedom to replicate it with a robot. You move your finger in three dimensions, not two. You twist your finger, adjusting how the chimes nicely fall off the tip of your finger.

My robot, the pan-tilt module, has only two degrees of freedom. I could use a more advanced robot module, but then again, the user interface is just a 2D image surface on the smartphone, where the user drags their finger in two dimensions. The robot will therefore touch the hanging chimes with a spheric object, allowing the chimes glide off the object smoothly, no matter what the movement direction is. In my tests, the ideal spheric object appeared to be a table tennis ball! It weighs almost nothing and its friction against the chimes is almost none.

The pan-tilt module is placed so that the pan movement axis is exactly at the centre of the arc of chimes. The height of the arm has to be at the point, where the arm reaches the middle point of the chimes, when the arm is horizontal. The arm must be able to tilt both down and up from that horizontal position. This means that when the tilt-pan module is in its lowest tilt position (kind of its horizontal level), the arm must point down, to reach a point below the longest chime!

I re-used the black plastic foam piece (8 mm thick), that protected the pins of the 1010. The pan-tilt is glued onto the plastic foam piece, which is glued onto the white piedestal, raising the vertical tilt part axis to a level just about the middle point of the row of chimes. No screws are used for attaching the pan-tilt device to the piedestal. This prevents vibrations from creating unwanted noisy resonances in the wooden parts.

The software

The user browses to the site (johanhalmen.xyz/cimini) and gets this image:

The html page (index.html) that shows this image, has a javascript section that records touch events (start, move and end events), their coordinates and timestamps. Each swipe creates a long list of data, which is sent to a PHP script (receive.php), which saves the list in a MySQL database. Each swipe will have its own ID number. After the first swipe, the static image is replaced by a dynamic one showing the path of the last swipe:

The 1010 runs its own scetch (cimino.cpp), which calls another PHP script (fetch.php), which fetches the data of the oldest (smallest ID number) swipe in the database. After being read, the particular swipe data gets erased from the database. The script outputs the data in a readable form (for debugging purposes, needs the 1010 to be connected to a computer). The 1010 parses the output and starts performing it. The user may start the swipe at any point in the image, but the robot arm must start the swipe from a rest position, which is below the middle point of the arc of chimes. The arm starts from this position, then, in a slow pace, moves first horizontally, then vertically to the starting point of the swipe. After the swipe, the arm moves down, then returns to the rest position and continues reading the fetch.php script.

The database

I have a website on Hostinger.com that holds the html page and the PHP scripts as well as the MySQL database for the swipe data. The database contains one single table, "coords", that holds the data. The structure of the table is shown here:

One instance in the table is one touch event. One swipe creates hundreds of touch events.

ndx is a unique number for each added touch event. Since the swipes are deleted from the table after being read, the table might become empty, if no new swipes are added. In that case the numbering starts again from 0. This data field serves two purposes. One is for including a key field in the table (may or may not be required), one is for enabling sorting of the events (again, may or may not be required).id is a number identifying which swipe the event belongs to.x, y are the coordinates of the touch event.timestamp is the time in milliseconds for the touch event, always starting at 0 for each new swipe.

Calibrating the movements

The image in the user interface is just a simple image of chimes. The chimes in my Cimini have slightly different dimensions. But it's important that each significant point in the image corresponds to the same point in the Cimini. If I swipe my finger along the bottom end of the chimes in the image, the arm should touch the botom ends of the chimes in the Cimini. If I swipe my finger to the left or the right end without touching the chimes, the arm should act similarily.

BOM

  • Arduino MKR WiFi 101027 $
  • Pan-tilt device with servos20 $
  • Wood parts, paint, glue, screws...0 $ - 10 $ (depending on what you have, what you have to get)
  • Chrome plated copper tube40 $
  • Copper thread3 $
  • Ornaments15 $ - 30 $

Additional costs for tools you might need to buy and web hosting services.

Some further ideas and development

When I lift the instrument ad move it around, the chimes start to swing and make a very beautiful sound. When the arm plays the chimes, I don't always get the same effect. My next build will have a small detail, which hopefully will add to the swing motion. Each of the four poles supporting the ceiling plate, will have a slightly rounded top, a cylindrical (or conical) surface with the axis pointing to the centre of the arc of chimes. This surface would be in both ends. This will allow the pole to slightly sway, making the ceiling plate twist back and forth, prolonging the swinging of the chimes. The screws or bolts in both ends of the poles would have rubber washers allowing the swaying, while still keeping everything steady enough. Every extra swing energy will hopefully go to the chimes swinging in the right direction, hitting each other.

The Cimini is complete as it is in this demonstration. But it has "hand made" written all over it! And as such, it is not ready for mass production. Each detail needs development. All software needs polishing and needs to be written in a generic style, easy to be changed by any user wanting to adapt it to their purposes. All hardware, both material and sizes, needs to be adapted to available items at stores.

The software

The concept of having the user just use their phone browser and not downloading any app, not logging into any special network, means ease for the user and more work for the owner of the Cimini. The owner must put up the web service on a site allowing PHP scripts and MySQL. To make it easier for the owner, the whole concept has to be re-designed. The user could have direct contact between the phone and the 1010.

Also, more details should be included in the user interface, where the user would have to enter some codes to allow access, and the owner could administrate the usage through an admin page.

The wooden parts

The bottom plate, as well as the ceiling plate could be ordered from the local wood supplyer, as ready cut plywood. The round corner poles could be of same measure as the piedestal that the pan-tilt device stands on, some standard measured material (either round or rectangular) that one could cut easily in desired lengths.

The ceiling plate needs the holes for the string holding the chimes. If the plate is cut with a CNC machine (laser or milling), the same machine could be programmed to make the holes.

The pan-tilt device and the arm

The pan-tilt device, the arm and the tip touching the chimes should need further development. It should be 3D printed so that the top most element would have the tilt axis (attached to the servo), the arm and the tip touching the chimes integrated in one print.

And while we're 3D printing, the front façade decorations could be designed and printed in various styles - barocque, classic, art-nouveau...

The chimes

The chrome plated copper tube really has a heavenly sound and there's no need to replace it. But to speed up the process of cutting the pieces, one would need a miter saw capable of cutting the copper tube at a 0.1 mm precision, as well as a pillar drill. The pieces still need to be tuned by hand, but the less one has to file off to get the right frequency, the faster the job will get done. The thread needs to be "sewed" in place by hand, but it would help to have a wooden arc placed between the ceiling plate and the chimes to get the right distance for each chime. This arc could as well be 3D printed.

The electronics

The servos definitely need to be chosen to be as silent as possible. Just for the case of elegancy, the 1010 should have a 3D printed case of its own, with just the micro USB contact sticking out. The whole system could therefore still be powered with a simple power bank or a phone charger attached to the wall.

To cut expenses, further development should be done to replace the Arduino (which really is just an evaluation board) with just the minimum of components soldered to a PCB.

A final product

So you would like to buy a Cimini to place it in your café, to let your customers enjoy playing with it. You would receive a package with all the parts. The bottom plate would have the legs attached, as well as the microcontroller in its own case. You would attach the pan-tilt arm to its place on the bottom plate with a bolt and a wing nut from the bottom. The four poles would likewise attach to their places with bolts and wing nuts. Then you place the ceiling plate with all chimes readily attached in their threads, onto the four poles. The plate has guiding pegs and the poles have holes for the pegs. Finally you snap on the 3D printed decoration, which locks on to the front side of the ceiling plate.

You connect a 5 V USB phone charger to the microcontroller underneath the bottom plate and you are ready to go. Just wait for the network to show up on your list of wireless networks on your phone and browse to the control page. Or choose the longer route, reprogram the 1010 to use a website on internet, to make it easier for anyone to operate the Cimini.

Code

fetch.phpPHP
This script reads the table in the database and finds the swipe with the lowest ID and outputs the coordinates and the timestamps. If the table is empty, a single letter a is outputted. The event output starts with a single letter b and ends with a single letter c. All the output is in readable html format for debugging purposes. One line contains comma separated data with the x-coordinate, the y-coordinate and the timestamp of one event. A line break is created with <br /> instead of <p></p> to avoid an annoying double line feed, which I got every time I wanted to paste the data from the page to a spreadsheet for analysis.
The MySQL query in 20-22 should include sorting of the data according to the event index number! But my code simply relies on the data being written and read in the same order as the event got created in the first place. It has worked so far.
<html>
 <head>
  <title>List of coordinates</title>
 </head>
 <body>
 <?php 
    

    $servername = "localhost";
    $username = "my_secret_username";
    $password = "my_secret_password";
    $dbname = "my_database_name";

    $conn = mysqli_connect($servername, $username, $password, $dbname);

    $rowSQL = mysqli_query( $conn, "SELECT MIN( id ) AS min FROM `coords`;" );
    $row = mysqli_fetch_array( $rowSQL );
    $smallestId = $row['min'];

    $sql = "SELECT x,y,timestamp FROM coords WHERE id='".$smallestId."' ";

    $result = mysqli_query($conn, $sql);
    
    if (mysqli_num_rows($result) > 0) {
        echo "b<br />\n";
        while($row = mysqli_fetch_assoc($result)) {
            echo $row["x"] . ", " . $row["y"] . ", " . $row["timestamp"] . "<br />\n";
        }
        echo "c<br />\n";
        // Delete the rows just shown!
        $sql = "DELETE FROM coords WHERE id='".$smallestId."' ";
        $result = mysqli_query($conn, $sql);
      
    }
    else
    {
        echo "a<br />\n";
    }


 ?> 
 </body>
</html>
cimini.cppC/C++
This is the scetch the Arduino 1010 runs to operate the Cimini. It relies on the software created by Tom Igoe and Federico Vanzati dealing with the WiFi and Internet stuff, as well as on the Servo library.
I've left a lot of lines with serial printing, as well as MyPlot, for debug purposes. They were very handy. I'm happy to discuss them further with anyone.
/*
 *    Cimini, a chimes instrument controlled via 
 *    a smartphone and a website
 */

/*

 This sketch connects to a a web server and makes a request
 using a WiFi equipped Arduino board.

 created 23 April 2012
 modified 31 May 2012
 by Tom Igoe
 modified 13 Jan 2014
 by Federico Vanzati

 http://www.arduino.cc/en/Tutorial/WifiWebClientRepeating
 This code is in the public domain.
 */

/*
 *    TESTING COORDS
 *    xmin = 40
 *    xmax = 940
 *    leftmost bar = 160
 *    rightmost bar = 850
 *    leftbottom = 435
 *    rightbottom = 210
 *    highestpoint = 130
 * 
 */
 
#include <SPI.h>
#include <WiFiNINA.h>
#include <utility/wifi_drv.h>
#include <MegunoLink.h>
#include <Servo.h>

#include "arduino_secrets.h" 
///////please enter your sensitive data in the Secret tab/arduino_secrets.h
char ssid[] = SECRET_SSID;        // your network SSID (name)
char pass[] = SECRET_PASS;    // your network password (use for WPA, or use as key for WEP)
int keyIndex = 0;            // your network key Index number (needed only for WEP)

int status = WL_IDLE_STATUS;
//using namespace std;
// Initialize the Wifi client library
WiFiClient client;
XYPlot MyPlot;
Servo horis, vertis;

// server address:
char server[] = "johanhalmen.xyz";
//IPAddress server(64,131,82,241);

unsigned long lastConnectionTime = 0;            // last time you connected to the server, in milliseconds
unsigned long postingInterval = 5L * 1000L; // delay between updates, in milliseconds

// class for one touch event
class triplet
{
  public:
    unsigned short x;
    unsigned short y;
    unsigned int timeStamp;
    triplet(unsigned short xin, unsigned short yin, unsigned int ts); 
    triplet *next;
    
};

triplet::triplet(unsigned short xin, unsigned short yin, unsigned int ts)
: x(xin), y(yin), timeStamp(ts)
{
  
}

// class to hold one swipe
class batonqueue
{
  public:
    batonqueue(void);
    triplet *queue, *last;  
    void add_instance(unsigned short x, unsigned short y, unsigned int timeStamp);
    void perform(void);
    void servo_action(unsigned short x, unsigned short y);

};

batonqueue::batonqueue(void)
{
  queue = NULL;
  last = NULL;
}

// The RGB LED of the Arduino Mkr 1010 Wifi
#define r_pin 26
#define g_pin 25
#define b_pin 27

batonqueue myQueue;

void failout(void)
{
  // In error cases, blink the RGB LED until power is off
  while (true)
  {
    if (millis() % 1000 > 500)
    {
      WiFiDrv::digitalWrite(r_pin, 1);
      WiFiDrv::digitalWrite(g_pin, 0);
      WiFiDrv::digitalWrite(b_pin, 0);
    }
    else
    {
      WiFiDrv::digitalWrite(r_pin, 0);
      WiFiDrv::digitalWrite(g_pin, 1);
      WiFiDrv::digitalWrite(b_pin, 1);
    }
  }
}


// Add one event to the queue
void batonqueue::add_instance(unsigned short x, unsigned short y, unsigned int timeStamp)
{
  triplet *myNew;
  myNew = new triplet(x, y, timeStamp);
  if (myNew == NULL)
    failout(); // oops, memory is full!
  myNew->next = NULL;
  if (queue == NULL)
    queue = myNew;
  else
    last->next = myNew;
  last = myNew;
  
}

// Start performing the action held in queue
void batonqueue::perform(void)
{
  unsigned int last_stamp, wait_time;
  static unsigned short last_x = 0, last_y = 0;
  unsigned short divisor;
  float dx = 2.5;
  float dy = 2.5;
  int servo_delay = 5;
  float t;
  triplet *i, *j;
  // sweep from some resting middle point
  unsigned long start_time;
  float sqlength, spead, tn, timeadd;

  unsigned long accu_time = 0;
  
  // Adjust the speed. Must not exceed 1 pixel per millisecond.
  // I'm assuming the unit used in the event coordinates is one pixel.
  for(i = queue; i != NULL; i = i->next)
  {
    if (i->next != NULL)
    {
      // calculate length to next point
      sqlength = sqrt(float((i->x - i->next->x) * (i->x - i->next->x) + /// Fixed this!
                 (i->y - i->next->y) * (i->y - i->next->y)));
      // calculate speed to next point, should be 1.
      spead = sqlength / (i->next->timeStamp - i->timeStamp);
      if (spead > 1.)
      {
        // set the speed to 1 by adding time to all the remaining events
        tn = spead * (i->next->timeStamp - i->timeStamp);
        timeadd = tn - (i->next->timeStamp - i->timeStamp);
        for (j = i->next; j != NULL; j = j->next)
          j->timeStamp += (int)timeadd;
      }
    }
  }
  // Now the time stamps have been stretched where needed, to
  // limit the speed to corresponding to 1 pixel per millisecond.
  // THE MAX SPEED IS HARD CODED TO 1 PIXEL PER MILLISECOND. 
  // CHANGE CODE TO ALLOW EASY ADJUST OF MAX SPEED.

  // Move from last position to new start, first horiz, then verti

  if (queue->x > last_x)
  {
    for (float i = last_x; i < queue->x; i += dx)
    {
      servo_action(int(i), 460);
      delay(servo_delay);
    }
  }
  else
  {
    for (float i = last_x; i > queue->x; i -= dx)
    {
      servo_action(int(i), 460);
      delay(servo_delay);
    }
  }
  
  for (float i = 460; i > queue->y; i -= dy)
  {
    servo_action(queue->x, int(i));
    delay(servo_delay);
  }   
  start_time = millis();
  last_stamp = 0;

  // START PERFORMING
  for(i = queue; i != NULL; i = i->next)
  {
    if (i->timeStamp - last_stamp > 40)
    { // divide the steps to 20 ms long steps
      divisor = (i->timeStamp - last_stamp) / 20; // integer division rounds down (hopefully)
      for (int ii = 1; ii <= divisor; ii++)
      {
        t = float(ii) / divisor;
        servo_action(t*i->x + (1.-t)*last_x, t*i->y + (1.-t)*last_y);
        wait_time = t*i->timeStamp + (1.-t)*last_stamp;
        while (millis() < start_time + wait_time);
      }
    }    
    else
    { // just follow the original timesteps, they are small enough
      servo_action(i->x, i->y);
      while (millis() < start_time + i->timeStamp);
    }
    last_stamp = i->timeStamp;
    last_x = i->x;
    last_y = i->y;
    
  }
  // Move down
  for (float i = last_y; i <= 460; i += dy)
  {
    servo_action(last_x, int(i));
    delay(servo_delay);
  }   
  

  // Ok, we're finished with the action. Delete the queue.
  i = queue;
  do
  {
    j = i->next;
    delete i;
    
    i = j;
  }
  while (i != NULL);
  queue = NULL;
}

// This function activates both servos for one coordinate
// This function converts the coordinate data to angle data for the servos
void batonqueue::servo_action(unsigned short x, unsigned short y)
{
/*
 *    TESTING COORDS
 *    xmin = 40
 *    xmax = 940
 *    leftmost bar = 160
 *    rightmost bar = 850
 *    leftbottom = 435
 *    rightbottom = 210
 *    highestpoint = 130
 * 
 */
  if (x < 40) x = 40;
  if (x > 940) x = 940;
  if (y < 130) y = 130;
  if (y > 460) y = 460;

    horis.write(map(x, 40, 940, 180, 10));
    vertis.write(map(y, 130, 460, 45, 3));
}

void setup() {
  WiFiDrv::pinMode(g_pin, OUTPUT);  //GREEN
  WiFiDrv::pinMode(r_pin, OUTPUT);  //RED
  WiFiDrv::pinMode(b_pin, OUTPUT);  //BLUE

  horis.write(90); // First set the start position...
  vertis.write(1);
  horis.attach(4); // ...then start. Servo turns to its start position.
  vertis.attach(5);
  
  //Initialize //Serial and wait for port to open:
  //Serial.begin(9600);
  //while (!Serial) {
    //; // wait for serial port to connect. Needed for native USB port only
  //}

  // check for the WiFi module:
  if (WiFi.status() == WL_NO_MODULE) {
    MyPlot.SetTitle("Communication with WiFi module failed!");
    // don't continue
    while (true) {};
  }

  String fv = WiFi.firmwareVersion();
  if (fv < WIFI_FIRMWARE_LATEST_VERSION) {
    ;
    MyPlot.SetTitle("Please upgrade the firmware");
  }

  // attempt to connect to Wifi network:
  while (status != WL_CONNECTED) {
    MyPlot.SetTitle("Attempting to connect to SSID: ");
    // Connect to WPA/WPA2 network. Change this line if using open or WEP network:
    status = WiFi.begin(ssid, pass);

    // wait 10 seconds for connection:
    delay(10000);
  }
  // you're connected now, so print out the status:
  printWifiStatus();

  MyPlot.SetTitle("My Analog Measurement");
  MyPlot.SetXlabel("Channel 0");
  MyPlot.SetYlabel("Channel 1");
  MyPlot.SetSeriesProperties("ADCValue", Plot::Magenta, Plot::Solid, 2, Plot::Square);
  MyPlot.SetXRange(0, 900);
  MyPlot.SetYRange(0, 900);
  

}

// Get one line from the feed from the fetch.php call
String get_next_line(void)
{
  String hlp = "";
  char c;
  do
  {
    if (client.available())
    {
      c = client.read();
    
      if (c >= 32)
        hlp += c;
    }
  }
  while (c != 10 && c != 13 && client.connected());
  return hlp;
}

// Get the crucial data from one line (x, y, timestamp)
// and add to the queue
void add_to_queue(String line)
{
  int second, first;
  int x, y, timeStamp;
  String hlp;
  
  if (line.endsWith("<br />"))
  {
    second = line.lastIndexOf(',');
    if (second == -1)
      return;
    hlp = line.substring(second+1);
    timeStamp = hlp.toInt();
    hlp = line.substring(0, second);
    first = hlp.lastIndexOf(',');
    if (first == -1)
      return;
    hlp = hlp.substring(first+1);
    y = hlp.toInt();
    hlp = line.substring(0, first);
    x = hlp.toInt();
    myQueue.add_instance(x, y, timeStamp);
    
  }
  return;
}

void perform_the_queue(void)
{
  MyPlot.Clear();
  myQueue.perform();
}

void loop() {
  String line;
  char mode = 0;  // 0: read line
                  //    if a<br />
                  //      make a new request (mind waiting)
                  //    if b<br />
                  //      set mode to 1, reset queue
                  // 1: read lines until c<br />
                  //    if line has two commas and end with <br />,
                  //      add to queue
                  //    if c<br />, set mode to 2
                  // 2: perform the queue
                  //    make a new request
                  //    set mode to 0
  WiFiDrv::digitalWrite(r_pin, 1);
  WiFiDrv::digitalWrite(g_pin, 0);
  WiFiDrv::digitalWrite(b_pin, 0);

  while (true)  // This is the loop of the scetch, not loop()!
                // The loop runs in three modes, according to
                // which stage the system is in (see comments in line 353-)
  {
    if (!client.available())
    {
      WiFiDrv::digitalWrite(r_pin, 1);
      WiFiDrv::digitalWrite(g_pin, 0);
      WiFiDrv::digitalWrite(b_pin, 0);
      while (millis() - lastConnectionTime < postingInterval); // Wait 10 s before next request
      httpRequest();
      mode = 0;
      // reset queue
    }
    switch(mode)
    {
      case 0 : // waiting for a new list to be read
        line = get_next_line();
        //Serial.println(line.c_str());
        if (line.equals(" a<br />"))  // Had accidentally a space there
        { 
          postingInterval += 250;
          if (postingInterval > 10000)
            postingInterval = 10000;
          while (millis() - lastConnectionTime < postingInterval); // Wait 10 s before next request
          httpRequest();
        }
        if (line.equals(" b<br />"))  // Had accidentally a space there
        {
          mode = 1;
          //Serial.println("mode 1");
          WiFiDrv::digitalWrite(r_pin, 0);
          WiFiDrv::digitalWrite(g_pin, 0);
          WiFiDrv::digitalWrite(b_pin, 1);
        }
        delay(500);
        break;
      case 1 : // reads lines from the list and adds to the queue
        line = get_next_line();
        if (line.equals("c<br />"))
        {
          mode = 2;
          //Serial.println("mode 2");
          WiFiDrv::digitalWrite(r_pin, 0);
          WiFiDrv::digitalWrite(g_pin, 1);
          WiFiDrv::digitalWrite(b_pin, 0);
          delay(500);
        }
        else
        {
          add_to_queue(line); // the called function does validity check
        }
        break;
      case 2 : // queue is complete and ready to be performed
        perform_the_queue(); // this keeps the Arduino occupied, no problem with that
        WiFiDrv::digitalWrite(r_pin, 1);
        WiFiDrv::digitalWrite(g_pin, 0);
        WiFiDrv::digitalWrite(b_pin, 0);
        postingInterval = 500;
        while (millis() - lastConnectionTime < postingInterval); // Wait before next request
        httpRequest();
        
        mode = 0;
        
        break;
    }
  }


}



// this method makes a HTTP connection to the server:
void httpRequest() {
  // close any connection before send a new request.
  // This will free the socket on the Nina module
  client.stop();

  // if there's a successful connection:
  if (client.connect(server, 80)) {
    //Serial.println("connecting...");
    // send the HTTP PUT request:
    client.println("GET /cimini/fetchfi.php HTTP/1.1");
    client.println("Host: johanhalmen.xyz");
    client.println("User-Agent: ArduinoWiFi/1.1");
    client.println("Connection: close");
    client.println();

    // note the time that the connection was made:
    lastConnectionTime = millis();
    //Serial.println("connected");
    
  } else {
    // if you couldn't make a connection:
    //Serial.println("connection failed");
  }
}


void printWifiStatus() {
  IPAddress ip = WiFi.localIP();
}
index.htmlHTML
This is the interface page, which the user gets by scanning a QR code. The code contains HTML and Javascript.

Lines 4-8
Avoid image getting read from cache. The image is dynamically created, which the browser might not recognise.

Lines 23-29
Defines the small textbox with info. If device is held in landscape orientation, this box won't be seen, but it doesn't contain any crucially important information.

Touch start, lines 61-
Reset everything for recording a new swipe.

Touch moved, lines 81-
Add each event to the array of coordinates and timestamps (punkter[]).

Touch end, lines 91-
Create a binary array to be sent to the receive.php script. When the POST is performed, redraw the user interface image of the chimes with the swipe path added (graphics.php). Line 130 adds a timestamp to the URL, the only purpose of which is to avoid the browser from loading a cached image.

Line 131 and 139-142
At some point I had the swipe path shown only for 3 seconds. After that, the whole page was reloaded.
<!DOCTYPE html>
<html>
<head>
<meta Http-Equiv="Cache-Control" Content="no-cache">
<meta Http-Equiv="Pragma" Content="no-cache">
<meta Http-Equiv="Expires" Content="0">
<meta Http-Equiv="Pragma-directive: no-cache">
<meta Http-Equiv="Cache-directive: no-cache">    
<style>
body
{
    background-color:                   #582815;
    background-repeat:                  no-repeat;
    background-attachment:              fixed;
    height:                             1220px;
    width:                              100%;
    background-size: 100%;
    overflow:                           hidden;
    position:                           fixed;
    top:                                0px;
}

div.relative {
  position: fixed;
  top: 900px;
  border: 3px solid #784835;
  color: #ffdd80;
  font-family: Arial, Helvetica, sans-serif;
  font-size: 50px;
}

</style>

</head>
<body ontouchstart="myFunction(event)" ontouchmove="myFunction(event)"  ontouchend="myEndFunction(event)" overflow="hidden">
<img id="theimage" src="cimini.jpg?cachepreventer=10783662" width="100%" style="border:0px" />
<div class="relative" id="info">
    Drag your finger through the bars.
    
</div>

<script>

var punkter = new Array(5000);
var xy = new Array(3);
var n = 0;
var start_time = 0;
var breakflag = false;


function myFunction(event) {
  var x = Math.floor(event.touches[0].clientX);
  var y = Math.floor(event.touches[0].clientY);
  xy[0] = x;
  xy[1] = y;
  xy[2] = event.timeStamp - start_time;
  
  var now = event.timeStamp;
  
if (event.type == "touchstart") 
  {
    document.getElementById("theimage").src= "cimini.jpg";
    if (y < 800)
    {
        start_time = event.timeStamp;
        xy[2] = 0;
        punkter.length = 0;
        n = 0;
        punkter[n] = xy.slice(0, 3);
        document.getElementById("info").innerHTML = "touched\n";
        breakflag = false;
    }
    else
    {
        document.getElementById("info").innerHTML = "Please, start the dragging inside the image\n";
        breakflag = true;
    }
  }
  else 
  {
    if (event.type == "touchmove" && n < 4999 && breakflag == false && breakflag == false) 
    {
      n++;
      punkter[n] = xy.slice(0, 3);
      document.getElementById("info").innerHTML = "moving\n";
    }

  }
}

function myEndFunction(event) {
    if (breakflag == true)
    {
        document.getElementById("info").innerHTML = "breaked\n";
        return;
    }
        
    


  xy[0] = punkter[n][0];
  xy[1] = punkter[n][1];
  xy[2] = event.timeStamp - start_time;
  n++;
  punkter[n] = xy.slice(0, 3);
 
  document.getElementById("info").innerHTML = "stopped\n";    
    // send the binary data
    
    var myArray = new ArrayBuffer(6*n + 10);
    var longInt16View = new Uint16Array(myArray);
    
    // generate some data
    longInt16View[0] = 1; // endian check
    longInt16View[1] = n;
    for (i = 0; i <= n; i++) 
    {
      longInt16View[3*i + 2] = punkter[i][0];
      longInt16View[3*i + 3] = punkter[i][1];
      longInt16View[3*i + 4] = punkter[i][2];
    }
    
    
    var xhr = new XMLHttpRequest;
    xhr.open("POST", "receive.php", true);
    xhr.onload = function () {
        // Uploaded.
        var d = new Date();
        document.getElementById("info").innerHTML = "succeeded in sending\n";
        document.getElementById("theimage").src="graphics.php?nocache="+d.getTime();
        //setTimeout(resetImage, 3000);
    };
    xhr.send(longInt16View);
    

    
}

function resetImage() {
  
  document.location.reload();
}


</script>

</body>
</html>
receive.phpPHP
This script receives the binary data chunk sent from index.html after a swipe. It saves it in a MySQL database table. The table contains the following data:
$largestNdx - a unique number for each event in the whole table
$largestId - an ID number for all events belonging to one swipe
$x, $y, $timestamp - coordinates and timestamp for one event

Line 66-
A function that gets one 16 bit integer from the received $rawdata chunk.

The $rawdata chunk has the number 1 stored in the first word. I used this number at some point to check the endianness of the data created. After I found out that the endianness was the same in all tested systems, I deleted code related to endianness, but this word I left, because it was a part of the raw data structure everywhere. The second word contains the number of events (max 5000). After that, everything is event data.
<html>
 <head>
  <title>PHP Test</title>
 </head>
 <body>
 <?php 
    
    echo "<h1>Hello2</h1>";
    $rawdata =  file_get_contents("php://input");
    
    $servername = "localhost";
    $username = "my_secret_username";
    $password = "my_secret_password";
    $dbname = "my_database_name";

    $conn = mysqli_connect($servername, $username, $password, $dbname);

    // Get the highest ID of all swipes currently in the database
    $rowSQL = mysqli_query( $conn, "SELECT MAX( id ) AS max FROM `coords`" );
    if ($rowSQL != NULL)
    {
        $row = mysqli_fetch_array( $rowSQL );
        $largestId = $row['max'] + 1;
        $rowSQL = mysqli_query( $conn, "SELECT MAX( ndx ) AS max FROM `coords`" );
        $row = mysqli_fetch_array( $rowSQL );
        $largestNdx = $row['max'] + 1;
    }
    else
    {
        $largestId = 0;
        $largestNdx = 0;
    }
    // Now $largestID holds the ID number of the new swipe to be stored

    $pointer = 4;
    $nofpoints = fetchUint16(2);
    // Now we know how many points the swipe contains
    for ($i = 0; $i <= $nofpoints; $i++)
    {
        $x = fetchUint16($pointer);
        $pointer += 2;
        $y = fetchUint16($pointer);
        $pointer += 2;
        $timestamp = fetchUint16($pointer);
        $pointer += 2;
        
        $sql = "INSERT INTO coords (ndx,id,x,y,timestamp) VALUES ('$largestNdx','$largestId','$x','$y','$timestamp')";
        
      // All echos in the codes are for debugging purposes and don't 
      // show anywhere, when Cimini is in action
    	if (mysqli_query($conn, $sql)) 
    	{
		    echo "New record created successfully !";
	    } 
	    else 
	    {
		    echo "Error: " . $sql . "" . mysqli_error($conn);
	    }
	    $largestNdx++;
    }
    
    
    	
    mysqli_close($conn);
    
    function fetchUint16($offs)
    {
        global $rawdata;
        
        $array = unpack("v", $rawdata, $offs);
        return $array[1];
    }
    
 ?> 
 </body>
</html>

Schematics

The connections
This is as simple as it gets. There's the power supply for the Arduino. There are two servos to be connected. The servos use same 5 V source as the Arduino. The Arduino can be powered with a 5 V mobile phone power bank, as long as it stays activated by the Arduino power consumption. Or the Arduino could be powered with a phone charger connected to the wall. In my connection, the servos take the power from the +5V pin of the Arduino, while the Arduino takes power from the USB connector.
Capture ewhvgtsan9

Comments

Similar projects you might like

Hookah Lighting

Project tutorial by Turai Botond

  • 3,968 views
  • 0 comments
  • 6 respects

Quarantine Enforcer

Project tutorial by Gaurav_Tag

  • 24 views
  • 0 comments
  • 1 respect

WiFi IR Blaster

Project tutorial by BuddyC

  • 111,328 views
  • 45 comments
  • 112 respects

Water Quality Monitoring Using MKR1000 and ARTIK Cloud

Project tutorial by Team Animo!

  • 13,699 views
  • 14 comments
  • 37 respects

Arduino Game Controller

Project tutorial by Elektro Punkz and Arnov Sharma

  • 207 views
  • 0 comments
  • 4 respects

Distance Monitoring mats

Project tutorial by Sujata

  • 63 views
  • 0 comments
  • 1 respect
Add projectSign up / Login