Project showcase
Classify Candy in Free Fall Using TinyML

Classify Candy in Free Fall Using TinyML © CC BY

Using the Arduino KNN library to classify the color of M&Ms we throw at it.

  • 1,748 views
  • 3 comments
  • 14 respects

Components and supplies

Necessary tools and machines

About this project


The Arduino KNN library offers a way to include some simple machine learning into your Arduino sketch quickly and easily.

Here's a quick project created to test how fast the color sensor sampling and inferencing can work (it turns out fast enough to classify candy in free fall!). One major difference between this sketch and the example color classifier in the library is it does not use proximity to detect objects, but instead checks for variations in light level - making it suitable for use in our tube.

With an Arduino Nano BLE Sense, some paper, scissors, tape and a bag of candy, you are able to create an M&M color classifier using TinyML.

Construction

This requires some paper, scissors and tape:

  • Create a paper tube just bigger than the candy that has to pass through
  • Cut a small window enough for the LEDs and sensor on the BLE Sense board
  • Tape the Arduino to the paper tube
  • Create a cone for the top so you can throw candy in there

Operation

Run the sketch and throw candy into the hopper. The positioning of the board and controlling the lighting conditions have a big impact on accuracy. I found a darkened room to be better (but not so dark you can't see where to throw the candy).

Emojis for Linux terminal output

You can use emojis for the object names in your sketch. In Linux it's then easy to view the serial output from your Arduino so you can see a log of what candy it saw for verification purposes.

Enjoy!

Code

Arduino sketchArduino
Classify object without using proximity, using illumination from the Arduino onboard LEDs
/*

  k-NN color classification
  -------------------------

  This sketch classifies objects using a color sensor.

  First you 'teach' the Arduino by putting an example of each object close to the color sensor.
  After this the Arduino will guess the name of objects it is shown based on how similar
  the color is to the examples it has seen.

  This example uses a simple case of k-Nearest Neighbour (k-NN) algorithm where k=1.

  HARDWARE: Arduino Nano BLE Sense

  USAGE: Follow prompts in serial console. Move object close to the board to sample its color, then move it away.

  Works best in a well lit area with objects of different colors.

  NOTE: Make sure Serial Monitor's line ending setting is configured for "Newline" or "Both NL & CR".


  This example code is in the public domain.

*/

#include <Arduino_KNN.h>
#include <Arduino_APDS9960.h>

const int INPUTS = 3; // Classifier input is color sensor data; red, green and blue levels
const int CLASSES = 3; // Number of objects we will classify (e.g. Apple, Banana, Orange)
const int EXAMPLES_PER_CLASS = 1; // Number of times user needs to show examples for each object

// K=1 means the classifier looks for the single closest color example it's seen previously
const int K = 1;

// Create a new KNNClassifier
KNNClassifier myKNN(INPUTS);

// Names for each class (object type)
String label[CLASSES];

// Array to store data to pass to the KNN library
float color[INPUTS];

// Threshold for color brightness
const float THRESHOLD = 0.98;
int initialAmbient = 0;

void setup() {

  Serial.begin(9600);
  while (!Serial);

  // Pins for the built-in RGB LEDs on the Arduino Nano 33 BLE Sense
  pinMode(LEDR, OUTPUT);
  pinMode(LEDG, OUTPUT);
  pinMode(LEDB, OUTPUT);

  if (!APDS.begin()) {
    Serial.println("Failled to initialized APDS!");
    while (1);
  }

  initialAmbient = readAmbient();

  Serial.println("Arduino k-NN color classifier");

  // Ask user for the name of each object
  for (int currentClass = 0; currentClass < CLASSES; currentClass++) {

    Serial.println("Enter an object name:");
    label[currentClass] = readName();

    // Ask user to show examples of each object
    for (int currentExample = 0; currentExample < EXAMPLES_PER_CLASS; currentExample++) {

      Serial.print("Show me an example ");
      Serial.println(label[currentClass]);

      // Wait for an object then read its color
      readColor(color);

      // Add example color to the k-NN model
      myKNN.addExample(color, currentClass);

    }
  }
}


void loop() {

  int classification;

  // Wait for the object to move away again
  while (!APDS.proximityAvailable() || APDS.readProximity() == 0) {}

  Serial.println("Let me guess your object");

  // Wait for an object then read its color
  readColor(color);

  // Classify the object
  classification = myKNN.classify(color, K);

  // Print the classification
  Serial.println(label[classification]);
  Serial.println();

}

int readAmbient() {
  int red, green, blue, ambient;
  while (!APDS.colorAvailable()) {};
  APDS.readColor(red, green, blue, ambient);
  return ambient;
}

void readColor(float color[]) {
  int red, green, blue, ambient = 0, colorTotal = 0;

  // Wait until shadow of object passing
  while (readAmbient() > initialAmbient * THRESHOLD) {}

  // Wait until we have a color bright enough
  while (ambient < initialAmbient * THRESHOLD) {

    // Sample if color is available and object is close
    if (APDS.colorAvailable()) {

      // Read color and proximity
      APDS.readColor(red, green, blue, ambient);
      colorTotal = (red + green + blue);
    }
  }

  // Normalise the color sample data and put it in the classifier input array
  color[0] = (float)red / colorTotal;
  color[1] = (float)green / colorTotal;
  color[2] = (float)blue / colorTotal;

  // Print the red, green and blue percentage values
  Serial.print(color[0]);
  Serial.print(",");
  Serial.print(color[1]);
  Serial.print(",");
  Serial.println(color[2]);
}


// reads a name from the Serial Monitor
String readName() {
  String line;

  while (1) {
    if (Serial.available()) {
      char c = Serial.read();

      if (c == '\r') {
        // ignore
        continue;
      } else if (c == '\n') {
        break;
      }

      line += c;
    }
  }

  return line;
}

Comments

Similar projects you might like

Cough Detection with TinyML on Arduino

Project tutorial by Edge Impulse

  • 5,186 views
  • 5 comments
  • 35 respects

Candy Dispenser with Google Assistant

Project tutorial by Arduino “having11” Guy

  • 18,801 views
  • 1 comment
  • 68 respects

GLaDOS Genesis

Project showcase by Little_french_kev

  • 2,559 views
  • 5 comments
  • 20 respects

Neuronium

Project showcase by Astrid Kraniger

  • 2,446 views
  • 1 comment
  • 17 respects

No Candy For YOU!

Project tutorial by Jonathan Tindal

  • 1,891 views
  • 5 comments
  • 8 respects

Gesture Recognition Using Accelerometer and ESP

Project tutorial by mellis

  • 48,732 views
  • 27 comments
  • 85 respects
Add projectSign up / Login