Project showcase
AI and Machine Learning for Kids

AI and Machine Learning for Kids © GPL3+

Teach ALL kids the fundamentals of AI and machine learning, through games and hardware and software design.

  • 5,461 views
  • 0 comments
  • 16 respects

Components and supplies

About this project

Our mission when creating this project was :

• Teach ALL kids the fundamentals of AI and machine learning

• Bring AI to life with games and hardware and software creation activities

• Send kids home with a computer they assembled and a machine learning application they trained

So we made a summer camp and a whole lot of software and hardware!

• AI fundamentals distilled into hardware and software design

• Consciously designed to be understandable by using simple peripherals – LEDs, buttons, dials, simple text…

• AI application software built around the concept of training a robot to work in a Box Factory

• Kids took home the box they built with access to the software and teaching materials

We worked closely with a middle school administration to identify talented kids who can not typically afford an expensive coding camp, found a corporate sponsor to underwrite the camp (many thanks Arm Inc!), and delivered the most advanced possible week-long computer science camp with no cost to the kids families.

How does it work?

Find out more at: www.fryden-learning.com

AI basics as a games: http://www.fryden-learning.com/be-the-machine

Advanced AI for kids: http://www.fryden-learning.com/fryden-machine-learning-class

Making the hardware: http://www.fryden-learning.com/fryden-assembly-instructions

The AI teaching application: http://www.fryden-learning.com/box-factory

...and most importantly the kids comments and testimonials:

Other games that teachers make, they would say it’s fun and educational but turns out to be really boring. I’m usually telling my parents that I hate summer schools because they take out time of my summer, but this time, I think it was really worth it.

I loved this camp because it is one of a kind due to the teachers and what we did like soldering, code an Arduino, and put together the computer and see the parts of one so over all I LOVED! My favorite part probably was the solding and the compettitions and also spending time with the classmates and teachers. Spending time with them was very funny and enjoyable.

I love building the fryden box because I like Lego.

I would say machine learning is the process of a machine learning from its mistakes and improving its accuracy at doing its objectives.

Well the camp was so fun is my favorite summer camp I had not be that fun since school ended, this camp is the best.

I loved soldering because it had a little danger to it

Errors are a good thing in machine learning because you can figure what which area your machine needs training on so it won’t make that error again

I have told all my friends about this but they all say it is bored and they rather stay at home but they are wrong, I spent a whole worth it week on this and I realllllllllllllllllly love it!

I was kinda of upset today because I was the sorter today and one of teammate was taking serious during the training mode and work mode worksheet so he kept saying do quick math in your head but I can’t do quick in my head but I tried my on the training mode and work mode, its just that I can’t do quick math in my head.

A one-dimensional problem could be a number line where you can split between the weights of steel weights(exercise weights.) A two-dimensional problem would be a graph of income on they axis and the months on the x axis. A three-dimensional problem would be a 3D graph of the world population, age, and the income of each person.

Code

fryden-due-boxfactory-camp-v1.76.inoC/C++
This is the main Arduino application used in: http://www.fryden-learning.com/box-facto
String verNum = "v1.76";
// FIXED camp errors

// *********TODO due
// use json or xml to send data to the simulator
// 
// test the serial tx state machine with random delays and noise
// fix reset mode - make sure everything resets so we dont have to pull the flash card
// (text for red to exit? on training/work modes)
// plan to prep all the flash cards...(delete settings file to ensure it boots)
// autodect i2c address and auto config on boot - not sure this is possible
// calibrate dials to 512 at the center
// screen save - bunches at the top?
// instructions text into the mode OR pull instructions mode.
// comment blocks above all functions and names at the top to search against
// boing score - capture 2 scorws based on length of rally and who dropped the ball
// NOT DONE - wav - files placeholders for all - welcome to the fryden audio
// NOT DONE - wav - audio interupt collisions when using the wavfile palyer and the tone lib....
// NOT DONE - wav - remember playwavs in the settings
// NOT DONE - tricky - shape the colors to make the mid case of orange better to see - less green...
// DONE - BUG - setting guild POT selects a blank option.
// DONE - BUG - entername - clear screen
// DONE - stop sending after 5 upload attemts
// DONE - redesign a safer state machine for the recieve bytes
// DONE - placeholder for welcome tune.
// DONE - all params to file ser/de-ser accross the serial link inc. glyphs
// DONE - names - ZAPPA SYNTH & boing
// DONE - make the binary save to file for if we want to save edited glyphs
// DONE - lock symbols over write the menu bar by wrapping around
// DONE - lock symbol appears on stettings menu
// DONE - ligthening spelt wrong
// DONE - screen save x secs after the menu interactions
// DONE - more phrases into the "thanks for teaching me" "yes i know..."
// DONE - pong sounds
// DONE - dial for 0.8 to 1.4 = be able to change learning rate and gbRadius by settings - gbradius makes the game harder
// DONE - make the glyph colors constants.
// DONE - basic sysnth code needs adding
// DONE - synth - make 2 on the joystick...
// DONE - screen save messages - code and vocab answers
// DONE - custom glyph on start screen & LED matrix.
// DONE - add the guild names to the name / description all the way through to the simluator / guild comes after naming
// DONE - enter custom name in settings -- // settings - add name from a file // add you own name for custom names - reset - if never set then set it 
// DONE - name flow - unnamed - must force naming after reset
// DONE - guild name into settings
// DONE - on power up and flash the guild symbols and say "I am..." of the
// DONE - clear the neo pixel on start up
// DONE - put Mr Brydens glyphs in.

// **********Notes and other less critical todo's
// BOER - needs a factory restet to be reprogrammed....
// Crash on processing APP if it reads corruot data - needs to recover
// upgrade two protos to work on PCB versions
// State of mind crashed for some reason - may have been the ~~ name markers?
// NOTE - 5 traing sets at 0.0001, 1.2 to get 100%
// design and doc the fastest test plan at this rate.
// add a factory reset - on boot - to reset the file system
// idelly make a split20 function for easier text creation - hard - needs to find spaces  
// lineraize joystick map potends 2-1020
// redo all the sounds to avoid low freq's // make new learning sound for the nn's
// change nn to perceptron - text and code
// make sure you cant add more than 16 menus or 32 pages lines - and test it
// find and remove all debug printlns // check the serial port for actvitity
// test SD card init fail message on LCD
// function grouping, coding, commenting and tidying
// go all through delay and costants in the code and #define
// Perceprotn notes: prediction 1 = large red    -1 = small green
// box size/color ranges
// boxSize = 0..31 ; 32 steps ; 8 leds x 4 level of brigthness 
// colVal = max 32 + 128 = 32..159 ; works best for neopixel display 

String boxName = "BRAIN BOX";
String boxGuild = "NINJA";

//Due board/shield IO
#define potLeft A0
#define potCenter A1
#define potRight A2
#define redButton 4
#define greenButton 3
#define blueButton 2
#define joyH A4
#define joyButton 5
#define joyV A5
#define ampEnable A3
#define sdCs 38
#define neoPinMatrix 7 
#define neoPinSingle 6 

// app params 
#define buttonReleaseDelay 150
#define LCDscrollSpeed 200
#define NEOscrollSpeed 75
#define makeBoxDuration 2000
#define makeBoxDurationBy2 1000
//#define makeSound true
#define analyzeBoxDuration 1500
//#define analyzeBoxDurationBy2 750
#define trainBoxDuration 2000
//#define trainBoxDurationBy2 750
#define cMax 400
#define learningRate 0.0001

#include <Wire.h>
#include <LiquidCrystal_PCF8574.h>
#include <Adafruit_NeoMatrix.h>
#include <Adafruit_NeoPixel.h>
#include <SD.h>
#include <SPI.h>
#include <Audio.h>

enum potPostion {
  left,
  right,
  center
}; 

enum buttonColor {
  blue,
  red,
  green,
  joy,
  any
}; 
//JF-
LiquidCrystal_PCF8574 lcd(0x3F);  // set the LCD address
//LiquidCrystal_PCF8574 lcd(0x27);  // set the LCD address

Adafruit_NeoMatrix matrix = Adafruit_NeoMatrix(8, 8, neoPinMatrix,
  NEO_MATRIX_LEFT     + NEO_MATRIX_TOP +
  NEO_MATRIX_ROWS + NEO_MATRIX_PROGRESSIVE,
  NEO_GRB            + NEO_KHZ800);

Adafruit_NeoPixel leds = Adafruit_NeoPixel(1, neoPinSingle, NEO_RGB + NEO_KHZ800);

unsigned long scrollNEOMilliSec;
String scrollNEOMessage = "............................................";
int scrollNEOMessageReset;
int scrollNEOMessagePos;
uint16_t scrollNEOColor;

unsigned long lastPressed;

float goodBoxRadius = 1.1;
int trainCycles = 0;
int totalTrainingExamples = 0;
int totalCorrect = 0;
float workModeAverage = 0;
int workModeBest = 0;

// menu & text-pages globals
unsigned long lastRead = 0;
enum joyVDir {
  down,
  up,
  middle
}; 
String bs = "                      ";
joyVDir posNow = middle;
joyVDir posLast = middle;
int entrySel=0;
int menuNumEntries=0;
String menuEntries[16] = {bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs};;
String menuHeading;
boolean menuUpdateLCD;
int pageNumEntries=0;
String pageEntries[32] = {bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs,bs};
String pageHeading;
boolean pageUpdateLCD;

// initialize the neural net waits and learning constant
float nnWeights[3] = {-1.0,1.0,0.0};
float nnLearningC = learningRate;

// main menu options
#define mainMenuSize 9
//"-----------------" 17 char ref
String mainMenuEntries[mainMenuSize] = {
  "Show brain state",
  "Training mode",
  "Work mode",
  "Upload to factory",
  "Analyze mode",
  "Play Boing",
  "Zappa Synth",
  "Instructions",
  "Settings"
};

boolean menuLocks[mainMenuSize] = {0,0,0,0,0,1,1,0,0};
//boolean menuLocks[mainMenuSize] = {0,0,0,0,0,0,0,0,0};
#define pongLock 5
#define synthLock 6

//"-----------------" 17 char ref
#define settingsMenuSize 6
String settingsMenuEntries[mainMenuSize] = {
  "Volume",
  "Set difficulty",
  "Name the brain",
  "Reset brain",
  "About",
  "Main menu"
};

enum modeType {
  welcome,
  work,
  brainstate,
  menu,
  train,
  analyze,
  factory,
  settings,
  instructions,
  pong,
  synth
};
modeType mode = welcome;
String modeDesc; 

#define SINE_SAMPLES 882 // 44.1khz / 50hz = 882 samples need for a full 50 hz sine wave
uint16_t sineWave[SINE_SAMPLES];
// the phase accumulator points to the current sample in our wavetable
uint32_t ulPhaseAccumulatorA = 0;
uint32_t ulPhaseAccumulatorB = 0;
// the phase increment controls the rate at which we move through the wave table
// higher values = higher frequencies
volatile uint32_t ulPhaseIncrementA = 0;   // 32 bit phase increment, see below
volatile uint32_t ulPhaseIncrementB = 0;   // 32 bit phase increment, see below
// full waveform = 0 to SAMPLES_PER_CYCLE
// Phase Increment for 1 Hz =(SAMPLES_PER_CYCLE_FIXEDPOINT/SAMPLE_RATE) = 1Hz
// Phase Increment for frequency F = (SAMPLES_PER_CYCLE/SAMPLE_RATE)*F
#define SAMPLE_RATE 44100.0
#define SAMPLES_PER_CYCLE 882
#define SAMPLES_PER_CYCLE_FIXEDPOINT (SAMPLES_PER_CYCLE<<20)
#define TICKS_PER_CYCLE (float)((float)SAMPLES_PER_CYCLE_FIXEDPOINT/(float)SAMPLE_RATE)
#define MIDI_NOTES 128
uint32_t nMidiPhaseIncrement[MIDI_NOTES];
float toneMixA;
float toneMixB; 
boolean playDual = false;

unsigned long tTime;
int tDuration;
boolean toneActive = false;

boolean sdInit = false;

#define paramLen 40
byte params[paramLen];
boolean loadedSettings = false;

int volume = 3;

String guildNames[3] = {"NINJA","PHOENIX","LIGHTNING"};

const uint32_t gRed = matrix.Color(128,0,0);        // R 
const uint32_t gOrange = matrix.Color(120,40,2);    // O
const uint32_t gYellow = matrix.Color(100,80,0);    // Y
const uint32_t gBlue = matrix.Color(0,0,128);       // B
const uint32_t gPurple = matrix.Color(128,0,128);   // P
const uint32_t gGreen = matrix.Color(0,128,0);      // G
const uint32_t gTeal = matrix.Color(32,64,64);      // T
const uint32_t gBlack = matrix.Color(0,0,0);        // B
const uint32_t gWhite = matrix.Color(100,80,100);   // W

char guildGlyphs[3][8][8] = {{
  {'O','O','O','O',' ',' ',' ','O'},
  {' ','O','O','O','O',' ','O','O'},
  {' ',' ','O','O','O','O','O','O'},
  {' ','O','O',' ',' ','O','O','O'},
  {'O','O','O',' ',' ','O','O',' '},
  {'O','O','O','O','O','O',' ',' '},
  {'O','O',' ','O','O','O','O',' '},
  {'O',' ',' ',' ','O','O','O','O'}
},{
  {' ',' ',' ',' ',' ',' ',' ',' '},
  {' ',' ',' ',' ','R',' ',' ',' '},
  {' ',' ',' ','R','R',' ',' ',' '},
  {' ','R',' ','R','R',' ','R',' '},
  {' ','R','R','R','O','R','R',' '},
  {' ','R','O','O','Y','O','R',' '},
  {' ','R','Y','Y','Y','Y','R',' '},
  {' ',' ','R','Y','Y','Y','R',' '}
},{
  {' ',' ',' ',' ','B','T','B',' '},
  {' ',' ',' ','B','T','B',' ',' '},
  {' ',' ','B','T','B',' ',' ',' '},
  {' ','B','T','T','T','T','T','B'},
  {'B','T','T','T','T','T','B',' '},
  {' ',' ',' ','B','T','B',' ',' '},
  {' ',' ','B','T','B',' ',' ',' '},
  {' ','B','T','B',' ',' ',' ',' '}
}};

int tile[8][8];
boolean moved[8][8];

byte lock[8] = {
  B01110,
  B11011,
  B10001,
  B10001,
  B11111,
  B11011,
  B11011,
  B11111
};

#define screenSaverTime 60000
#define saverTextMinShow 5000
#define saverTextNum 33
String saverText[33][4] = {
  {"PERCEPTRON: is a one","NODE NEURAL NETWORK","that can CLASSIFY", "LINEAR SEPARATE DATA"}, 
  {"CLASSIFY: sort data","into different","categories or types","by giving it a name"},
  {"LINEAR SEPARATE","DATA: data that can","separated by a line","drawn on a graph"}, 
  {"NEURAL NETWORK: is a","simple model of how","biological neurons","cooperate in a brain"},
  {"NEURAL NETWORK: can","learn by TRAINING to","perform tasks by","being shown examples"},
  {"NEURAL NETWORK: is a","network of NODES","that are connected","together by WEIGHTS"},
  {"WEIGHT MEMORY: is a","store of WEIGHTS","that acts as long","term memory"},
  {"WEIGHTS: numbers","that show a strength","of connection","between the NODES"},
  {"WEIGHTS: are learnt","and adjusted during","TRAINING and connect","NODES together"},
  {"NODES: are single","processing units","that mimic a real","biological neuron"}, 
  {"NODES: calculate","-inputs X WEIGHTS-","and outputs a signal","to other nodes"},
  {"NODES: fires an","output signal when","the sum of its input","exceeds a THRESHOLD"},
  {"THRESHOLD: a number","to compare against","to say what category","something is"},
  {"MAPPING: turns real","world data like","color and size into","INPUT SIGNALS"},
  {"MAPPING: turns the","OUTPUT SIGNALS","into real world data","like -box type-"},
  {"INPUT SIGNALS: are","numbers that feed","the inputs of a","NEURAL NETWORK"},
  {"OUTPUT SIGNALS: are","calculated by the","NEURAL NETWORK to","CLASSIFY input data"}, 
  {"LEARNING: getting","better at performing","a task by receiving","TRAINING"},
  {"LEARNING: knowledge","gained by seeing","many good and bad","examples of a thing"}, 
  {"LEARNING: NEURAL","NETWORKS learn by a","a method called BACK","PROPAGATION"}, 
  {"TRAINING: a process","of structured","LEARNING that uses","TRAINING EXAMPLES"},
  {"TRAINING EXAMPLES:","a pre-planned set of","good & bad examples","to teach with"}, 
  {"BACK PROPAGATION:","uses an ERROR SIGNAL","to adjust the WEIGHT","MEMORY contents"},
  {"BACK PROPAGATION: is","the type of LEARNING","used by a NEURAL","NETWORK"},
  {"ERROR SIGNAL: is the","diff. between OUTPUT","SIGNALS and the","TRAINING EXAMPLES"},
  {"ERROR SIGNAL: is a","way to tell a NEURAL","NETWORK if it is","correct or not"},
  {"<box name> uses a","PERCEPTRON to","CLASSIFY different","size and color boxes"},
  {"<box name> is a","simple example of","MACHINE LEARNING and","NEURAL NETWORKS"},
  {"MACHINE LEARNING: is","a subset of AI and","is a branch of study","in Computer Science"},  
  {"MACHINE LEARNING:","allows machines to","learn new tasks","without programming"},
  {"MACHINE LEARNING:","commonly uses NEURAL","NETWORKS as the main","form of intelligence"},
  {"AI: aims to make","computers as smart","or smarter than a","human!"},
  {"AI: uses MACHINE","LEARNING as a tool","to becoming more","intelligent"}, 
};

#define potMax 700
#define potMin 300

enum pongGameMode {
  pongPlay,
  pongShowScore,
  pongWait,
  pongExit
};

pongGameMode pongMode = pongWait;

int ballSpeed = 200;
int ballPosX;
int ballPosY;
int ballDirX;
int ballDirY;
int rightPaddlePos;
int leftPaddlePos;
boolean stickBallToPaddle;
int ballOnPaddle;
unsigned long lastBallUpdate;
boolean checkBall;

int pongLives;
unsigned long int pongScore;
unsigned long int pongScoreIncrement = 10;

String trainCorrect[5] = {
  "YES, I knew it was",
  "HA! as I thought, ",
  "OH YEAH - that's",
  "Thankyou. I know its",
  "BOOM. I was right!" 
};

String trainWrong[5] = {
  "NO! I was sure its",
  "OH, I thought it was",
  "HUH I was wrong with",
  "I'm shocked its not",
  "Now I see its not" 
};

void setup() {

  Serial.begin(115200);
  
  pinMode(potLeft, INPUT);
  pinMode(potCenter, INPUT);
  pinMode(potRight, INPUT);
  pinMode(greenButton, INPUT);
  pinMode(blueButton, INPUT);
  pinMode(redButton, INPUT);
  pinMode(joyButton, INPUT);
  pinMode(joyH, INPUT);
  pinMode(joyV, INPUT);
  pinMode(ampEnable, OUTPUT);

  // setup SD-card
  if (!SD.begin(sdCs)) {
    sdInit = false;
  } else {
    sdInit = true;
  }
    
  /* turn on the timer clock in the power management controller */
  pmc_set_writeprotect(false);
  pmc_enable_periph_clk(ID_TC4);
  /* we want wavesel 01 with RC */
  TC_Configure(/* clock */TC1,/* channel */1, TC_CMR_WAVE | TC_CMR_WAVSEL_UP_RC | TC_CMR_TCCLKS_TIMER_CLOCK2);
  TC_SetRC(TC1, 1, 238); // sets <> 44.1 Khz interrupt rate
  TC_Start(TC1, 1); 
  // enable timer interrupts on the timer 
  TC1->TC_CHANNEL[1].TC_IER=TC_IER_CPCS;
  TC1->TC_CHANNEL[1].TC_IDR=~TC_IER_CPCS; 
  /* Enable the interrupt in the nested vector interrupt controller */
  /* TC4_IRQn where 4 is the timer number * timer channels (3) + the channel number (=(1*3)+1) for timer1 channel1 */
  NVIC_EnableIRQ(TC4_IRQn);
  // make the look up tables
  createNoteTable(SAMPLE_RATE);
  makeSineWave();
  //set up the dac
  analogWriteResolution(12);
  analogWrite(DAC1, 0);

  lcd.begin(20, 4); // initialize the lcd
  lcd.setBacklight(1); 
  lcd.createChar(0, lock);

  matrix.begin();
  matrix.setTextWrap(false);
  matrix.setBrightness(60);
  matrix.clear();
  matrix.clear();
  leds.begin();  
  leds.setBrightness(60);
  leds.clear();
  leds.show();

  // JF-
  // uncomment lines below if you need to delete a corrupt log file
  //if (SD.exists("settings.txt"))
  //  SD.remove("settings.txt");

  loadedSettings = readSettings();

  seedRNG();

  lastBallUpdate = millis();
  checkBall = false;
  readPaddlePosition();
  stickBallToPaddle = true;
  ballOnPaddle = random(2);

  
  leds.clear();
  leds.show();

  Serial.print(boxName);
  Serial.print(" flush serial port....");
  Serial.flush();

}

void enterName() {

  // 16 chars
  // show cursor
  // A-Z
  // joy stick v - move cursor
  // joy buuton to select - any othe button to finish
  // middle dial to select guild

  matrix.clear();
  matrix.show();
  
  lcd.clear();
  lcd.setCursor(0,0);
  lcd.print("Use the dials to ");
  lcd.setCursor(0,1);
  lcd.print("name your brain and");
  lcd.setCursor(0,2);
  lcd.print("select your guild.");

  while(!buttonPressed(any)) {
  }

  lcd.clear();

  int cPos = 0;
  int cSel = 0;
  int cPosLast = -1;
  int cSelLast = -1;
  int pv = 0;

  String newName = padString(12,boxName);
  
  boolean nameSet = false;
  while(!nameSet) {

    pv = analogRead(potRight);
    pv = floor(pv/342);  

    // need to copy this if modded to welcome/set-name
    for (int x=0; x<8; x++) {
      for (int y=0; y<8; y++) {
        uint32_t c = matrix.Color(0,0,0);
        switch (guildGlyphs[pv][x][y]) {
          case 'R':
            c = gRed;
            break;
          case 'O':
            c = gOrange;
            break;
          case 'Y':
            c = gYellow;
            break;
          case 'B':
            c = gBlue;
            break;
          case 'T':
            c = gTeal;
            break;
          case 'G':
            c = gGreen;
            break;
          case 'P':
            c = gPurple;
            break;
          case 'W':
            c = gWhite;
            break;
          case ' ':
            c =gBlack;
            break;
          default:
            c = gBlack;
            break;  
        }
        matrix.drawPixel(y,x,c);
      }
    }

    cPos = analogRead(potLeft);
    cPos = floor(cPos/86);

    cSel = analogRead(potCenter);
    cSel = floor(cSel/39);

    if (cSel != cSelLast) {
      char c = cSel + 65;
      if (cSel == 26)
        c = ' ';
      newName.setCharAt(cPos,c);
      lcd.setCursor(4,0);
      lcd.print(newName);
    }

    
    if (cPos != cPosLast) {
      lcd.setCursor(0,1);
      lcd.print(padString(20," "));
      lcd.setCursor(cPos+4,1);
      lcd.print("^");
    }
    
    //lcd.noCursor();
    lcd.setCursor(0,3);
    lcd.print(padString(20,guildNames[pv]));

    matrix.show();

    cSelLast = cSel;
    cPosLast = cPos;

    if (buttonPressed(any))
      nameSet = true;

  }

  boxName = newName; 
  boxName.trim();
  boxGuild = guildNames[pv];
  logSettings();

  matrix.clear();
  matrix.show();
  
}

void factoryMode() { 

  unsigned long t = 0;//millis();
  union {byte b[4];float f;} temp;
  byte * b;

  logSettings();


  lcd.clear();
  lcd.setCursor(0,0);
  lcd.print("Uploading brain");
  lcd.setCursor(0,1);
  lcd.print("to the box factory.");

  int selected = 0;
  selected = pageUpdate();

  int i = 0;

  while(!buttonPressed(any) && (i<5)) {

    if (millis()-t > 5000) {
   
      Serial.print("SoS");
      Serial.flush();
      Serial.print(padString(20,boxName));
      Serial.flush();
      Serial.print(padString(20,boxGuild));

      int pv = 0;
      if (boxGuild == "PHOENIX")
        pv = 1;
      if (boxGuild == "LIGHTNING")
        pv = 2;
      for (int x=0; x<8; x++) {
        for (int y=0; y<8; y++) {
          Serial.print(guildGlyphs[pv][x][y]);         
        }
        Serial.flush();
      }
      
      for (int i=0; i<paramLen; i++) {
          temp.b[i%4] = params[i];
          if (i%4 == 3) {
            b = (byte *) &temp.f;
            Serial.write(b[0]);
            Serial.write(b[1]);
            Serial.write(b[2]);
            Serial.write(b[3]);   
            Serial.flush();   
          }        
      }



      i++;
      
      lcd.setCursor(0,3);
      lcd.print("sent:"+String(i));
      
      t = millis();
    
    }
    
  }

  mode = menu;
}


void loop() {

  switch (mode) {
    case welcome:
      modeDesc = "Welcome Mode";
      welcomeMode();
      break;
    case work:
      modeDesc = "Work Mode";
      workMode();
      break;
    case brainstate:
      modeDesc = "State Mode";
      stateOfMind();
      break;
    case menu:
      modeDesc = "Menu Mode";
      mainMenu();
      break;
    case train:
      modeDesc = "Train Mode";
      trainMode();
      break;
    case analyze:
      modeDesc = "Analyze Mode";
      analyzeMode();
      break;
    case factory:
      modeDesc = "Factory Mode";
      factoryMode();
      break;
    case settings:
      modeDesc = "Settings Mode";
      settingsMode();
      break;    
    case instructions:
      modeDesc = "Instr. Mode";
      instructionsMode();
      break;  
    case pong:
      modeDesc = "Pong Mode";
      pongLoop();
      break;  
    case synth:
      modeDesc = "Synth";
      synthMode();
      break;        
  }
   
}

// ***************************************************************************************
// stateOfMind
// ***************************************************************************************
void stateOfMind() {

  lcd.clear();
  pageClear();
  pageAddHeading("----BRAIN--STATE----");

  pageAddEntry(0, "I am " + boxName);
  pageAddEntry(1, boxGuild + " guild");
  pageAddEntry(2, "Training sets=" + String(trainCycles));
  pageAddEntry(3, "Accuracy=" + String(workModeAverage,2));
  pageAddEntry(4, "Best score=" + String(workModeBest));
  pageAddEntry(5, "Perceptron weight:");
  pageAddEntry(6, "--W0=" + String(nnWeights[0],13));  
  pageAddEntry(7, "--W1=" + String(nnWeights[1],13));
  pageAddEntry(8, "--W2=" + String(nnWeights[2],13)); 
  pageAddEntry(9, "Learning rate:");
  pageAddEntry(10, "--rate="+ String(nnLearningC,11));  
  pageAddEntry(11, "Box generators:");
  pageAddEntry(12, "--cMax=" + String(cMax));
  pageAddEntry(13, "--gbRadius=" + String(goodBoxRadius,4));

  int selected = 0;
  selected = pageUpdate();
  
  while(!buttonPressed(any)) {
    selected = pageUpdate();

    for (int x=0; x<8; x++) {
      for (int y=0; y<8; y++) {
        int xs = random(0, 100); //cmax*2 / 8 = 100
        int ys = random(0, 100);
        xs += (x-4)*100;
        ys += (y-4)*100;    
        float prediction = (nnFeedForward(xs, ys) + 1.0)/2.0;
        prediction *= 127.0;
        uint16_t RGB = matrix.Color(prediction, 128-prediction, 0);
        matrix.drawPixel(x, 7-y, RGB);   
      }      
    }

 
     matrix.show();
    
  }


  /*
  // map the learning state to a color: 50 or lower bad = red .... 100 good = green
  float col = ((workModeAverage-50)/50)*128;
  if (col < 0)
    col = 0;  
  setNEOScrollMessage("SHOWING MY STATE OF MIND  ",matrix.Color(128-col, col, 0));

  int selected = 0;
  while(!buttonPressed(any)) {
    updateNEOScrollMessage();
    selected = pageUpdate();
  }*/

  mode = menu;
  
}


void analyzeMode() {

  lcd.clear();
  matrix.clear();
  matrix.show();


  
  pageClear();
  pageAddHeading("----ANALYZE-MODE----");
  pageAddEntry(0,"Use the dials to");
  pageAddEntry(1,"make colored");
  pageAddEntry(2,"boxes .....");
//need nore
    
  int selected = 0;
  selected = pageUpdate();
    
  while(!buttonPressed(any)) {
    //updateNEOScrollMessage();
    selected = pageUpdate();
  }
 
  lcd.clear();
  
  float xs = 0;
  float ys = 0;

  // make a box until any button is pushed
  while(!buttonPressed(blue)) {
    xs = (((float)analogRead(potLeft) / 511.5)-1.0)*cMax;
    if (xs < -374) xs = -374;
    ys = (((float)analogRead(potRight) / 511.5)-1.0)*cMax;
    int boxSize = ((xs+cMax)/(cMax*2))*31;
    int boxColor = (((ys+cMax)/(cMax*2))*127)+32;
    drawBox(boxSize, boxColor, true);
    // run the neural net forward
    int prediction = nnFeedForward(xs, ys); 
      // prediction 1 = large red    -1 = small green
    
    lcd.setCursor(0, 0);
    lcd.print(padString(20,"Use the dials to"));
    lcd.setCursor(0, 1);
    lcd.print(padString(20,"make boxes."));
    lcd.setCursor(0, 2);
    lcd.print(padString(20,"I think this is a "));
    lcd.setCursor(0, 3);
    if (prediction == 1) 
      lcd.print(padString(20,"large red box."));
    else  
      lcd.print(padString(20,"small green box."));
  }

  lcd.clear();
  matrix.clear();
  matrix.show();
  
  mode = menu;
  return;

}

void aboutMode() {

  lcd.clear();
  pageClear();
  pageAddHeading("--FRYDEN-LEARNING--");

  pageAddEntry(0, "Box factory ver:");
  pageAddEntry(1, verNum);
  pageAddEntry(2, "Please visit:");
  pageAddEntry(3, "fryden-learning.com");
  pageAddEntry(4, "Supported by:");
  pageAddEntry(5, "Arm Inc.");
  pageAddEntry(6, "Sunstone Circuits");

  while(!buttonPressed(any)) {
    int selected = pageUpdate();
  }

}

void instructionsMode() {
  lcd.clear();
  lcd.setCursor(0, 0);
  lcd.print("...in "+modeDesc);
  delay(2000);
  mode = menu;
}

void settingsMode() {

  boolean leave = false;

  while (!leave) {
  
    lcd.clear();
    menuClear();
    menuAddHeading("------SETTINGS------");
    for (int i=0; i<settingsMenuSize; i++) {
      menuAddEntry(i, settingsMenuEntries[i]);
    }
  
    int selected = 0;  
    while(!buttonPressed(any)) {
      selected = menuUpdate(false);
    }

    switch (selected) {
      case 0:
        setVolume();
        break;
      case 1:
        setDifficulty();
        break;
      case 2:
       enterName();
       break;
      case 3:
        resetBrain();
        break;
      case 4:
        aboutMode();
        break;
      case 5:
        leave = true;
        break;  
      default:
        return;
    }
       
  }

  mode = menu;

}

void resetBrain() {

  lcd.clear();
  lcd.setCursor(0, 0);
  lcd.print("Green = reset brain");
  lcd.setCursor(0, 1);
  lcd.print("Red = cancel");

  boolean answered = false;
  while(!answered) {
   
    if (buttonPressed(red)) {
      answered = true; 
      lcd.setCursor(0, 3);
      lcd.print("Brain not reset.");
    } else if (buttonPressed(green)) {
      answered = true;
        boxName = "BRAIN BOX"; 
        boxGuild = "NINJA";
        nnWeights[0] = -1.0;
        nnWeights[1] = 1.0;
        nnWeights[2] = 0.0;
        nnLearningC = learningRate;
        trainCycles = 0;
        totalTrainingExamples = 0;
        totalCorrect = 0;
        workModeAverage = 0;
        workModeBest = 0;
        goodBoxRadius = 1.1;
        lcd.setCursor(0, 3);
        lcd.print("Brain reset.");
    }      
      
  }
  removeSettingsLog();
  delay(1500);
  return;
}

void setVolume() {

  lcd.clear();
  lcd.setCursor(0,0);
  lcd.print("Use the middle dial");
  lcd.setCursor(0,1);
  lcd.print("to adjust the volume");
  
  unsigned long toneTimeStamp = millis();
  boolean hiLow = false;
  
  ampOn();
  while(!buttonPressed(any)) {

    int pv = potValue(center, 0.01);
    if (pv < 0)
      pv = 0;
    if (pv > 10)
      pv = 10;

    volume = pv;

    lcd.setCursor(0,3);
    lcd.print("Volume="+String(volume)+"    ");

    if (millis() - toneTimeStamp > 300) { 
        toneTimeStamp = millis();
        if (hiLow) 
          playTone(60,1000);
        else
          playTone(90,1000);
        hiLow = !hiLow;   
    } 
   
  }
  ampOff();

  return;
}

void removeSettingsLog() {
  if (sdInit) {
    if (SD.exists("settings.txt"))
      SD.remove("settings.txt");
  }
}

void logSettings() {
  if (sdInit) {

    if (SD.exists("settings.txt"))
      SD.remove("settings.txt");
   
    paramsToBytes();
  
    File settings = SD.open("settings.txt", FILE_WRITE);
  
    if (settings) {
      settings.print("~" + boxName + "~");
      settings.print("!" + boxGuild + "!");
      for (int i=0; i<paramLen; i++) {
        settings.write(params[i]);
      }
      settings.close();
    } else {
      
    }
  }
}

boolean readSettings() {

  boolean loaded = false;
  
  if (sdInit) {
    if (SD.exists("settings.txt")) {
      File settings = SD.open("settings.txt");
      char c;
      int tf=0;
      String n = "";
      while (tf<2) {
        c = (char) settings.read();
        if (c=='~')
          tf++;
        n += c;
      }

      tf = 0;
      String g = "";
      while (tf<2) {
        c = (char) settings.read();
        if (c=='!')
          tf++;
        g += c;
      }

      int i=0;
      byte b;
      while(settings.available() && i<paramLen) {
        b = (byte) settings.read();
        params[i++] = b;  
      }

      loaded = true;
      bytesToParams();  
      
      boxName = n.substring(1,n.length()-1);   
      boxGuild = g.substring(1,g.length()-1); 
    }
  }


  return loaded;
  
}

void paramsToBytes() {
    int i=0;
    byte * b;
    union {byte b[4];float f;} temp;

    temp.f = nnLearningC; params[i++] = temp.b[0]; params[i++] = temp.b[1]; params[i++] = temp.b[2]; params[i++] = temp.b[3];
    temp.f = goodBoxRadius; params[i++] = temp.b[0]; params[i++] = temp.b[1]; params[i++] = temp.b[2]; params[i++] = temp.b[3];
    temp.f = nnWeights[0]; params[i++] = temp.b[0]; params[i++] = temp.b[1]; params[i++] = temp.b[2]; params[i++] = temp.b[3];
    temp.f = nnWeights[1]; params[i++] = temp.b[0]; params[i++] = temp.b[1]; params[i++] = temp.b[2]; params[i++] = temp.b[3];
    temp.f = nnWeights[2]; params[i++] = temp.b[0]; params[i++] = temp.b[1]; params[i++] = temp.b[2]; params[i++] = temp.b[3];
    temp.f = workModeAverage; params[i++] = temp.b[0]; params[i++] = temp.b[1]; params[i++] = temp.b[2]; params[i++] = temp.b[3];
    b = (byte *) &trainCycles; params[i++] = b[0]; params[i++] = b[1]; params[i++] = b[2]; params[i++] = b[3];
    b = (byte *) &totalCorrect; params[i++] = b[0]; params[i++] = b[1]; params[i++] = b[2]; params[i++] = b[3];
    b = (byte *) &totalTrainingExamples; params[i++] = b[0]; params[i++] = b[1]; params[i++] = b[2]; params[i++] = b[3];
    b = (byte *) &workModeBest; params[i++] = b[0]; params[i++] = b[1]; params[i++] = b[2]; params[i++] = b[3];

}

void bytesToParams() {
  int i=0;
  union {byte b[4];float f;} temp;
  
  temp.b[0] = (params[i++]); temp.b[1] = (params[i++]); temp.b[2] = (params[i++]); temp.b[3] = (params[i++]);
  nnLearningC = temp.f;
  temp.b[0] = (params[i++]); temp.b[1] = (params[i++]); temp.b[2] = (params[i++]); temp.b[3] = (params[i++]);
  goodBoxRadius = temp.f;
  temp.b[0] = (params[i++]); temp.b[1] = (params[i++]); temp.b[2] = (params[i++]); temp.b[3] = (params[i++]);
  nnWeights[0] = temp.f;
  temp.b[0] = (params[i++]); temp.b[1] = (params[i++]); temp.b[2] = (params[i++]); temp.b[3] = (params[i++]);
  nnWeights[1] = temp.f;
  temp.b[0] = (params[i++]); temp.b[1] = (params[i++]); temp.b[2] = (params[i++]); temp.b[3] = (params[i++]);
  nnWeights[2] = temp.f;
  temp.b[0] = (params[i++]); temp.b[1] = (params[i++]); temp.b[2] = (params[i++]); temp.b[3] = (params[i++]);
  workModeAverage = temp.f;
  trainCycles = (int)(((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));
  totalCorrect = (int)(((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));
  totalTrainingExamples = (int)(((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));
  workModeBest = (int)(((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));
}

void playIntroTune() {
  /*
  ampOn();

  playDualTone(75, 105, 0.1, 100000);
  delay(1000);
  playDualTone(75, 105, 0.33, 100000);
  delay(1000);
  playDualTone(75, 105, 0.66, 100000);
  delay(1000);
  playDualTone(75, 105, 0.9, 100000);
  delay(1000);

  ampOff();
  */
}


// ***************************************************************************************
// welcomeMode
// ***************************************************************************************
void welcomeMode() {

  unsigned long welcomeTimeStamp;

  lcd.clear();

  if (!loadedSettings) {
    enterName();
  }

  if (!sdInit) { 
    lcd.setCursor(0, 0);
    lcd.print("SD Card init failed!");
    delay(5000);
  } 
  lcd.clear();

  lcd.setCursor(0, 0);
  lcd.print("Box Factory  " + verNum + "   ");
  lcd.setCursor(0, 2);
  lcd.print("I am " + boxName);
  lcd.setCursor(0, 3);
  lcd.print(boxGuild + " guild");
  

  welcomeTimeStamp = millis();

  int pv = 0;
  if (boxGuild == "PHOENIX")
    pv = 1;
  if (boxGuild == "LIGHTNING")
    pv = 2;

  playIntroTune();
  
  while ((millis() - welcomeTimeStamp < 15000) && (!buttonPressed(any))) {

    // need to copy this if modded to welcome/set-name
    for (int x=0; x<8; x++) {
      for (int y=0; y<8; y++) {
        uint32_t c = matrix.Color(0,0,0);
        switch (guildGlyphs[pv][x][y]) {
          case 'R':
            c = gRed;
            break;
          case 'O':
            c = gOrange;
            break;
          case 'Y':
            c = gYellow;
            break;
          case 'B':
            c = gBlue;
            break;
          case 'T':
            c = gTeal;
            break;
          case 'G':
            c = gGreen;
            break;
          case 'P':
            c = gPurple;
            break;
          case 'W':
            c = gWhite;
            break;
          case ' ':
            c = gBlack;
            break;
          default:
            c = gBlack;
            break;  
        }
        matrix.drawPixel(y,x,c);
      }
    }
    
    matrix.show();
  
  }
  matrix.fillScreen(0);
  mode = menu;
    
}



void mainMenu() {

  pongMode = pongWait;

  matrix.clear();
  matrix.show();
  leds.clear();
  leds.show();
  
  lcd.clear();
  menuClear();
  menuAddHeading("-----MAIN--MENU-----");
  for (int i=0; i<mainMenuSize; i++) {
    menuAddEntry(i, mainMenuEntries[i]);
  }

  int selected = 0;
  int lastSelected = 0;
  int sct = millis();

  boolean answered = false;
  
  while(!answered) {
    selected = menuUpdate(true);
    if (lastSelected != selected)
      sct = millis();
    lastSelected = selected;
    if (millis()-sct > screenSaverTime) {
      screenSaver();
      menuUpdateLCD = true;
      sct = millis();  
      matrix.clear();
      matrix.show();
      lcd.clear();
    }
    if (buttonPressed(any) && !menuLocks[selected])
      answered = true; 
  }
  
  switch (selected) {
  case 0:
    mode = brainstate;
    return;
    break;
  case 1:
    mode = train;
    return;
    break;
  case 2:
    mode = work;
    return;
    break;  
  case 3:
    mode = factory;
    return;
    break; 
  case 4:
    mode = analyze;
    return;
    break; 
  case 5:
    mode = pong;
    return;
    break; 
  case 6:
    mode = synth;
    return;
    break; 
  case 7:
    mode = instructions;
    return;
    break; 
  case 8:
    mode = settings;
    return;
    break; 
  default:
    mode = menu;
    return;
    break;
  }
  
}

void workMode() {

  int boxSize = 0;
  int boxColor = 0;
  unsigned long makeBoxTimeStamp;
  unsigned long analyzeBoxTimeStamp;
  unsigned long toneTimeStamp;
  int toneDuration;
  int score = 0;

  matrix.clear();
  matrix.show();
  lcd.clear();
  pageClear();
  pageAddHeading("-----WORK--MODE-----");
  pageAddEntry(0,"My brain will work");
  pageAddEntry(1,"on a small set of 10");
  pageAddEntry(2,"examples to see how");
  pageAddEntry(3,"how well I can tell");
  pageAddEntry(4,"the differenence");
  pageAddEntry(5,"between small green");
  pageAddEntry(6,"boxes and large red");
  pageAddEntry(7,"boxes.");

  int selected = 0;
  selected = pageUpdate();

  boolean answered = false;
  while(!answered) {
    selected = pageUpdate();
    if (buttonPressed(red)) {
      mode = menu;
      return; 
    }
    if (buttonPressed(blue) || buttonPressed(green) || buttonPressed(joy)) {
      answered = true;
    }
  }

  delay(500);

  for (int j=0; j<10; j++) {

    lcd.clear();
    lcd.setCursor(0, 0);
    lcd.print("Making a sample box");
    lcd.setCursor(0, 1);
    lcd.print("Please wait.....");

    makeBoxTimeStamp = millis();
    toneTimeStamp = 0;

    int boxType = 0;
    float xs = 0;
    float ys = 0;
    
    ampOn();
    while (millis() - makeBoxTimeStamp < makeBoxDuration) {

      matrix.clear();

      float x = millis() - makeBoxTimeStamp;
      x = ((x-makeBoxDurationBy2)/makeBoxDurationBy2)*6;
      float sCurve = 1/(1+exp(-x*1.3));
      int sparkleProb = (int) (sCurve * 300);
      sCurve = 1/(1+exp(-x*1.0));
      int boxRate = (int) (sCurve * 200) + 30;

      drawBox(boxSize, boxColor, false);
  
      for (int i=0; i<64; i++) {
        int x = random(8);
        int y = random(8);
        int b = random(128);
        uint16_t RGB = matrix.Color(0, 0, b);
        if (random(sparkleProb) == 0)
          matrix.drawPixel(x, y, RGB);
      }

      if (millis() - toneTimeStamp > toneDuration) { 
        toneTimeStamp = millis();
        toneDuration = boxRate;
        playTone(random(60)+60,toneDuration);
        toneDuration = toneDuration * 1.3;
        
        boolean valid = false;
      
        while(!valid) {
          xs = random(26, cMax*2);
          ys = random(0, cMax*2);    
          float vDist = sqrt((xs*xs) + (ys*ys));
          if (vDist < (goodBoxRadius*cMax)) {
            int bc = (int) random(2);
            if (bc==1) {
              xs -= cMax;
              ys -= cMax;
              boxType = -1;
            } else {
              float txs = xs;
              xs = (ys*-1)+cMax;
              ys = (txs*-1)+cMax;
              boxType = 1;
            }
            valid = true;  
          }
        }

        boxSize = ((xs+cMax)/(cMax*2))*31;
        boxColor = (((ys+cMax)/(cMax*2))*127)+32;
      } // if for making tone.
      
      matrix.show();
            
    } // while
    
    ampOff();
    clearTone();

    // eval box
    drawBox(boxSize, boxColor, true);

    lcd.clear();
    lcd.setCursor(0, 0);
    lcd.print("Analyizing box");
    lcd.setCursor(0, 1);
    lcd.print("Please wait...");

    int prediction = nnFeedForward(xs, ys);
    String predictionDesc = "SMALL GREEN box?";
    if (prediction == 1) 
      predictionDesc = "LARGE RED box?";

    delay(250);
    analyzeBoxTimeStamp = millis();
    toneTimeStamp = millis();
    toneDuration = 100;


    ampOn();
    while (millis() - analyzeBoxTimeStamp < analyzeBoxDuration) {
      if (millis() - toneTimeStamp > toneDuration * 1.3) { 
        toneTimeStamp = millis();
        playTone(random(40)+30, toneDuration);
      } // if for making tone.         
    } // while
    ampOff();
    clearTone();

    // run NN
    // show result     
    lcd.clear();
    lcd.setCursor(0, 0);
    lcd.print("I think this box is ");
    lcd.setCursor(0, 1);
    lcd.print("a ");
    lcd.setCursor(2, 1);
    lcd.print(predictionDesc);

    ampOn();
    if(prediction == boxType) {
      score++;
      lcd.setCursor(0, 3);
      lcd.print("------CORRECT-------");  
      playTone(68,250);
      delay(300);
      playTone(68,150);
      delay(180);
      playTone(80,1000);
      delay(1300);
    } else {
      lcd.setCursor(0, 3);
      lcd.print("--------WRONG-------");  
      playTone(62,300);
      delay(390);
      playTone(56,300);
      delay(390);
      playTone(45,1000);
      delay(1300);  
    }
    ampOff();
    clearTone();
    delay(1000);
   
  }

  // show result     
  lcd.clear();
  lcd.setCursor(0, 1);
  lcd.print(padString(20,"Work Results:"));   
  lcd.setCursor(0, 2);
  String finalScore = "";
  if (score < 10)
    finalScore += "0" + String(score);
  else
    finalScore += String(score);
  finalScore += "/10";
    
  lcd.print(padString(20,finalScore)); 
  delay(3000);

  totalCorrect += score;
  totalTrainingExamples += 10;

  workModeAverage = 100 * ((float)totalCorrect / (float)totalTrainingExamples);
  
  if (score > workModeBest)
    workModeBest = score;

  if (score == 10)
    menuLocks[pongLock] = 0; 

  ampOff();

  logSettings();
  
  mode = brainstate;

}

void trainMode() {

  lcd.clear();
  matrix.clear();
  matrix.show();
  pageClear();
  pageAddHeading("---TRAINING--MODE---");
  pageAddEntry(0,"Use the dials to");
  pageAddEntry(1,"make five colored");
  pageAddEntry(2,"boxes as examples to");
  pageAddEntry(3,"teach my brain.");
  pageAddEntry(4,"It will help me");
  pageAddEntry(5,"learn if you use");
  pageAddEntry(6,"well thought-out");
  pageAddEntry(7,"examples.  ");
  pageAddEntry(8,"I will guess what");
  pageAddEntry(9,"the boxes are, and ");
  pageAddEntry(10,"learn from my ");
  pageAddEntry(11,"mistakes.");

  int score = 0;
    
  int selected = 0;
  selected = pageUpdate();
  
  boolean answered = false;
  while(!answered) {
    selected = pageUpdate();
    if (buttonPressed(red)) {
      mode = menu;
      return; 
    }
    if (buttonPressed(blue) || buttonPressed(green) || buttonPressed(joy)) {
      answered = true;
    }
  }
 
  for (int i=0; i<5; i++) {

    lcd.clear();
    lcd.setCursor(0, 0);
    lcd.print(padString(20,"Use the dials to"));
    lcd.setCursor(0, 1);
    lcd.print(padString(20,"make a box.")); 
    lcd.setCursor(0, 3);
    lcd.print(padString(20,"Press blue when done."));     
    
    float xs = 0;
    float ys = 0;
    int error = 0;

    // make a box until the blue button is pushed
    while(!buttonPressed(blue)) {
      xs = (((float)analogRead(potLeft) / 511.5)-1.0)*cMax;
      if (xs < -374) xs = -374;
      ys = (((float)analogRead(potRight) / 511.5)-1.0)*cMax;
      int boxSize = ((xs+cMax)/(cMax*2))*31;
      int boxColor = (((ys+cMax)/(cMax*2))*127)+32;
      drawBox(boxSize, boxColor, true);
      int prediction = nnFeedForward(xs, ys); // run a sneaky prediction 
      // prediction 1 = large red    -1 = small green
      uint32_t ledRGB = leds.Color(8,0,0);
      if (prediction == -1)
        ledRGB = leds.Color(0,8,0); 
      leds.setPixelColor(0, ledRGB);
      leds.show();

    }

    // run the neural net forward
    int prediction = nnFeedForward(xs, ys); 
    // prediction 1 = large red    -1 = small green

    boolean answered = false;
    while(!answered) {
     
      lcd.setCursor(0, 0);
      lcd.print(padString(20,"What type of box")); 
      lcd.setCursor(0, 1);
      lcd.print(padString(20,"is this?")); 
      lcd.setCursor(0, 2);
      lcd.print(padString(20,"Green = small green."));  
      lcd.setCursor(0, 3);
      lcd.print(padString(20,"Red = large red.")); 

      if (buttonPressed(red)) {
        answered = true;
        error = 1 - prediction; 
      } else if (buttonPressed(green)) {
        answered = true;
        error = -1 - prediction;
      }      
        
    }

    lcd.clear();

    if (error != 0) {
  
      nnAdjustWeights(xs, ys, error);

      lcd.setCursor(0, 0);
      lcd.print(padString(20,trainWrong[random(5)]));
      lcd.setCursor(0, 1);
      if (prediction == -1)
        lcd.print(padString(20,"a small green box"));
      else
        lcd.print(padString(20,"a larger red box"));
      lcd.setCursor(0, 3);
      lcd.print(padString(20,"Applying learning..."));
       
      unsigned long learningTimeStamp = millis();
      unsigned long toneTimeStamp = millis();
      int toneDuration = 100;
      
      ampOn();
      while (millis() - learningTimeStamp < trainBoxDuration) {
        if (millis() - toneTimeStamp > toneDuration * 1.3) { 
          toneTimeStamp = millis();
          playTone(random(40)+30, toneDuration);
        } // if for making tone.         
      } // while
         
      ampOff();
      clearTone();
      score++;

    } else {
      
      lcd.clear();
      lcd.setCursor(0, 0);
      lcd.print(padString(20,trainCorrect[random(5)]));
      lcd.setCursor(0, 1);
      if (prediction == -1)
        lcd.print(padString(20,"a green box"));
      else
        lcd.print(padString(20,"a red box"));
      
      unsigned long learningTimeStamp = millis();      
      while (millis() - learningTimeStamp < trainBoxDuration) {
      } // while
    }
  }

  trainCycles++; 

  logSettings();
  
  mode = brainstate;

  if (score==5) 
    menuLocks[synthLock] = 0;
  
  return;
  
}

void drawBox(int boxSize, int boxColor, boolean drawImmediate){
  
  int boxSizeInt = boxSize >> 2;
  int boxSizeFrac = boxSize & 3;
  int r = boxColor;
  int g = 160-boxColor;
  int b = 0;
    
  //if (boxSize > 16) {
  //r = 96-colVal;
  //g = 0;
  //b = colVal;
  //}
  
  uint16_t boxColorRGB = matrix.Color(r, g, b);

  if (drawImmediate)
    matrix.clear();
  
  for (int x=0; x<boxSizeInt; x++) {
    for (int y=0; y<boxSizeInt; y++) {
      matrix.drawPixel(x, 7-y, boxColorRGB);
    }   
  }

  if (boxSizeFrac == 0) {
    r = 0;
    g = 0;
    b = 0;
  }
  if (boxSizeFrac == 1) {
    r = r >> 2;
    g = g >> 2;
    b = b >> 2;
  }
  if (boxSizeFrac == 2) {
    r = r >> 1;
    g = g >> 1;
    b = b >> 1;
  }
  if (boxSizeFrac == 3) {
    r = (r >> 1) + (r >> 2);
    g = (g >> 1) + (g >> 2);
    b = (b >> 1) + (b >> 2);
  }
  boxColorRGB = matrix.Color(r, g, b);
    
  for (int y=0; y<boxSizeInt; y++) {
    matrix.drawPixel(boxSizeInt, 7-y, boxColorRGB);
  }
  for (int x=0; x<=boxSizeInt; x++) {
    matrix.drawPixel(x, 7-boxSizeInt, boxColorRGB);
  }     

  if (drawImmediate)
    matrix.show();

}


// ***************************************************************************************
// potValue
// ***************************************************************************************
int potValue(potPostion whichPot, float scale) {
  int value = -1;
  switch (whichPot) {
    case left:   
      value = analogRead(potLeft);
      break;
    case right:
      value = analogRead(potRight); 
      break;
    case center:
      value = analogRead(potCenter);
      break;
  }
  value = (int) value * scale;
  return value;
}

// ***************************************************************************************
// buttonPressed
// ***************************************************************************************
boolean buttonPressed(buttonColor whichButton) {
  int joyPress = digitalRead(joyButton);
  int greenPress = digitalRead(greenButton);
  int bluePress = digitalRead(blueButton);
  int redPress = digitalRead(redButton);
  boolean pressed = false;

  switch (whichButton) {
    case blue:
      if (bluePress == 0)
        pressed = true;
      break;
    case red:
      if (redPress == 0)
        pressed = true;
      break;
    case joy:
      if (joyPress == 0)
        pressed = true;
      break;
    case green:
      if (greenPress == 0)
        pressed = true;
      break;
    case any:
      if ((bluePress == 0) || (redPress == 0) || (joyPress == 0) || (greenPress == 0))
        pressed = true;
      break;
    default:
      pressed = false;
  }

  unsigned long timeSince = millis() - lastPressed;
  if ((pressed) && (timeSince < buttonReleaseDelay)) {
    lastPressed = millis();
    pressed = false;
  } else if (pressed) {
    lastPressed = millis();
  }

  return pressed;
  
}

// ***************************************************************************************
// setNEOScrollMessage
// ***************************************************************************************
void setNEOScrollMessage(String message, uint16_t color) {
  scrollNEOMessage = message;
  scrollNEOMilliSec = millis();
  scrollNEOMessageReset = -6 * scrollNEOMessage.length();
  scrollNEOMessagePos = matrix.width();
  scrollNEOColor = color;
}

// ***************************************************************************************
// updateNEOScrollMessage
// ***************************************************************************************
void updateNEOScrollMessage(){
  if (millis()-scrollNEOMilliSec > NEOscrollSpeed) {
    matrix.fillScreen(0);
    matrix.setCursor(scrollNEOMessagePos, 0);
    matrix.setTextColor(scrollNEOColor);
    matrix.print(scrollNEOMessage);
    if(--scrollNEOMessagePos < scrollNEOMessageReset) {
      scrollNEOMessagePos = matrix.width();
    }
    matrix.show();
    scrollNEOMilliSec = millis();
  }
}

// ***************************************************************************************
// nnFeedForward
// ***************************************************************************************
int nnFeedForward(float xs, float ys) {
  float sum = 0;
  float inputs[3];
  inputs[0] = xs; 
  inputs[1] = ys;
  inputs[2] = 1.0;
  for (int i=0; i<3; i++) {
    sum += inputs[i] * nnWeights[i];  
  }
  return nnActivate(sum);
}

// ***************************************************************************************
// nnActivate
// ***************************************************************************************
int nnActivate(float sum){
  if (sum>0) return 1;
  else return -1;
}

// ***************************************************************************************
// nnAdjustWeights
// ***************************************************************************************
int nnAdjustWeights(float xs, float ys, int error) {
  float inputs[3];
  inputs[0] = xs; 
  inputs[1] = ys;
  inputs[2] = 1.0;
  for (int i=0; i<3; i++) {
    nnWeights[i] += nnLearningC * (float)error * inputs[i];         
  }
}

int menuUpdate(boolean showIcons) {

  if (millis() - lastRead > 150) {
    int v = analogRead(joyV);

    posLast = posNow;
    
    if (v > 800)
      posNow = down;
    else if (v < 300)
      posNow = up;
    else if (v < 600 && v > 400)
      posNow = middle;

    if (posNow==middle && posLast==down){
      entrySel++;
      menuUpdateLCD = true;
      if (entrySel == menuNumEntries)
        entrySel = menuNumEntries-1;
    }
    else if (posNow==middle && posLast==up) {
      entrySel--;
      menuUpdateLCD = true;   
      if (entrySel < 0)
        entrySel = 0;
    }    
    lastRead = millis();


  }

  if (menuUpdateLCD) { 
    
    lcd.setCursor(0, 0);
    lcd.print(menuHeading);
    
    lcd.setCursor(0, 2);
    lcd.print("->");
  
    
    
    lcd.setCursor(2, 2);
    if (menuLocks[entrySel] && showIcons) {
      lcd.write(byte(0));
      lcd.setCursor(3, 2);
    } 
    lcd.print(menuEntries[entrySel]);
  
    if (entrySel==0) {
      lcd.setCursor(0, 1);
      lcd.print(padString(20," "));  
    } else {
      lcd.setCursor(2, 1);
      if (menuLocks[entrySel-1] && showIcons) {
        lcd.write(byte(0));
        lcd.setCursor(3, 1);
      }
      lcd.print(menuEntries[entrySel-1]);
    } 
  
    if (entrySel==menuNumEntries-1) {
      lcd.setCursor(0, 3);
      lcd.print(padString(20," "));  
    } else {
      lcd.setCursor(2, 3);
      if (menuLocks[entrySel+1] && showIcons) {
        lcd.write(byte(0));
        lcd.setCursor(3, 3);
      }
      lcd.print(menuEntries[entrySel+1]);
    }
    menuUpdateLCD = false;
  }

  return entrySel;
  
}

void menuAddHeading(String h) {
  menuHeading = padString(20,h);
}

void menuAddEntry(int i, String e) {
  if (i<16) {
    menuEntries[i] = padString(17,e); 
    menuNumEntries++;
    menuUpdateLCD = true;
  }

}

void menuClear() {
  menuNumEntries = 0;
  entrySel=0;
  menuHeading = " ";
  for (int i=0; i<16; i++) {
    menuEntries[i] = " ";    
  }
  menuUpdateLCD = true;
}

int pageUpdate() {

  if (millis() - lastRead > 150) {
    int v = analogRead(joyV);

    posLast = posNow;
    
    if (v > 900)
      posNow = down;
    else if (v < 200)
      posNow = up;
    else if (v < 576 && v > 448)
      posNow = middle;

    if (posNow==middle && posLast==down){
      entrySel++;
      pageUpdateLCD = true;
      if (entrySel == pageNumEntries)
        entrySel = pageNumEntries-1;
    }
    else if (posNow==middle && posLast==up) {
      entrySel--;
      pageUpdateLCD = true;   
      if (entrySel < 0)
        entrySel = 0;
    }    
    lastRead = millis();

  }

  if (pageUpdateLCD) { 
    
    lcd.setCursor(0, 0);
    lcd.print(pageHeading);
  
    lcd.setCursor(0, 2);
    lcd.print(pageEntries[entrySel]);
  
    if (entrySel==0) {
      lcd.setCursor(0, 1);
      lcd.print(padString(20," "));  
    } else {
      lcd.setCursor(0, 1);
      lcd.print(pageEntries[entrySel-1]);
    }
  
    if (entrySel==pageNumEntries-1) {
      lcd.setCursor(0, 3);
      lcd.print(padString(20," "));  
    } else {
      lcd.setCursor(0, 3);
      lcd.print(pageEntries[entrySel+1]);
    }
    pageUpdateLCD = false;
  }

  return entrySel;
  
}


void pageAddHeading(String h) {
  pageHeading = padString(20,h);
}

void pageAddEntry(int i, String e) {
  if (i<32) {
    pageEntries[i] = padString(20,e); 
    pageNumEntries++;
    pageUpdateLCD = true;
  }
}

void pageClear() {
  pageNumEntries = 0;
  entrySel=0;
  pageHeading = " ";
  for (int i=0; i<32; i++) {
    pageEntries[i] = " ";    
  }
  pageUpdateLCD = true;
}


String padString(int len, String in) {
  String out;
  if (in.length() < len) { 
    out = in;
    int l = len-in.length();
    for (int i=0; i<l; i++) {
      out+=" ";  
    }
  } else {
    out = in.substring(0,len);
  }
  return out;  
}

void playTone(int midiA, int duration) {
  tTime = millis();
  tDuration = duration;
  toneActive = true;
  playDual = false;

  // 882 to go = 658-665 zero crossing

  //samples = ((duration * 1000) / 22.7);// - 882;
  //if (samples < 0) samples = 0; 
  //countSamples = 0;

  ulPhaseIncrementA = nMidiPhaseIncrement[midiA];
}

void playDualTone(int midiA, int midiB, float mix, int duration) {
  tTime = millis();
  tDuration = duration;
  toneActive = true;
  playDual = true;

  ulPhaseIncrementA = nMidiPhaseIncrement[midiA];
  ulPhaseIncrementB = nMidiPhaseIncrement[midiB];

  if (mix <0.0) mix = 0.0;
  if (mix >1.0) mix = 1.0;

  toneMixA = mix;
  toneMixB = 1-0 - mix;
  
}

void ampOn() {
  digitalWrite(ampEnable,HIGH); 
}

void ampOff() {
  digitalWrite(ampEnable,LOW); 
}

void clearTone() {
  toneActive = false;
}

void TC4_Handler()
{
  
  
  // We need to get the status to clear it and allow the interrupt to fire again
  TC_GetStatus(TC1, 1);
 
  if (toneActive) {

    ulPhaseAccumulatorA += ulPhaseIncrementA;   // 32 bit phase increment, see below
    ulPhaseAccumulatorB += ulPhaseIncrementB;  
  
    // if the phase accumulator over flows - we have been through one cycle at the current pitch,
    // now we need to reset the grains ready for our next cycle
    if(ulPhaseAccumulatorA > SAMPLES_PER_CYCLE_FIXEDPOINT)
      ulPhaseAccumulatorA -= SAMPLES_PER_CYCLE_FIXEDPOINT;
    if(ulPhaseAccumulatorB > SAMPLES_PER_CYCLE_FIXEDPOINT)
      ulPhaseAccumulatorB -= SAMPLES_PER_CYCLE_FIXEDPOINT;
      
    uint32_t phaseAdjustPAccA = (ulPhaseAccumulatorA>>20) % SINE_SAMPLES;
    uint32_t phaseAdjustPAccB = (ulPhaseAccumulatorB>>20) % SINE_SAMPLES;

    // get the current sample   
    uint32_t ulOutputA = sineWave[phaseAdjustPAccA];
    uint32_t ulOutputB = sineWave[phaseAdjustPAccB];

    uint32_t ulOutput;

    if (!playDual)
      ulOutput = ulOutputA;
    else {
      ulOutput = (uint32_t)(ulOutputA * toneMixA) + (uint32_t)(ulOutputB * toneMixB);
    }

    ulOutput = ulOutput * (volume/10.0);
   
    dacc_write_conversion_data(DACC_INTERFACE, ulOutput);

    if (millis() - tTime > tDuration) {
      toneActive = false; 
      //digitalWrite(41,LOW);
    }
  } 

}

void makeSineWave()
{
  for(int i = 0;i < SINE_SAMPLES; i++) {
    sineWave[i] = (uint16_t)  (((1+sin(((2.0*PI)/SINE_SAMPLES)*i))*4095.0)/2);
  }
}

// fill the note table with the phase increment values we require to generate the note
void createNoteTable(float fSampleRate)
{
  for(uint32_t unMidiNote = 0;unMidiNote < MIDI_NOTES;unMidiNote++) {
    // Correct calculation for frequency
    float fFrequency = ((pow(2.0,(unMidiNote-69.0)/12.0)) * 440.0);
    nMidiPhaseIncrement[unMidiNote] = fFrequency*TICKS_PER_CYCLE;
  }
}

void screenSaver() {

  lcd.clear();
  matrix.clear();
  matrix.show();

  for (int x=0; x<8; x++) {
    for (int y=0; y<8; y++) {
      tile[x][y] = 0;
    }
  }
  
  for (int i=0; i<32; i++) {
    boolean set = false;
    while (!set) {
      int x = random(8);
      int y = random(8);
      if (tile[x][y]==0) {
        tile[x][y] = random(128)+64;
        set = true;
      }
    }
  }

  int sts = 0;
  int tts = 0;
  int tWait = saverTextMinShow;

  while (!buttonPressed(any)) {

    if (millis()-tts > tWait) {
      int r = random(saverTextNum);
      for (int i=0; i<4; i++) {
        lcd.setCursor(0,i);

        String temp = saverText[r][i];
        if (temp.substring(0,10).equals("<box name>"))
          temp = boxName + temp.substring(10);
        
        lcd.print(padString(20,temp)); 
      } 
      tts = millis();
      tWait = random(10000)+saverTextMinShow;   
    }

    // every 100ms - shuffle the board
    if (millis()-sts > 100) {
      
      matrix.clear();
      for (int x=0; x<8; x++) {
        for (int y=0; y<8; y++) {
          if (tile[x][y]!=0) {
            matrix.drawPixel(x,y,matrix.Color(tile[x][y],0,192-tile[x][y]));
          }         
        }
      }
      matrix.show(); 

      for (int x=0; x<8; x++) {
        for (int y=0; y<8; y++) {
          moved[x][y] = false;
        }
      }

      for (int x=0; x<8; x++) {
        for (int y=0; y<8; y++) {
    
          if (tile[x][y]!=0 & !moved[x][y]) {
            boolean free = false;
            int tries = 0;
            while (!free) {
              int xd = random(3)-1;
              int yd = random(3)-1;
              
              if ((xd!=0 && yd==0) || (xd==0 && yd!=0) ){
              
                int xp = x+xd;
                if (xp<0) xp = 0;
                if (xp>7) xp = 7;
                int yp = y+yd;
                if (yp<0) yp = 0;
                if (yp>7) yp = 7;
                          
                if (tile[xp][yp]==0) {
                  tile[xp][yp] = tile[x][y];
                  tile[x][y] = 0;
                  moved[xp][yp] = true;
                  free = true;
                }
                
                tries ++;
                if (tries == 10) {
                  free = true;  
                }
              }
              
            }
          }
        }
      }
      
      sts = millis();
        
    }// evry 100ms shuffle the tiles
    
  }
  
}

void setDifficulty() {

  lcd.clear();
  pageClear();
  pageAddHeading("-----DIFFICULTY-----");

  pageAddEntry(0, "Use the dials to set");
  pageAddEntry(1, "the difficulty of");
  pageAddEntry(2, "Work Mode.");
  pageAddEntry(3, "This setting adjusts");
  pageAddEntry(4, "the radius of the");
  pageAddEntry(5, "area that the box");
  pageAddEntry(6, "factory will make");
  pageAddEntry(7, "boxes.");
  
  while(!buttonPressed(any)) {
    int selected = pageUpdate();
  }

  lcd.clear();
  
  while(!buttonPressed(any)) {

    int pv = potValue(center, 0.01);
    if (pv < 0)
      pv = 0;
    if (pv > 10)
      pv = 10;

    goodBoxRadius = 0.9 + ((float)(pv>>1) / 10.0);  

    String diffA = "MEDIUM: make some";
    String diffB = "medium or orange";
    String diffC = "boxes";
    if (goodBoxRadius <= 1.05) {
      diffA = "EASY: make less";
      diffB = "medium or orange";
      diffC = "boxes";
    }
    if (goodBoxRadius >= 1.25) {
      diffA = "HARD: make more";
      diffB = "medium or orange";
      diffC = "boxes";
    }
    lcd.setCursor(0,0);
    lcd.print(padString(20,diffA));
    lcd.setCursor(0,1);
    lcd.print(padString(20,diffB));
    lcd.setCursor(0,2);
    lcd.print(padString(20,diffC));
    lcd.setCursor(0,3);
    lcd.print("Box Radius = " + String(goodBoxRadius,1));

  }

  logSettings();
  return;
}



void synthMode() {
  lcd.clear();
  lcd.setCursor(0,0);
  lcd.print("----ZAPPA--SYNTH----");
  lcd.setCursor(0,1);
  lcd.print("Use the dials and   ");
  lcd.setCursor(0,2);
  lcd.print("joystick to make");
  lcd.setCursor(0,3);
  lcd.print("synthesizer sounds");
  
  int tempVolume = volume;

  ampOn();

  while(!buttonPressed(any)) {
   
    int pv = potValue(left, 0.01);
    if (pv < 0)
      pv = 0;
    if (pv > 10)
      pv = 10;
    volume = pv;

    int horiz = analogRead(joyH);
    int vert = analogRead(joyV);
    horiz = (horiz>>4)+40;
    vert = (vert>>4)+40;
    int mix = potValue(center, 1.0);
    float mixS = (float)mix/1023.0;
    playDualTone(horiz,vert,mixS,100000000);
    
  }
  
  mode = menu;
  volume = tempVolume;
  clearTone();
  ampOff();
}

// ***************************************************************************************
// seedRNG
// ***************************************************************************************
void seedRNG() {
  long b0_b9 = (long) analogRead(potLeft);
  long b10_b19 = (long) analogRead(potCenter);
  long b20_b29 = (long) analogRead(potRight);
  long seed = b0_b9 | (b10_b19 << 10) | (b20_b29 << 20);
  randomSeed(seed);
}

void pongLoop() {

  switch (pongMode) {
    case pongWait:
      pongWaitMode();
      break;
    case pongShowScore:
      pongShowScoreMode();
      break;
    case pongPlay:
      pongPlayMode();
      break;
    case pongExit:
      mode = menu;
      break;
  }
  
}

void pongPlayMode() {
 
  matrix.clear();
  
  readPaddlePosition();

  if (stickBallToPaddle) {

    if (ballOnPaddle == 0) {
      ballPosX = 1;
      ballPosY = leftPaddlePos;
    } else {
      ballPosX = 6;
      ballPosY = rightPaddlePos;
    }

    if (buttonPressed(any)) {
      stickBallToPaddle = false;
      if (ballOnPaddle == 0)
        ballDirX = 1;
      else
        ballDirX = -1;
      ballDirY = random(3) - 1; 
      lastBallUpdate = millis(); 
    }
          
  } else {
    if (millis() - lastBallUpdate > ballSpeed) {
      ballPosX += ballDirX;
      ballPosY += ballDirY;

      lastBallUpdate = millis();
      checkBall = true;

    }
  }
    
  matrix.drawPixel(ballPosX, ballPosY,matrix.Color(0,180,0));
  for (int i=-1; i<2; i++) {
    matrix.drawPixel(0,leftPaddlePos+i,matrix.Color(180,0,0));
    matrix.drawPixel(7,rightPaddlePos+i,matrix.Color(0,0,180));
  }
  matrix.show();

  if (checkBall) {

    // bounce off the left paddle
    if (ballPosX == 1) {
      if (ballPosY <= leftPaddlePos+1 && ballPosY >= leftPaddlePos-1) {
        ballDirX *= -1;
        ballDirY = ballPosY - leftPaddlePos;        
        if (random(5) == 0) 
          ballDirY = random(3) - 1;
        if (ballDirY == 1 && ballPosY == 7)
          ballDirY = -1;
        else if (ballDirY == -1 && ballPosY == 0)
          ballDirY = 1;
        
        checkBall = false; 
        pongScore += map(ballSpeed, 30, 200, 9877, 10);
        playTone(102,50);
        ballSpeed *= 0.98;
        if (ballSpeed < 30) {
          ballSpeed = 30;       
        }
        lcd.setCursor(0, 0);
        lcd.print(pongScore);
      }
    }
  
    // bounce off the right paddle
    if (ballPosX == 6) {
      if (ballPosY <= rightPaddlePos+1 && ballPosY >= rightPaddlePos-1) {
        ballDirX *= -1;
        ballDirY = ballPosY - rightPaddlePos;
        if (random(5) == 0) 
          ballDirY = random(3) - 1;
        if (ballDirY == 1 && ballPosY == 7)
          ballDirY = -1;
        else if (ballDirY == -1 && ballPosY == 0)
          ballDirY = 1;
        checkBall = false;
        playTone(107,50);
        pongScore += map(ballSpeed, 30, 200, 9877, 10);
        ballSpeed *= 0.98;
        if (ballSpeed < 30) {
          ballSpeed = 30;
        }
        lcd.setCursor(0, 0);
        lcd.print(pongScore);
      }
    }

  }

  if (checkBall) {
    // bounce off the top and bottom walls 
    if ((ballPosY == 7 || ballPosY == 0) && ballDirY != 0) {
      ballDirY *= -1;
      checkBall = false;
    }
  }

  if (checkBall) {
    // if the balls misses the paddle and has gone off the screen - reset the ball
    if (ballPosX < -1 || ballPosX > 8) {
      stickBallToPaddle = true;
      ballOnPaddle = random(2);
      checkBall = false;
      ballSpeed = 200;
      pongLives--;
      lcd.setCursor(0, 1);
      lcd.print("Lives left = "+String(pongLives-1));
      playTone(65,300);
      delay(390);
      playTone(59,300);
      delay(390);
      playTone(46,1000);
      delay(1300);  
      if (pongLives == 0) {
        pongMode = pongShowScore;
        ampOff();
      }     
    }
  }


}

void pongShowScoreMode() {

  matrix.clear();
  matrix.show();

  lcd.clear();
  lcd.setCursor(0, 0);
  lcd.print("---- GAME  OVER ----");
  lcd.setCursor(0, 1);
  lcd.print(pongScore);
  while (!buttonPressed(any)) {
  } 

  pongMode = pongWait;
  
}

void readPaddlePosition() {

  int leftPotVal = analogRead(potLeft);
  int rightPotVal = analogRead(potRight);
  if (leftPotVal > potMax)
    leftPotVal = potMax;
  if (leftPotVal < potMin)
    leftPotVal = potMin;     
  if (rightPotVal > potMax)
    rightPotVal = potMax;
  if (rightPotVal < potMin)
    rightPotVal = potMin;

  leftPaddlePos = ((leftPotVal-potMin)/44.4)-1;
  rightPaddlePos = 8-((rightPotVal-potMin)/44.4);

  if (stickBallToPaddle) {
    if (leftPaddlePos > 6)
      leftPaddlePos = 6;
    if (leftPaddlePos < 1)
      leftPaddlePos = 1;  
    if (rightPaddlePos > 6)
      rightPaddlePos = 6;
    if (rightPaddlePos < 1)
      rightPaddlePos = 1;  
  }
  
}

void pongWaitMode() {

  matrix.clear();
  matrix.show();

  lcd.clear();
  lcd.setCursor(0, 0);
  lcd.print("-------BOING!-------");
  lcd.setCursor(0, 2);
  lcd.print("Push green to play");
  lcd.setCursor(0, 3);
  lcd.print("Push red to exit");

  ampOff();

  boolean answered = false;
  while (!answered) {

    if (buttonPressed(green)) {
      
      ampOn();
      
      pongLives = 4;
      pongScore = 0;
      pongMode = pongPlay;
      lcd.clear();
      lcd.setCursor(0, 0);
      lcd.print(pongScore);
      lcd.setCursor(0, 1);
      lcd.print("Lives left = "+String(pongLives-1));
      playTone(69,250);
      delay(300);
      playTone(69,150);
      delay(180);
      playTone(80,1000);
      delay(1300);
      answered = true;      
    }
    if (buttonPressed(red)) {
      pongMode = pongExit;
      answered = true;
    }
    
  }
  
}

/* Midi, Freq, Phase Acc/Inc
0 8.18 171458
1 8.66 181654
2 9.18 192456
3 9.72 203900
4 10.30 216024
5 10.91 228870
6 11.56 242479
7 12.25 256898
8 12.98 272174
9 13.75 288358
10 14.57 305505
11 15.43 323671
12 16.35 342917
13 17.32 363308
14 18.35 384912
15 19.45 407800
16 20.60 432049
17 21.83 457740
18 23.12 484959
19 24.50 513796
20 25.96 544348
21 27.50 576716
22 29.14 611010
23 30.87 647342
24 32.70 685835
25 34.65 726617
26 36.71 769824
27 38.89 815600
28 41.20 864098
29 43.65 915480
30 46.25 969918
31 49.00 1027592
32 51.91 1088696
33 55.00 1153433
34 58.27 1222020
35 61.74 1294685
36 65.41 1371671
37 69.30 1453235
38 73.42 1539649
39 77.78 1631201
40 82.41 1728197
41 87.31 1830961
42 92.50 1939836
43 98.00 2055184
44 103.83 2177392
45 110.00 2306867
46 116.54 2444040
47 123.47 2589370
48 130.81 2743343
49 138.59 2906470
50 146.83 3079298
51 155.56 3262402
52 164.81 3456395
53 174.61 3661923
54 185.00 3879672
55 196.00 4110369
56 207.65 4354785
57 220.00 4613734
58 233.08 4888081
59 246.94 5178741
60 261.63 5486686
61 277.18 5812940
62 293.66 6158596
63 311.13 6524805
64 329.63 6912791
65 349.23 7323847
66 369.99 7759345
67 392.00 8220739
68 415.30 8709570
69 440.00 9227469
70 466.16 9776162
71 493.88 10357483
72 523.25 10973372
73 554.37 11625881
74 587.33 12317193
75 622.25 13049611
76 659.26 13825582
77 698.46 14647694
78 739.99 15518690
79 783.99 16441479
80 830.61 17419140
81 880.00 18454938
82 932.33 19552324
83 987.77 20714966
84 1046.50 21946744
85 1108.73 23251762
86 1174.66 24634386
87 1244.51 26099222
88 1318.51 27651164
89 1396.91 29295388
90 1479.98 31037380
91 1567.98 32882958
92 1661.22 34838280
93 1760.00 36909876
94 1864.66 39104648
95 1975.53 41429932
96 2093.00 43893488
97 2217.46 46503524
98 2349.32 49268772
99 2489.02 52198444
100 2637.02 55302328
101 2793.83 58590776
102 2959.96 62074760
103 3135.96 65765916
104 3322.44 69676560
105 3520.00 73819752
106 3729.31 78209296
107 3951.07 82859864
108 4186.01 87786976
109 4434.92 93007048
110 4698.64 98537544
111 4978.03 104396888
112 5274.04 110604656
113 5587.65 117181552
114 5919.91 124149520
115 6271.93 131531832
116 6644.88 139353120
117 7040.00 147639504
118 7458.62 156418592
119 7902.13 165719728
120 8372.02 175573952
121 8869.84 186014096
122 9397.27 197075088
123 9956.06 208793776
124 10548.08 221209312
125 11175.30 234363104
126 11839.82 248299040
127 12543.85 263063664
*/
Processing & java for the factory simulatorProcessing
This is the laptop based factory simulator.
// TODO ----  FOR THE FACTORY
// delete factories from the lists
// box factory onto the projector
// takes 2 uploads to get going
// double name from serial creates a factory !!! clear the read fryden state....
// bug when 1st uploading - a com port has crap left in it uncleaned
// add a delete key to remove a factory
// comment blocks

import processing.serial.*;
import java.nio.ByteBuffer;
import java.util.*;

int factorySel = 0;
int factorySpeed = 8;

PGraphics pg;
PImage machineImg;
PImage robotImg;
PImage greenTruckImg;
PImage redTruckImg;
PImage inspectorImg;
PImage panelImg;
PImage frydenImg;

float xmax =  400;
float ymax =  400;
float xmin = xmax * -1;
float ymin = ymax * -1;

int ms;

String[] boxNames = new String[5];

ArrayList<Serial> recievers = new ArrayList<Serial>();
HashMap<String,Integer> nameCnts = new HashMap<String,Integer>();
HashMap<String,dueParams> data = new HashMap<String,dueParams>();
HashMap<String,factory> factories = new HashMap<String,factory>();

color gRed = color(192,0,0);        // R 
color gOrange = color(180,60,7);    // O
color gYellow = color(150,150,0);    // Y
color gBlue = color(0,0,192);       // B
color gPurple = color(192,0,192);   // P
color gGreen = color(0,192,0);      // G
color gTeal = color(94,192,192);      // T
color gBlack = color(0,0,0);        // B
color gWhite = color(180,180,180);   // W


void setup() {
  size(3500,2000,P2D);
  
  pg = createGraphics(1100,500);
  
  String ports[] = Serial.list();
  println(ports);
  for (int i=0; i<ports.length; i++) {
    recievers.add(new Serial(this, ports[i], 115200));
  }  
  delay(2000);
  
  Iterator<Serial> itr = recievers.iterator();
  while (itr.hasNext()) { 
    Serial s = itr.next();  
    while (s.available() > 0) {
      char c = s.readChar();
      print(c);
      delay(1);
    }
    s.clear();
  }  
  println();
  println();
    
  machineImg = loadImage("machine.PNG");
  robotImg = loadImage("robot.PNG");
  greenTruckImg = loadImage("greentruck.PNG");
  redTruckImg = loadImage("redtruck.PNG");
  inspectorImg = loadImage("inspector.jpg");
  panelImg = loadImage("panel.jpg");
  frydenImg = loadImage("fryden.png");
  
  ms = 0;


}

void draw() {
  
  // check for upload conenctions
  if (millis() - ms > 3000) {
    readFrydens();
    ms = millis();
  }
  
  // run the factories 
  for (String k: factories.keySet()) {
    factory f = factories.get(k);
    dueParams dp = data.get(k);
    int nc = nameCnts.get(k);
    if (nc > f.lastUpload) {
      f.upLoaded = true;
      f.lastUpload = nc;
      
      float[] w = new float[3];
      w[0] = dp.nnWeights[0];
      w[1] = dp.nnWeights[1];
      w[2] = dp.nnWeights[2];
      f.ptron.setWeights(w);
      f.ptron.c = dp.nnLearningC;
      f.goodBoxRadius = dp.goodBoxRadius ;
      f.trainCycles = dp.trainCycles;
      f.totalTrainingExamples = dp.totalTrainingExamples;
      f.totalCorrect = dp.totalCorrect;
      f.workModeAverage = dp.workModeAverage;
      f.workModeBest = dp.workModeBest;
      
    }
    
    // check to see if a new upload has happened
    
    if (f.upLoaded) {
      f.run();
    }
  }
  
  background(0);

  
  if (factories.size() > 0) {
  
    // fetch the selected factory
    factory f = factories.get(boxNames[factorySel]);
  
    if (f.upLoaded) {
      renderFactory(f);
    } else {    
      textSize(150);
      text("brain not yet uploaded",1000,600);  
    }
  
    dueParams dp = data.get(boxNames[factorySel]);
    // glyph
    pushMatrix();        
      translate(50,200);
      String ss = "A ";      
      ss += dp.guildName; 
      ss += " factory";  
      fill(255);
      textSize(60);
      text(ss,0,0);
      for (int x=0; x<8; x++) {
        for (int y=0; y<8; y++) {
          color c = color(0,0,0);
          switch (dp.guildGlyph[x][y]) {
            case 'R':
              c = gRed;
              break;
            case 'O':
              c = gOrange;
              break;
            case 'Y':
              c = gYellow;
              break;
            case 'B':
              c = gBlue;
              break;
            case 'T':
              c = gTeal;
              break;
            case 'G':
              c = gGreen;
              break;
            case 'P':
              c = gPurple;
              break;
            case 'W':
              c = gWhite;
              break;
            case ' ':
              c =gBlack;
              break;
            default:
              c = gBlack;
              break;  
          }
          fill(c);
          stroke(0);
          strokeWeight(5);
          rect(80+x*40,70+y*40,40,40);
        }
      }
    popMatrix(); 
  
    image(frydenImg, 2800, 1200, 1276*0.5,1400*0.5);
  
  
    // score lines
    pushMatrix();
      
      translate(50,1000);
      
      stroke(128);
      strokeWeight(8);
      line(0,600,2500,600);
      textSize(35);
      fill(128);
      for (int xm=30; xm<=100; xm+=10) {
        float xmp = (xm-25) * (2500/75);
        text(xm+"%",xmp,560);
      }
            
      int yp =0;
      for (String k: factories.keySet()) {
        f = factories.get(k);               
        
        // plot min 25% to 100% - into 2500 pixels
        float score = f.score;
        if (score<25.0)
          score = 25;
        
        float xPos = (score-25) * (2500/75);
        
        pushMatrix();
        
          translate(0,600);
          fill(255);
          stroke(255);
          strokeWeight(4);
          ellipse(xPos,0,40,40);

          if (!f.name.equals(boxNames[factorySel])) {
            line(xPos,0,xPos,yp*70+60);
            textSize(50);
            fill(255);
            text(f.name,xPos+10,yp*70+60);
          }
        
          if (f.name.equals(boxNames[factorySel])) {
            fill(255,255,0);
            line(xPos,0,xPos,-500);
            textSize(100);
            text(f.name,xPos+10,-400);
            
            fill(255);
            String s = "Accuracy = " + String.format("%.0f", f.score) + "%";
            textSize(60);
            text(s,xPos+10,-320);
            s = "Score = " + f.correct + " out of " + f.inspected;
            text(s,xPos+10,-240);
            s = "Training exmaples = " + (f.trainCycles*5);
            text(s,xPos+10,-160);
            
          }
       
        popMatrix();
                
        yp++;
      }
      
    popMatrix();
    
  } else {
    textSize(150);
    text("no brains uploaded!",1000,600);  
  }

 
} 





void renderFactory(factory f) {
  
  pushMatrix();
  
    translate(2800,ymax+200);
   
    textSize(30);
    // draw the boundary / box gen circles  
    strokeWeight(2);
    stroke(0,128,0);
    noFill();
    ellipse(-400,400,f.goodBoxRadius*800,f.goodBoxRadius*800);
    stroke(128,0,0);
    noFill();
    ellipse(400,-400,f.goodBoxRadius*800,f.goodBoxRadius*800);
    
    // draw the graphs, and the descision lines
    strokeWeight(2);
    stroke(128);
    float x1 = xmin;
    float y1 = f(x1,-1,0);
    float x2 = xmax;
    float y2 = f(x2,-1,0);
    line(x1,-1*y1,x2,-1*y2);
  
  
    // Draw the line based on the current weights
    // Formula is weights[0]*x + weights[1]*y + weights[2] = 0
    stroke(200,200,0);
    strokeWeight(2);
    float[] weights = f.ptron.getWeights();
    x1 = xmin;
    y1 = (-weights[2] - weights[0]*x1)/weights[1];
    x2 = xmax;
    y2 = (-weights[2] - weights[0]*x2)/weights[1];
    // trap div 0 and >>> inf.
    if (y1 > 32000) y1 = 32000;
    if (y1 < -32000) y1 = -32000;
    if (y2 > 32000) y2 = 32000;
    if (y2 < -32000) y2 = -32000;
    line(x1,-1*y1,x2,-1*y2);
    
    //blank out some stuff
    fill(0);
    noStroke();
    rect(-1200,-400,800,800);
    rect(-1200,400,2800,1200);
    rect(-800,-800,2800,400);
    rect(400,-400,800,800);
  
   
    //draw and label the axis
    stroke(255);
    strokeWeight(8);
    line(-405,405,-405,-405);
    line(-405,405,405,405);
    fill(255);
    text("small",-405,439);
    text("large",350,439);
    text("green",-498,400);
    text("red",-470,-390);
    
    // plot the data
    noStroke();
    for (int i=0; i<f.boxes.size(); i++) {
      box b = f.boxes.get(i);  
      
      float c = ((b.col+1)/2)*255;
      
      fill(c,255-c,0);
      rect(b.xs, -1*b.ys, (b.size+1)*25,(b.size+1)*25);
      
      if (b.classified) {
        if (b.classification==1) {
          fill(255,0,0);
          text("large red",b.xs+(b.size+1)*13,(-1*b.ys));
          }
        else {
          fill(0,255,0);
          text("small green",b.xs+(b.size+1)*13,(-1*b.ys));
        }
      }
      else {
        fill(255);
        text("not tested",b.xs+(b.size+1)*13,(-1*b.ys));
      }
  
    }
  
  popMatrix();
  
  
  pg.beginDraw();

    pg.background(0);
    
    // draw the inpectors
    pg.image(inspectorImg, 800, 385-135*0.6, 90*0.6,135*0.6);
    pg.image(inspectorImg, 800, 385-150*0.8-135*0.6, 90*0.6,135*0.6);
    pg.noFill();
    pg.stroke(200);
    pg.strokeWeight(5);
    pg.rect(800, 385-135*0.6, 90*0.6,135*0.6);
    pg.rect(800, 385-150*0.8-135*0.6, 90*0.6,135*0.6);
  
    // draw the boxes
    pg.stroke(192);
    pg.strokeWeight(2);
    for (int i=0; i<f.boxes.size(); i++) {
      box b = f.boxes.get(i);
      float c = ((b.col+1)/2)*255;
      pg.fill(c,255-c,0);
      if (b.classification == 0 || b.classification == -1) 
        pg.rect(b.xPos, 400-15-((b.size+1)*25), (b.size+1)*25,(b.size+1)*25);
      else
        pg.rect(b.xPos, 400-15-((b.size+1)*25) - (150*.8), (b.size+1)*25,(b.size+1)*25);
   
      //text(b.xs,b.xPos-25, 500);
      //text(b.ys,b.xPos-25, 520);
    }
  
    //draw the front conveyor
    pg.fill(128);
    pg.stroke(200);
    pg.strokeWeight(5);
    pg.rect(0,400-15,505,15);
    
    //draw the back conveyors
    pg.fill(128);
    pg.stroke(200);
    pg.strokeWeight(5);
    pg.rect(505,400-15,365,15);
    pg.rect(505,400-150*0.8-15,365,15);
      
    //draw the machine
    pg.tint(255, 127);
    pg.image(machineImg, 0, 400-163, 250,163);
    pg.noFill();
    pg.stroke(200);
    pg.strokeWeight(15);
    pg.rect(0,400-163,250,163);
    pg.noTint();
    
    // draw the panel
    pg.image(panelImg, 482,0, 540*0.35,580*0.35);
    
    // draw the robot
    pg.tint(255, 200);
    pg.image(robotImg, 500, 400-200, 150,200);
    pg.noFill();
    pg.stroke(200);
    pg.strokeWeight(15);
    pg.rect(500, 400-200, 150,200);
    pg.noTint();
    
    // draw the green truck
    pg.image(greenTruckImg, 875, 425-150*0.8, 260*0.8,150*0.8);
    
    // draw the red truck
    pg.image(redTruckImg, 875, 425-300*0.8, 260*0.8,150*0.8);
  
    // draw the matrix
    int gridSize = 102;
    int xPos = 527;
    int yPos = 48;
    float ledSpace = (gridSize / 8);
    float ledSize = ((gridSize / 8) * 0.7);
    
    float bs = (f.currentBoxSize+1)*4;
    int bsInt = (int) floor(bs); 
    float rc = (f.currentBoxCol+1)/2;
    float gc = 1-((f.currentBoxCol+1)/2);
    float bc = 0;
    rc*=255;
    gc*=255;
    bc*=255;
    
    pg.noStroke();
    pg.fill(64);
    pg.rect(xPos,yPos+ledSize,gridSize,gridSize);
    
    for (int c=0; c<8; c++) {
      for (int r=0; r<8; r++) {
        pg.fill(0);
        pg.rect(c*ledSpace+xPos+ledSize/2,gridSize + yPos - (r*ledSpace) - ledSize/2,ledSize,ledSize);
      }
    } 
    
    for (int c=0; c<bsInt; c++) {
      for (int r=0; r<bsInt; r++) {
        pg.fill(color(rc,gc,bc));
        pg.rect(c*ledSpace+xPos+ledSize/2,gridSize + yPos - (r*ledSpace) - ledSize/2,ledSize,ledSize);
      }
    } 
    
    if (bsInt<8) {
      for (int r=0; r<bsInt+1; r++) {
        int c = bsInt;
        pg.fill(color(rc*(bs-bsInt),gc*(bs-bsInt),bc*(bs-bsInt)));
        pg.rect(c*ledSpace+xPos+ledSize/2,gridSize + yPos - (r*ledSpace) - ledSize/2,ledSize,ledSize);
      } 
     for (int c=0; c<bsInt+1; c++) {
        int r = bsInt;
        pg.fill(color(rc*(bs-bsInt),gc*(bs-bsInt),bc*(bs-bsInt)));
        pg.rect(c*ledSpace+xPos+ledSize/2,gridSize + yPos - (r*ledSpace) - ledSize/2,ledSize,ledSize);
      } 
    }
  
    // show the current guess on the LCD
    pg.fill(64,128,255);
    pg.textSize(15);
    String g = "";
    if (f.currentGuess == -1)
      g = "Small Green";
    else if (f.currentGuess == 1)
      g = "Large Red";
    pg.text(g,532,39);  
  

    
  pg.endDraw();
  
  image(pg, 50, 200,1100*1.9,500*1.9);
  
}


void keyPressed() {
  if (key == CODED) {
    if (keyCode == UP) {
      factorySel++;
      if (factorySel == factories.size())
        factorySel = 0;
    } else if (keyCode == DOWN) {
      factorySel--;
      if (factorySel < 0)
        factorySel = factories.size()-1;
    } else if (keyCode == LEFT) {
      factorySpeed--;
      if (factorySpeed < 1)
        factorySpeed = 1;
      setSpeeds();
    } else if (keyCode == RIGHT) {
      factorySpeed++;
      if (factorySpeed > 30)
        factorySpeed = 30;
      setSpeeds();
    }
  } else {
   
    if (key == 'r' || key == 'R') 
      resetScores();
        
  }
}

void setSpeeds() {
  for (String k: factories.keySet()) { 
    factory f = factories.get(k);
    f.setSpeed(factorySpeed);
  }
}

void resetScores() {
  for (String k: factories.keySet()) { 
    factory f = factories.get(k);
    f.resetScores();
  }
}


float f(float x,float m, float c) {
  return m*x+c;
}

// time out after s.avail - 1 seconds


void readFrydens() {
   
  Iterator<Serial> itr = recievers.iterator();
  while (itr.hasNext()) { 
    Serial s = itr.next(); 
    
    String state = "SoS";
    char inByteT0 = '-';
    char inByteT1 = '-';
    char inByteT2 = '-';
    boolean done = false;
    boolean complete = false;
    int expN = 20;
    int nCnt = 0;
    String name = "";
    int expG = 20;
    int gCnt = 0;
    String guild = "";
    
    boolean avail = false;
    int ms = millis();
    
    int glCnt = 0;
    int expGl = 64;
    char[][] glyphIn = new char[8][8];
    
    int plCnt = 0;
    int expPl = 40;
    byte[] paramsIn = new byte[40];
    
    
    while ((s.available() > 0) && !done) {
        
        if (!avail) {
          ms = millis();
          avail = true;
        }
        
        if (millis()-ms > 5000) {
          done = true;  
        }
        
        inByteT2 = inByteT1;
        inByteT1 = inByteT0;
        inByteT0 = s.readChar();  
          
        switch(state) {
          case "SoS": 
            if (inByteT0 == 'S' && inByteT1 == 'o' && inByteT2 == 'S') {
              state = "Name";
              nCnt = 0;
              name = "";
            }          
            break;
          case "Name":
            name += inByteT0;
            nCnt++;
            if (nCnt == expN) {
              state = "Guild";
              gCnt = 0;
              guild = "";
            }
            break;
          case "Guild":
            guild += inByteT0;
            gCnt++;
            if (gCnt == expG) {
              state = "Glyph";
              glCnt = 0;
            }
            break;         
          case "Glyph":
            int x = glCnt%8;
            int y = glCnt>>3;
            glyphIn[x][y] = inByteT0;
            glCnt++;
            if (glCnt == expGl) {
              state = "Params";   
              plCnt = 0;
            }
            break;
          case "Params":
            paramsIn[plCnt] = (byte) inByteT0;
            plCnt++;
            if (plCnt == expPl) {
              done = true;
              complete = true;
            }
            break;
          default:             
            break;
        } 
      delay(1); 
    }
    
    if (complete) {
      name = name.trim();
      guild = guild.trim();
      println("Processing");
      if (data.containsKey(name)) {
        nameCnts.put(name, nameCnts.get(name) + 1);
        dueParams dp = data.get(name);
        dp.bytesToParams(paramsIn);
        dp.boxName = name;
        dp.guildName = guild;
        dp.guildGlyph = glyphIn;
        data.put(name,dp);
      } else {
        // we have a new box so create the entries in the tables
        if (factories.size()<5) {
          factory f = new factory(name);
          factories.put(name,f);
          nameCnts.put(name,0);
          dueParams dp = new dueParams();
          dp.bytesToParams(paramsIn);
          dp.boxName = name;
          dp.guildName = guild;
          dp.guildGlyph = glyphIn;
          data.put(name,dp);
          boxNames[factories.size()-1] = name;
        } 
      }
    }
    
    if (avail) {
      println("***Serial Acticity***");
      println(s.port);
      println(complete);
      println(name);
    }
       
  }
  
}


/*
void readFrydens() {
  
  boolean readName = false;
  boolean readGuild = false;
  boolean readGlyph = false;
  boolean readFloats = false;
  //boolean inState = false;
  boolean skipByte;    
  //boolean haveWeights = false;
  String nameIn = "";
  String guildIn = "";
  int expChars = 64;
  char[][] glyphIn = new char[8][8];
  int expBytes = 40;
  byte[] bytesIn = new byte[40];
    
  char inByte = '0';
  char lastByte = '0';
  int numBytes = 0;
  int numChars = 0;
  
 
  Iterator<Serial> itr = recievers.iterator();
  while (itr.hasNext()) { 
    Serial s = itr.next(); 
    
    readName = false;
    readGuild = false;
    readGlyph = false;
    readFloats = false;
    inByte = '0';
    lastByte = '0';
    nameIn = "";
    guildIn = "";
    numChars = 0;
    numBytes = 0;
    
 
    println("**********************");
 
    while (s.available() > 0) {
      
      lastByte = inByte; 
      inByte = s.readChar();
      skipByte = false;    
      
      if (lastByte == '~' && inByte == '^') {
        readName = true;
        readGuild = false;
        readGlyph = false;
        readFloats = false;
        nameIn = "";
      }
      if (lastByte == '^' && inByte == '~') {
        readName = false;
        readGuild = false;
        readGlyph = false;
        readFloats = false;
      }
      if (lastByte == '#' && inByte == '!') {
        readName = false;
        readGuild = true;
        readGlyph = false;
        readFloats = false;
        guildIn = "";
      }
      if (lastByte == '!' && inByte == '#') { // end of guild name - beg. of glyph
        readName = false;
        readGuild = false;  
        readGlyph = true;
        readFloats = false;
        skipByte = true;
        numChars = 0;
      }
      if ((numChars == expChars) && (!readFloats)) { // end of glyph - beg. of bytes
        readName = false;
        readGuild = false;  
        readGlyph = false;
        readFloats = true;
        numBytes = 0;
        numChars = 0;
      }
     
      
      if (!skipByte) {
        
        if (readName) {
          nameIn += inByte;   
          println("fomr name");
          println(nameIn);
          println(guildIn);
        } else if (readGuild) {
          guildIn += inByte;
          println("formguild");
          println(nameIn);
          println(guildIn);
        } else if (readGlyph) {
          int x = numChars%8;
          int y = numChars>>3;
          glyphIn[x][y] = inByte;
          numChars++;
          println("assemble glyph");
          println(nameIn);
          println(guildIn);
        } else if (readFloats) {
          bytesIn[numBytes++] = (byte) inByte;
         // PROCESS ALL the BYTES we have collected
          if (numBytes == expBytes) {
            readFloats = false;
            readName = false;
            readGuild = true;
            readGlyph = false;
            readFloats = false;
            numChars = 0;
            numBytes = 0;
            println("conv bytes");
            println(nameIn);
            println(guildIn);
            nameIn = nameIn.substring(1,nameIn.length()-1);
            guildIn = guildIn.substring(1,guildIn.length()-1);
            if (data.containsKey(nameIn)) {
              nameCnts.put(nameIn, nameCnts.get(nameIn) + 1);
              dueParams dp = data.get(nameIn);
              dp.bytesToParams(bytesIn);
              dp.boxName = nameIn;
              dp.guildName = guildIn;
              dp.guildGlyph = glyphIn;
              data.put(nameIn,dp);
            } else {
              // we have a new box so create the entries in the tables
              if (factories.size()<5) {
                factory f = new factory(nameIn);
                factories.put(nameIn,f);
                nameCnts.put(nameIn,0);
                dueParams dp = new dueParams();
                dp.bytesToParams(bytesIn);
                dp.boxName = nameIn;
                dp.guildName = guildIn;
                dp.guildGlyph = glyphIn;
                data.put(nameIn,dp);
                boxNames[factories.size()-1] = nameIn;
              }
            }
          }// PROCESS BYTES      
        } // read floats
             
      }// skip byte
           
      
      delay(1); 
    }   
    


       
  }
  
}*/
more processing code...Processing
class box {
 
  int xPos;
  int yPos;
  float xs;
  float ys;
  float size;
  float col;
  float cMax;
  float goodBoxRadius;
  int classification;
  int answer;
  boolean kill;
  boolean classified;
  boolean inspected;
  
  box(int xPos, int yPos, float cMax, float goodBoxRadius) {
    this.xPos = xPos;
    this.yPos = yPos;
    this.cMax = cMax;
    this.goodBoxRadius = goodBoxRadius;
    setSize();
    classification = 0;
    kill = false;
    classified = false;

    
  }
  
  void incBoxPosX(int xInc) {
    xPos += xInc;  
  }
  
  void incBoxPosY(int yInc) {
    yPos += yInc;  
  }
  
  void killBox() {
    kill = true;  
  }
  
  void addClassifier(int classification) {
     this.classification = classification;
     classified = true;
  }
  
  void addInspection() {
     inspected = true;
  }
  
  void setSize() {
    //make the box - set the answer...
    
    boolean valid = false;
    while(!valid) {
      xs = random(26, cMax*2);
      ys = random(0, cMax*2);    
      float vDist = sqrt((xs*xs) + (ys*ys));
      if (vDist < (goodBoxRadius*cMax)) {
        int bc = (int) random(2);
        if (bc==1) {
          xs -= cMax;
          ys -= cMax;
          answer = -1;
        } else {
          float txs = xs;
          xs = (ys*-1)+cMax;
          ys = (txs*-1)+cMax;
          answer = 1;
        }
        valid = true;  
      }
    }
    
    size = xs / cMax;
    col = ys /cMax;
    
    
  }
  

  
}
more processing code...Processing
class dueParams {
  
String boxName;
String guildName;
float[] nnWeights = new float[3];
float nnLearningC;
int trainCycles;
int totalTrainingExamples;
int totalCorrect;
float workModeAverage;
int workModeBest;
float goodBoxRadius;
char[][] guildGlyph = new char[8][8];
  
  dueParams() {
    
  }
    
  void bytesToParams(byte[] params) {
  
    int i=0;  
    int temp;
    temp = (((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));    
    nnLearningC = Float.intBitsToFloat(temp);
    temp = (((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));    
    goodBoxRadius = Float.intBitsToFloat(temp);
    temp = (((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));       
    nnWeights[0] = Float.intBitsToFloat(temp);
    temp = (((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));       
    nnWeights[1] = Float.intBitsToFloat(temp);
    temp = (((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));      
    nnWeights[2] = Float.intBitsToFloat(temp);
    temp = (((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));      
    workModeAverage = Float.intBitsToFloat(temp);
    trainCycles = (int)(((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));
    totalCorrect = (int)(((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));
    totalTrainingExamples = (int)(((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));
    workModeBest = (int)(((params[i++] & 0xff) << 0) | ((params[i++] & 0xff) << 8) | ((params[i++] & 0xff) << 16) | ((params[i++] & 0xff) << 24));
  
  }
  
  
}
more processing code...Processing
import java.util.Iterator;

class factory {
  
  ArrayList<box> boxes = new ArrayList<box>(); 
  String name;
 
  int trainCycles;
  int totalTrainingExamples;
  int totalCorrect;
  float workModeAverage;
  int workModeBest;
 
  perceptron ptron;
  int correct = 0;
  int inspected = 0;
  float score = 0;
  
  int lastUpload = 0;
  
  float currentBoxSize = -1;
  float currentBoxCol = 0;
  int currentGuess = 0;
  
  // run time params
  int speed = 12;
  float cMax =  400;
  float goodBoxRadius = 1.1;
  
  boolean upLoaded;
  
  factory(String name) {
    
    this.name = name;
    //this.goodBoxRadius = goodBoxRadius;
    ///this.cMax = cMax;
       
    ptron = new perceptron(3, 0.0001);
    boolean upLoaded = false;

  }
  
  void run() {
  
    // make new boxes
    if (boxes.size() < 1)
      makeNewBox();
    else {
      box b = boxes.get(boxes.size()-1);
    if (b.xPos - b.size > 100)
      makeNewBox();
    }
    
    //update all boxes 
    for (int i=0; i<boxes.size(); i++) {
      box b = boxes.get(i);
      // move the box along
      b.incBoxPosX(speed);
      // is the box in the truck? if so then kill it
      if (b.xPos > 900) {
        b.killBox(); 
      }
      // is the box in the inspector
      if (b.xPos > 800 && !b.inspected) {
        inspected++;
        if (b.answer == b.classification)
          correct++;
        score = 100.0 * float(correct)/float(inspected);
        b.addInspection();
      }      
      // is the box in the robot? if so then classifiy it
      if (b.xPos > 550 && !b.classified) {
        float[] vecIn = {b.xs,b.ys,1.0};
        int c = ptron.feedforward(vecIn);
        b.addClassifier(c);
        currentBoxSize = b.size;
        currentBoxCol = b.col;
        currentGuess = c;
      }
    }
    
    // finally remove any killed boxes
    Iterator<box> boxesIterator = boxes.iterator();
    while (boxesIterator.hasNext()) {
      box b = boxesIterator.next();
      if (b.kill)
        boxesIterator.remove();
    }
    
  }
  
  void makeNewBox() {
    box b = new box(0, height/2, cMax, goodBoxRadius);
    boxes.add(b);  
  }
  
  void setSpeed(int speed) {
    this.speed = speed;  
  }
  
  void resetScores() {
   correct = 0;
   inspected = 0;
  }  
}
more processing code...Processing
// Daniel Shiffman
// The Nature of Code
// http://www.shiffman.net/teaching/nature
// Simple Perceptron Example
// See: http://en.wikipedia.org/wiki/Perceptron

// Perceptron Class

class perceptron {
  float[] weights;  // Array of weights for inputs
  float c;          // learning constant

  // Perceptron is created with n weights and learning constant
  perceptron(int n, float c_) {
    weights = new float[n];
    // Start with random weights
    //for (int i = 0; i < weights.length; i++) {
    //  weights[i] = random(-1,1); 
    //}
    // fix the wieghts to give the 50/50 case 
    weights[0] = -1;
    weights[1] = 1;
    weights[2] = 0;
    c = c_;
  }

  // Function to train the Perceptron
  // Weights are adjusted based on "desired" answer
  void train(float[] inputs, int desired) {
    // Guess the result
    int guess = feedforward(inputs);
    // Compute the factor for changing the weight based on the error
    // Error = desired output - guessed output
    // Note this can only be 0, -2, or 2
    // Multiply by learning constant
    float error = desired - guess;
    // Adjust weights based on weightChange * input
    for (int i = 0; i < weights.length; i++) {
      weights[i] += c * error * inputs[i];         
    }
  }

  // Guess -1 or 1 based on input values
  int feedforward(float[] inputs) {
    // Sum all values
    float sum = 0;
    for (int i = 0; i < weights.length; i++) {
      sum += inputs[i]*weights[i];
    }
    // Result is sign of the sum, -1 or 1
    return activate(sum);
  }
  
  int activate(float sum) {
    if (sum > 0) return 1;
    else return -1; 
  }
  
  // Return weights
  float[] getWeights() {
    return weights; 
  }
  
  void setWeights(float[] weights) {
    for (int i=0; i<weights.length; i++) {
      this.weights[i] = weights[i];
    }
  }
}

Schematics

Build instructions

Comments

Similar projects you might like

El Toro Grande: Self-Driving Car Using Machine Learning

Project showcase by dantuluri

  • 4,299 views
  • 0 comments
  • 22 respects

AI LCD friend!

Project tutorial by aip06

  • 3,474 views
  • 8 comments
  • 7 respects

Arduino Train for Kids 2

Project showcase by Steve Massikker

  • 10,754 views
  • 7 comments
  • 44 respects

A Microwave Interface for the IKEA Duktig Kids Kitchen

Project showcase by Myles Eftos

  • 4,091 views
  • 13 comments
  • 19 respects

CNC drawing machine with joysticks

Project showcase by TechnoFabrique

  • 2,145 views
  • 0 comments
  • 5 respects

Electronic Weighing Machine

Project tutorial by Boaz Lawnce

  • 10,841 views
  • 10 comments
  • 22 respects
Add projectSign up / Login