Project tutorial

Calculating Reading Time with TinyML and Arduino Nano 33 BLE © MIT

Do you want to know remaining reading time? This little TinyML Arduino device was trained to do exactly that.

  • 3,745 views
  • 2 comments
  • 23 respects

Components and supplies

Apps and online services

About this project


if (typeof(lightBoxImages) == 'undefined') { lightBoxImages = {}; } lightBoxImages['3ea9034188'] = [{ URL: 'https://hackster.imgix.net/uploads/attachments/1275121/pxl_20210317_143224710_LozB3EEyuL.jpg?auto=compress%2Cformat&w=1280&h=960&fit=max', caption: 'Remaining reading time calculation with TinyML and Arduino Nano BLE 33', type: 'image' }];

Being a writer born in the seventies, of course I do like paper books but after many years using also a Kindle I’ve got used to some features I don’t want to lose.

What features? remaining reading time for instance. Can such feature be implemented for paper books? How?

First thought (worst thought) was adding pressing a button with every page turns.

Will it work? Sure but it will be kind of annoying and not at all interesting from engineering point of view.

if (typeof(lightBoxImages) == 'undefined') { lightBoxImages = {}; } lightBoxImages['96eeb885e1'] = [{ URL: 'https://hackster.imgix.net/uploads/attachments/1275133/pxl_20210317_150336503_65ZbkDeaEz.jpg?auto=compress%2Cformat&w=1280&h=960&fit=max', caption: 'Testing the device', type: 'image' }];

What about using audio instead? Can you teach Arduino how a page turn it is supposed to sound like?

Sure. With a little Machine Learning help.

if (typeof(lightBoxImages) == 'undefined') { lightBoxImages = {}; } lightBoxImages['59090807b2'] = [{ URL: 'https://hackster.imgix.net/uploads/attachments/1275138/screenhojas_sS6xhAgfrt.PNG?auto=compress%2Cformat&w=1280&h=960&fit=max', caption: '', type: 'image' }];

What do you need for this project?
  • Arduino BLE 33 Sense (which has an on board microphone and a powerful Nina B3 chip)
  • Edge Impulse free account, which is a platform to simplify machine learning tasks.
  • One Adafruit Oled 128x32 screen (to display reading time)
  • One potentiometer (to setup book pages)

Circuits

Oled Screen VCC to Arduino 3.3v, GND to Arduino GND, SDA to A4, SCL to A5

Potentiometer to VIN, GND and A0

if (typeof(lightBoxImages) == 'undefined') { lightBoxImages = {}; } lightBoxImages['5fe1e1b97c'] = [{ URL: 'https://hackster.imgix.net/uploads/attachments/1275124/circuitnanoble33tinyml_dM93DffomR.jpg?auto=compress%2Cformat&w=1280&h=960&fit=max', caption: 'Circtuis', type: 'image' }];

TinyML with Edge Impulse

After creating your free Edge Impulse account, go to Data Acquisition, Collect Data, Use your computer. Allow browser permissions and Save at least 5 minutes with 2 seconds between every page turn. When you finish, the audio will be uploaded and a temporary popup will let you know when it is ready. Then you will be able to see the data in Training Data. Do the same but now recording room noise.

if (typeof(lightBoxImages) == 'undefined') { lightBoxImages = {}; } lightBoxImages['b398fa9c04'] = [{ URL: 'https://hackster.imgix.net/uploads/attachments/1275125/mldataacquisition_sQfvi3bqP5.jpg?auto=compress%2Cformat&w=1280&h=960&fit=max', caption: 'Data acquisition', type: 'image' }];

Now, make a click in the 3 dots, at the right of the first of the 1 min collected data audios and Split the sample. Do the same for all the samples.

Rebalance the data set from Dashboard so you can have 80% in training and 20% in testing.

Now design your Impulse with MFCC, 2000ms windows size, 1000ms window increase. Save. Go to MFCC Generate Features and confirm that Window size is 101. Then go to NN Classifier, 300 training cycles, 0.05 learning rate, 0.80 confidence rating, Noise high, Mask time bands low and mask freq none. Finally, Start Training.

if (typeof(lightBoxImages) == 'undefined') { lightBoxImages = {}; } lightBoxImages['28d2fb3b9e'] = [{ URL: 'https://hackster.imgix.net/uploads/attachments/1275126/modeltesting_srkeC9Kkl6.jpg?auto=compress%2Cformat&w=1280&h=960&fit=max', caption: '', type: 'image' }];

Now you can go to Model Testing and check the global accuracy. If you get a good percentage you are ready to deploy the model to the Arduino. Go to Deployment, Arduino Library. Save the file and add the zip with Arduino IDE, Include Library, Add Zip library.

Then you can load the example nanoble33 microphone and test the model.

You can then download my complete code for this project at Github Reading Time You should change the include line #include <bandini-project-1_inference.h> to use the same.h indicated in the nanoble33microphone example.

if (typeof(lightBoxImages) == 'undefined') { lightBoxImages = {}; } lightBoxImages['2197d0df83'] = [{ URL: 'https://hackster.imgix.net/uploads/attachments/1275134/readingtime_9fw9fE54HF.jpg?auto=compress%2Cformat&w=1280&h=960&fit=max', caption: 'That is... 119 hours, ok then', type: 'image' }];

Reading time calculation

Let’s say that the book has 100 pages. If I’ve read 2 pages per minute and I’ve still got 50 pages to go, I will end the book in 50/2 x 1 minute. That is 25 minutes.

This is the code to do that calculation

int remainingMinutes=(elapsedTime/pagesCounter)*(pages-pagesCounter)/1000/60;

Note: 1000 to convert from milliseconds to seconds, then 60 for minutes conversion.

What if we avoid all that Machine Learning trouble using a loud sound detection?

There are 2 problems with this approach. The first one is that any loud sound like a door slam or the phone will be counted and the second problem is that doing that is no fun whatsoever. In the video example below, you can verify that only page turn sounds are detected.

3d Printed Case

With Fusion 360 I've made a minimalistic case holder.

You can find the file at Thingiverse. It is just one piece and it can be printed without support.

if (typeof(lightBoxImages) == 'undefined') { lightBoxImages = {}; } lightBoxImages['56bb4ea07c'] = [{ URL: 'https://hackster.imgix.net/uploads/attachments/1275141/fusion360_v7zytUeGfi.PNG?auto=compress%2Cformat&w=1280&h=960&fit=max', caption: 'Fusion 360 design', type: 'image' }];

Demo video
Hunter S Thompson book is completely optional :)

Some troubleshooting

Serial Monitor problems: you may have to switch between port 5 and port 6 for uploading and serial monitor.

Windows 10 compilation problems: Due to file names too long issue, I did create a temp folder at root and added build.path inside preference file Users/Your User/ AppData/Local/Arduino15/Preferences

Edge Impulse versions: if you are exporting new versions, you may have to delete the old one from Documents/Arduino/Libraries or a previous.h file could be used.

Interested in other projects related to literature and electronic arts?

/// Follow me on Instagram

Code

https://github.com/ronibandini/ReadingTimeArduino
No preview (download only).

Custom parts and enclosures

Schematics

Circuit
Circuitnanoble33tinyml zry1vswdor

Comments

Similar projects you might like

Sound Spectrum Visualizer with Arduino Nano 33 BLE

Project tutorial by Enrique Albertos

  • 10,158 views
  • 12 comments
  • 22 respects

TinyML Water Sensor - Based on Edge Impulse & Arduino Sense

Project tutorial by enzo

  • 2,319 views
  • 0 comments
  • 13 respects

Spectrino: TinyML Arduino & IoT Based Touch-Free Solutions

Project tutorial by Dhruv Sheth

  • 13,688 views
  • 12 comments
  • 45 respects

Cough Detection with TinyML on Arduino

Project tutorial by Team Edge Impulse

  • 12,692 views
  • 8 comments
  • 48 respects

ECG Analyzer Powered by Edge Impulse

Project tutorial by Manivannan

  • 11,784 views
  • 7 comments
  • 64 respects
Add projectSign up / Login