TinyMLjs

Making TinyML truly client-side. Giving Makers full control of the process, user friendly, private and protected

Version 0.56.3-215 Note: WebSerial microcontroller connection works on Chrome or Edge for Mac, Android (Pixel Phones early verson) or Windows, only works on Edge for Linux

This website makes machine learning models from WebSerial connected micro-controller sensors using TensorflowJS. That model can be saved and converted to a tFlite model, then to a C-header model.h file using Tensorflow command line or Python converters. I simplify the conversion using a Gitpod. Finally the model.h file is combined with C/C++ code using an Arduino IDE ready library called RocksettaTinyML and compiled to the device for testing.

Presently, for complex vision or sound data it is easier to use EdgeImpulse.com as it will achieve the needed model compression that we have not yet achieved.

Label:
CSV FileName:

...

Machine Learning models often need very specific data.
Count CSV: Count Senses: Count Total:
Number of Samples/count: Number of Senses/sample:

Enter number of epochs: , Learning rate:

...

Convert the exported model to Arduino ready c-header model.h file as well as a model.tflite file using one of the following methods:

1:	tensorflowjs-to-arduino-for-tinymljs	Best to install the modules needed to client-side do the conversions yourself. This github shows what you need to do. The installation might be different on your computer. I always find this hard to do at school.
2:	iPython Notebook TFJS to TFlite	This web based Google Colab iPython notebook (Needs a google login) loads the necessary code then you click on an upload button to load your tensorflowjs exported "model.json" and "model.weights.bin" files and it zips and downloads the tflite and c-header files.
3:	Use a Gitpod: https://github.com/hpssjellis/Gitpod-auto-tensorflowJS-to-arduino or direct load the gitpod: Gitpod Here	A Gitpod browser docker like program that auto loads the necessaray python files and then runs a bash program to do the command line conversions. All code is easy to view, if you have a github account it loads fast.

Use https://netron.app/ Use the online netron.app to check and visualize your downloaded model.tflite
Once you have made a model.h file then install this Arduino Library RocksettaTinyML based on EloquentArduino to load the code onto your Arduino IDE.

...

Select model file (.json):
Select weights file (.bin):
Select labels file (.txt):
Enter Labels (comma seperated):

...

https://github.com/hpssjellis/tinyMLjs/blob/main/public/acceleration/a00-accell-nano33-fancy.txt

here

/*
 * webSerial for testing javascript connection with an arduino
 * 
 * Latest work at   https://github.com/hpssjellis/webMLserial
 * 
 * Note: On the Arduino Serial monitor make sure no line-ending or if statements will not work
 * 
 * Android https://hpssjellis.github.io/web-serial-polyfill/index.html
 * Laptops Desktops  https://hpssjellis.github.io/my-examples-of-arduino-webUSB-webSerial/public/index.html
 * IOS     not really sure
 * 
 */

#include <Arduino.h> // Only needed for https://platformio.org/
#include <Arduino_LSM9DS1.h>

#define FREQUENCY_HZ        36     // how many samples per second 
#define COLLECTION_SECONDS  1     // how many seconds to collect samples
#define INTERVAL_MS  (1000 / (FREQUENCY_HZ + 1)) // need for the timer 
#define CONVERT_G_TO_MS2    9.80665f   // accleration conversion

int myMaxData = FREQUENCY_HZ * COLLECTION_SECONDS;
int myCount = 0;
int myDelay = INTERVAL_MS;   // non-block delay in milliseconds
unsigned long myStart;

String readString;
bool mySendData = true;

void setup() {
  Serial.begin(115200);
  pinMode(LED_BUILTIN, OUTPUT);
  digitalWrite(LED_BUILTIN, HIGH);   // onboard LED, HIGH = off
  //while (!Serial) {}                 // do nothing and wait
  
  myStart = millis();   // set delay
  randomSeed(analogRead(A0));  // AO pin on XIAO does not have to be connected to anything
  Serial.println("accX,accY,accZ");  // CSV file heading titles
  
  if (!IMU.begin()) {
     Serial.println("Failed to initialize IMU!");
     while (1);
  }
}

void loop() {
  float x, y, z;
  if ( (millis() - myStart ) >= myDelay) {       
     myStart = millis();      //  reset the delay time
     myCount += 1;
     IMU.readAcceleration(x, y, z);
     // convert raw acceleration to acceleration due to gravity
     x *= CONVERT_G_TO_MS2;
     y *= CONVERT_G_TO_MS2;
     z *= CONVERT_G_TO_MS2;
     //Serial.println( String(myStart)+ "," + String(analogRead(A0)) + "," + String(analogRead(A1)) + "," +  String(analogRead(A2))  );
     if (myCount >= myMaxData){
        mySendData = false;   // stop sending data when amount reached
     }
     if (mySendData){
       // Serial.println( String(x) + "," + String(y) + "," +  String(z)  );
        Serial.println( String(x) + "," + String(y) + "," +  String(z) );
     } 
   }

while (Serial.available()) {
    delay(3);  
    char myChar = Serial.read();
    readString += myChar; 
  }

if (readString.length() > 0) {
    readString.trim();  // get rid of last weird character
    if (readString == "a"){
      digitalWrite(LED_BUILTIN, LOW); //onboard LED LOW = on
    }
    if (readString == "b"){
      digitalWrite(LED_BUILTIN, HIGH);
    }
    if (readString == "start"){
      mySendData = true;
      myStart = millis();      //  reset the delay time
      myCount = 0;
    }
    if (readString == "stop"){
      mySendData = false;
      Serial.println("Stopping at count: "+ String(myCount));  // CSV file heading titles
    }
    if (readString == "firstline"){
      Serial.println("accX,accY,accZ");  // CSV file heading titles
    }
    readString="";
  } 
  
}

Use at your own risk!
By Jeremy Ellis @rocksetta
Github at

https://github.com/hpssjellis/tinyMLjs/tree/main

https://hpssjellis.github.io/tinyMLjs/public/index.html

https://hpssjellis.github.io/tinyMLjs/public/acceleration/a00-best-acceleration.html

A couple of gotchas (as of Jun 22, 2023):

File names have to be in the format "name-lable.csv" or "name-lable (1).csv" or "name-lable (2).csv" etc. Unfortunately Android and iPhone don't auto make the numbering for you.
Android and Apple device have an opposite orientation, so I have made negative all the android motion data so when your phone is on a table z = -9.8 m/s^2 etc. When looking veritacally at your phone y = -9.8 m/s^2. The auto detect of this only works if an Android phone is in mobile format not "desktop site"
Real data has lots of rough data, machine learning models do not like missing data. If your results show "NaN" either your training data or classification data has errors. Note: If the loss is not changing your trained data probably has errors. The "clean,trim,fill" buttons might help.
Presently a CSV label upload bug happens sometimes. Easy to fix by entering the correct labels in the correct order. I will try to fix the issue when I figure out what is causing it.