WebMLSerial How to teach the Future of tinyML

Note: This is now a research page the summary of best practices page is at https://hpssjellis.github.io/tinyMLjs/public/index.html This page is about an attempt to bring TensorflowJS together with WebSerial for direct computer and cell phone Arduino style microcontroller connectivity to allow every University student to work on TinyML arduino style devices but with the power of a cell phone.

### Inspiration for this site: 1. My own desire to bring simple hands on Robotics and TinyML to all undergrads at all Universities, my Maker100 course is a starting point for that. 2. Andrej Karpathy's original work on Machine Learning using Javascript. 3. Vijay Janapa Reddi A Harvard Engineering Education Professor who mentioned to our TinyML4D group "How are we going to teach the future of TinyML now?" 4. Marcelo Jose Rovai Engineer, writer and forever student. Who asked if what I was doing with Javascript and webSerial could be used to simplify upload EdgeImpulse data using a .csv file. . .

csv-to-web-to-csv.html Proof CSV saving it works

webSerial-to-csv.html Proof WebSerial CSV works

webSerial-auto-csv.html Try this first with the Nano 33 Ble Sense for acceleration CSV file saving

z-webSerial-single.html Try this second with the Nano 33 Ble Sense for slow sensor data collection to CSV

New Style of using folders for the 3 files

a01/web-serial-csv.html Trying to allow frequency and duration changes

a02/ws-variable-resistor.html

a03-hug/index.html Testing HuggingFace object detection with a servo motor to point ot objects like a dial.

a04/a04-light-index.html April 2023 Testing light ML sensor.

Best Here

a05-resistor/a05-resistor-index.html USE THIS for CSV data files and EdgeImpulse WASM Javascript activation

Now I am just having fun doing some research

...

./research-may-2023/imageUpLatest This one gets shared. Latest drafts below

...

./research-may-2023/ml-app01.html

./research-may-2023/imageup27.html

./research-may-2023/imageup28.html

./sound-research/sound-ml-latest.html Share this one

./sound-research/simple-sound01.html just trying to save sounds from a webpage

./sound-research/simple-sound02.html just trying to save sounds from a webpage

./sound-research/simple-sound03.html just trying to save sounds from a webpage

./sound-research/simple-sound04.html just trying to save sounds from a webpage

...

./sound-research/tfjs-sound01.html

...

./sound-research/sound-ml01.html

./sound-research/sound-ml02.html

./sound-research/sound-ml03.html

...

./convert/xxd01.html

...

TensorflowJS conversions

./convert/xxd01.html

./convert/xxd02.html

./convert/xxd03.html

./convert/tfjs01-convert.html

./convert/tfjs02-convert.html

./convert/tfjs03-convert.html

...

Back to webSerial and trying to fix the android connectivity

./a06-android/index.html

./a06-android/accel01-webserial.html

./a06-android/accel02-webserial.html

./a06-android/accel03-webserial.html

./a06-android/accel04-google.html this one is not as good as the one before it
So it seems that some pixel phones work with web serial but many android and all IOS phones don't. I may look into bluetooth again here at https://github.com/hpssjellis/my-examples-of-web-BLE but for now I will leave this

Next it gets a bit more complex.

We need to convert the tensorflowJS file to tensorflowLite and then to a c-header file to finally put it on a microcontroller A fully loaded Gitpod can be built here https://gitpod.io/#github.com/hpssjellis/Gitpod-auto-tensorflowJS-to-arduino (If taking this option many of the below commands are automated.) The github for this site is at https://github.com/hpssjellis/Gitpod-auto-tensorflowJS-to-arduino So that site is working fine and can make a c-header file that works with many of the examples on this page https://github.com/hpssjellis/my-examples-for-the-arduino-portentaH7/tree/master/m09-Tensoflow My latest problem is that tensorflowLite does not work out of the box for the XIAO esp32S3. I might have to leave this issue for later.

Back to webSerial and the acceleration example. Can this be made to make it's own neural network

I guess it is best to work with the Nano33BleSense for acceleration but this code should be easy to adjust for the XIAO esp32s3 here and the Portenta using the MKR IU code here

./a07-accel-ml/a07-accel-ml.html This is the starting point

./a07-accel-ml/a07-accel-ml02.html Now lets make it just a basic file for uploading to a tfjs model

./a07-accel-ml/tfjs-accel01.html Testing making a tfjs model not having much luck

./a07-accel-ml/tfjs-accel02.html Testing making a tfjs model not having much luck

./a07-accel-ml/tfjs-accel03.html Testing making a tfjs model

...

./a07-accel-ml/a07-accel-nano33.txt

...

./a07-accel-ml/data-add01.html

...

./a07-accel-ml/cell-phone-accelerometer01.html This actually works

./a07-accel-ml/cell-phone-accelerometer02.html Trying this out