Handy

What is this?

A neural network that tries to identify single numbers that you draw!

Edit: now works on phones!

Handy says...

Press "Read" when loaded!

The small image on the right is what's actually fed into the network.

What I did

1. First I trained a convolutional neural network through keras with the MNIST dataset. This just used the keras repo's defaults. It has an accuracy of 99.25% and could still be improved but is good enough for now.

Here are some examples of the training data:

2. Then I saved the model and used `tensorflowjs_converter` to convert. Here is a nifty tutorial.

3. In predict.js you'll find the tensorflow.js code used to import the saved model, convert an image into a tensor, then predict the class!

4. In draw.js you'll find the js code that allows drawing on the canvas. If you want to see the whole picture check out the repo.

Some thoughts

My motivation was more to muck around tensorflow.js, allow user input, and see if I could make a client-side implementation work.

Even if the network got a 99.25% score, you'll notice it misclassifies your drawings at times.

My current intuition is this is because I haven't done the needed work to preprocess the input from the canvas like for example, detecting the center of the drawing and trimming out whitespace or sharpening the image.

Also, I think accuracy could be improved by augmenting the training dataset by applying a range of effects like rotating, skewing, and playing with scale and then retraining.

If you want to make sure Handy sees your digits, try filling in the whole canvas with your digit. :)

---

Made with 🔥 by Beato.