How to write a machine learning program

Artificial Intelligence, Machine learning, Data science…
There are plenty of names for defining one of the most trendy topic in IT companies today.
Using powerful solutions won’t make you (and me) a specialist or a data scientist.
Even if we are trained enough with these solutions.

I wanted to allow me some time to create the step 1 of a machine learning framework.
I don’t wish to sell it or to opensource it, but understanding the principles will certainly be useful.
Because in the next years I want to be able to know how is made the software I use everyday.

Following the 3brown1blue serie on youtube which is very intuitive (https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi), I have tried to create that algorithm on my own, without looking at any snippet of code.

Here is my solution : https://github.com/libetl/neural-network/blob/master/network.ts#L242-L293 (those 50 lines is the learning algorithm)
Basically, a network is several objects, ordered in layers, holding each a weight of each object of the next layer + one bias value.
Every weight and bias is random at the begining.
We compute the result using weight1 * x + … + weightK * x + bias, x being the input.
To avoid the result being out of reasonable ranges, we output an activation function of that result (the result can only be between 0 and 1 for example, or between -n1 and n2)

When doing that with random weights and biases, the results at the rightmost layer is of course random.

So here we go with the learning part :
For each dataset, each weight and bias is changed to see if we can get a result closer to the expected one. we move each weight and each bias right or left depending on the result until we find a “minimum” (local minimum because there can be plenty of them very far but it would be too costly to find)
You can find the search of the local minimum in my repository at this location : https://github.com/libetl/neural-network/blob/master/network.ts#L361-L395

What can you do at your scale ?
– Try that user interface related to my small library : https://libetl.github.io/neural-network . Use a theory and calibrate the parameters / layers to make the green surface match the dotted surface.
You can save a model and import it, here is an example of successful model matching x^2+y^2<1 : https://gist.github.com/libetl/90afddabd27b89848d0b7bebaf26d0dd
I have made a screencast of the training process : https://www.youtube.com/watch?v=yGEZJC1lYD4
– Try the TensorFlow playground and get started with TensorFlow : https://playground.tensorflow.org
– Learn more about machine learning and neuron networks by looking at the serie from 3brown1blue on youtube.
– Write your own program to make sure you have understood the principles

Now, you can teach a machine to learn.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.