Basic Calculus Explained for Machine Learning

So the purpose of this simple article is to explain the concept of simple calculus to understand how does gradient descent work.

CONCEPT

So let’s say in my office, it takes me 10 seconds (time) to travel 25 meters (distance) to that cute colleague and this how a concept is represented as the graph below:

If we want to express the above situation as a function, then it would be:

distance = speed * time

which the speed is 1 meter per second.

So for this case, speed is the first derivative of the distance function above.

As speed describes the rate of change of distance over time. When people say taking the first derivative of a certain function. It means finding out the rate of change of a function.

Continue with the example, turned out that I didn’t walk a constant speed towards my colleague but I accelerated (my speed increased over time):

It still took me 10 seconds to travel from my seat to my colleague’s seat but I walked faster and faster:

speed = acceleration * time

So the first derivative of the speed function is acceleration:

Now you will realise that the speed function is closely related to the distance function right? In fact, the second derivative of the distance function is acceleration, which is denoted by:

WHY IS THIS USEFUL?

Especially why is this useful for understanding Machine Learning?

Let’s use another example, the below function denotes the per-unit cost function for producing an iWatch:

If you look at the function above, you will see as you produce more units, the per-unit costs continue to decrease until a point where it starts to increase.

If I want to optimise the per-unit production cost at its minimal to optimise efficiency, this will require me to find the rate of change at zero which is when the per-unit production costs begin to change from decreasing to increasing. For this, I would need to take the first derivative of the per-unit cost function to obtain:

So now if you use to obtain the first derivative and make it equal zero and solve the equation, you can obtain the optimal quantity/unit to produce.

Although it looks very simple with this example and even with the human eye, you can see the optimal unit is around 45. But in real life, the per-unit cost function is very complicated and it is not only determined by how many units but perhaps it is also affected by the oil prices, the number of workers available etc. This will make the function to be multi-dimensional and extremely complicated, which in those scenarios, calculus will be the key to optimise functions!

BACK TO MACHINE LEARNING

In many situations in Machine Learning, the cost function/loss function with respect to the model’s parameters are also multi-dimensional, hence we always turn to Calculus to optimise our model’s parameters.

To learn more about Machine Learning/Deep Learning, you can also take a look at my latest article.

Related posts:

It had been 8 to 9 months working in a technology startup and I was pretty focused on making a difference. I remember telling my family how I was growing in my job and how 2022 was going to be a…

Security Analysis of the Android Phones usually requires a rooted phone along with custom requirements for certain conditions like a minimum Android tier, certain Hardware, peripherals, etc. This…

Shit. Shit. Shitty shit shit. Whaddo I do? OK, Ben, think. THINK. You’re basically a doctor. Someday, you’re going to be faced with patients. Hopefully they won’t be bleeding out their eyeballs and…