This article is part of a series on Common Beginner Questions.
One very common “naive beginner” question I get is: “Why do I have to implement machine learning algorithms?”
It must drive these people NUTS to hear me constantly repeat the phrase: “if you can’t implement it, then you don’t understand it”.
“I understand it just fine!“, you might proclaim in protest.
Hey, if that’s you, I’d love to hear from you. Email me any time using the contact form above and tell me why I’m wrong.
It must drive these people to the brink of insanity to learn that the famed physicist Richard Feynman had a similar quote:
“What I cannot create, I do not understand.”
Here are 2 common objections:
1) It’s pointless to reinvent the wheel
2) Everybody learns differently
Both of these are flat out wrong. I will explain why below.
But, if you don’t trust what I have to say, then you should at least trust what others have to say. Jump down to the final section on “Support From Unbiased Sources” if that’s you.
It’s pointless to reinvent the wheel
People often confuse the “learning process” with “application of knowledge”.
They often try to jump straight to “application of knowledge”, foregoing the “learning process”, ultimately leading to “application of faulty knowledge”.
“But I will never do this on the job”, they say.
Of course you will probably never implement a well-established algorithm on the job. You are missing the point.
Implementing things requires you to understand every piece of the puzzle.
If you don’t understand every piece of the puzzle such that implementing ML algorithms is easy/fun for you, then clearly, this implies there are holes in your understanding.
Many beginners simply assume their understanding is perfect (mostly because they have never implemented anything, so they don’t even know what it truly means to understand something well).
But sometimes, you will implement algorithms on the job. This often happens in higher-level (and consequently higher-paid) roles. That sounds nice, doesn’t it?
In these roles, you’re not just plugging and playing SVMs which someone would learn to do in their first day of ML training. In order to beat the competition, you need to build new things. Things the world hasn’t seen before. Things that need to be implemented from scratch because they simply do not exist yet.
Back to the point about the “learning process” – you may think I’m full of it.
We use pre-built stuff all the time: strings, lists, dictionaries, HTTP servers, etc.
It is a well-known rule among professional software engineers that you should never “roll your own”.
That is a correct fact, but you are applying this correct fact incorrectly.
In fact, implementing things from scratch is always part of learning anything that involves coding.
In your programming 101 class (i.e. first programming class in college), you will implement things like strings, lists, hash tables, etc.
Will you ever do that in the so-called “real world”?
Of course not!
It would be insane to implement strings yourself on your own project.
But if you think that’s what I’m saying, then again you are missing the point. It means you are confusing the “learning process” with the “application of knowledge”.
This pattern repeats through all computer science courses.
In computer graphics courses, you’ll implement computer graphics algorithms from scratch.
In data structures and algorithms, you’ll implement binary trees, graph traversal algorithms, sorting, etc. from scratch.
Why would you implement sorting in the real-world when it’s already built-in to every popular language we use today?
To learn!
So too, do we implement ML algorithms to ensure we have a good understanding of them.
Summary of these points:
- Don’t confuse the “learning process” with “application of knowledge”
- Implementing is about learning, and you need to learn before you apply, lest you apply incorrectly
- Some jobs (the best-paying ones) do require you to implement
- Implementing is always part of learning, this has been happening since programming 101
Everybody learns differently
This is the worst excuse. Students always just use this excuse as a default when they don’t get what they want.
It literally applies to everything.
Fact: implementing an ML algorithm will improve your understanding of the algorithm.
i.e. “understanding after > understanding before”
It is impossible for the opposite to be true.
I double dog dare you to tell me that after implementing an ML algorithm, your understanding of it decreased.
i.e. it should never be the case that “understanding before > understanding after”.
Again, if this is you, I’d be very excited to hear what you have to say. Email me and tell me your thoughts.
What about the case where “understanding before == understanding after”?
This is an interesting case, but again it is unlikely to be relevant.
Supposing that implementing the ML algorithm did not improve your understanding at all, this implies that your understanding prior to the implementation was already good enough to perform the implementation with ease.
And if that were the case, why would you be complaining about it?
Thus, there’s a contradiction here.
You might claim that “implementing the algorithm didn’t help my understanding”, but this is clearly just you being dishonest because if it were really that simple for you, then there would be nothing to complain about.
Likely, you are frustrated because implementing is not simple for you. You might equate “frustration” to “not learning anything”, but this is simply because you are poor at learning.
You are a lazy learner because you are too afraid to get frustrated, too afraid to get things wrong. You avoid challenge. And by avoiding challenge, you do not learn. So perhaps you are right and it really is true: “understanding before == understanding after”, in the sense that both are poor. But obviously you don’t want that to be the case for you.
Support From Unbiased Sources
It’s important that you know it’s not just me saying this.
Aside from the fact that every serious college / university course around takes this approach (see above), everyday practitioners and learners find this to be true as well.
