Lazy Programmer

Your source for the latest in deep learning, big data, data science, and artificial intelligence. Sign up now

Featured partners

Neural Ordinary Differential Equations

December 15, 2018

Very interesting paper that got the Best Paper award at NIPS 2018.

“Neural Ordinary Differential Equations” by Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud.

Comes out of Geoffrey Hinton’s Vector Institute in Toronto, Canada (although he is not an author on the paper).

For those of you who have ever programmed simulations of systems of differential equations, the motivation behind this should be quite intuitive.

Recall that a derivative is the same thing as the slope of a tangent line, and can be approximated by the usual “rise over run” formula for small time steps \( \Delta t \).

$$ \frac{dh}{dt} \approx \frac{h(t + \Delta t) – h(t)}{\Delta t}$$

Here’s a picture of that if you forgot what it looks like:


Normally, the derivative is known to be some function \( \frac{dh}{dt} = f(h, t) \).

Your job in writing a simulation is to find out how \( h(t) \) evolves over time.

Here’s a picture of how that works (using different symbols):


Since our job is to find the next value of \( h(t) \), we can rearrange the above to get:

$$ h(t + \Delta t) = h(t) + f(h(t), t) \Delta t $$

Typically the time step is just \( 1 \), so we can rewrite the above as:

$$ h_{t+1} = h_t + f(h_t, t) $$

Researchers noticed that this looks a lot like the residual network layer that is often used in deep learning!

In a residual network layer, \( h_t \) represents the input value, \( h_{t+1} \) represents the output value, and \( f(h_t, t) \) represents the residual.

Here’s a picture of that (using different symbols):



At this point, the question to ask is, if a residual network layer is just a difference equation that approximates a differential equation, can there be a neural network layer that is an actual differential equation?

How would backpropagation be done?

This paper goes over all that and more.

Read the paper here!

Go to comments

Black Friday 2018 – Udemy’s BIGGEST Sale of the YEAR is back!

November 14, 2018

Deep Learning and AI Courses for just $9.99

Black Friday 2018

Udemy’s BIGGEST Sale of the YEAR is back!

I know a lot of you have been waiting for this – well here it is – the LOWEST price possible on ALL Udemy courses (yes, the whole site!)

For the next 7 days, ALL courses on Udemy (not just mine) are available for just $9.99!

For my courses, please use the coupons below (included in the links below), or if you want, enter the coupon code: NOV2018.

For prerequisite courses (math, stats, Python programming) and all other courses (yoga, guitar, photography, whatever else you want to learn), follow the links at the bottom.

Since ALL courses on Udemy are on sale, if you want any course not listed here, just click the general (site-wide) link, and search for courses from that page.


And just as important, $9.99 coupons for some helpful prerequisite courses. You NEED to know this stuff to understand machine learning in-depth:

General (site-wide):
Calc 1
Calc 2
Calc 3
Linalg 1
Linalg 2
Probability (option 1)
Probability (option 2)
Probability (option 3)



As you know, I’m the “Lazy Programmer”, not just the “Lazy Data Scientist” – I love all kinds of programming!


iOS courses:

Android courses:

Ruby on Rails courses:

Python courses:

Big Data (Spark + Hadoop) courses:

Javascript, ReactJS, AngularJS courses:



Into Yoga in your spare time? Photography? Painting? There are courses, and I’ve got coupons! If you find a course on Udemy that you’d like a coupon for, just let me know and I’ll hook you up!

Go to comments

NEW course! Recommender Systems and Deep Learning in Python

September 13, 2018

Recommender Systems and Deep Learning in Python

So excited to tell you about my new course!

[if you don’t want to read my little spiel just click here to get your coupon:]

Believe it or not, almost all online businesses today make use of recommender systems in some way or another.

What do I mean by “recommender systems”, and why are they useful?

Let’s look at the top 3 websites on the Internet, according to Alexa: Google, YouTube, and Facebook.

Recommender systems form the very foundation of these technologies.

Google: Search results (why Google is a billion dollar company!)

YouTube: Video dashboard (and recommendations to the right of every video)

Facebook: News feed

This course is a big bag of tricks that make recommender systems work across multiple platforms.

We’ll look at popular news feed algorithms, like RedditHacker News, and Google PageRank.

We’ll look at Bayesian recommendation techniques that are being used by a large number of media companies today.


But this course isn’t just about news feeds.

Companies like AmazonNetflix, and Spotify have been using recommendations to suggest products, movies, and music to customers for many years now.

These algorithms have led to billions of dollars in added revenue.

So I assure you, what you’re about to learn in this course is very real, very applicable, and will have a huge impact on your business.


For those of you who like to dig deep into the theory to understand how things really work, you know this is my specialty and there will be no shortage of that in this course. We’ll be covering state of the art algorithms like matrix factorization and deep learning (making use of both supervised andunsupervised learning), and you’ll learn a bag full of tricks to improve upon baseline results.


Whether you sell products in your e-commerce store, or you simply write a blog – you can use these techniques to show the right recommendations to your users at the right time.

If you’re an employee at a company, you can use these techniques to impress your manager and get a raise!


I’ll see you in class!



Note: this course is NOT a part of my deep learning series (it’s not Deep Learning part 11) because while it contains a major deep learning component, a lot of the course uses non-deep learning techniques as well. The deep learning parts apply modified neural network architectures and deep learning technologies to the recommender problem.

Go to comments

Special Announcment: Deep Learning Keras Book!

September 12, 2018

Simple Deep Learning for Programmers

Learn Deep Learning via Keras examples with absolutely no math

I’m always intrigued when students tell me they want to learn deep learning without doing any math.

I was explaining to someone just yesterday – if you look at <insert famous deep learning book by famous deep learning researcher here> – the entire thing is actually cover to cover equations. Ha!

Anyhow, I wanted to test this hypothesis. How far can one get, if they try to learn deep learning via an API?

So I made this little book. It’s full of Keras examples, starting from a basic feedforward neural network, then adding some modern techniques like dropout and batch norm, then moving to more advanced architectures like CNNs and RNNs.

Of course, if you are a reader of my newsletter, you probably aren’t afraid of math!

But, I thought I’d share this book with you anyway, since it contains some interesting examples that you haven’t seen in my courses before.

– CIFAR dataset
– time series prediction using an RNN
– machine translation using a Bidirectional RNN (not a seq-to-seq model as in my Advanced NLP course)

This would also be a great opportunity to brush up on your Keras skills, which are going to be useful for my next course (hopefully coming out in a few days!)

Finally – I’ve also linked below my related book, “Simple Machine Learning for Programmers” – it is a similar experiment in teaching about machine learning using an API with no math. It’s the same as the machine learning section of my Numpy course but I know some students like to have written versions of things so they can read on the subway / airplane. If so, check it out!

Get the book now
Go to comments

Using GANs for “Dance Transfer” (image-to-image translation with spatio-temporal smoothing)

August 24, 2018

Go to comments