See the corresponding YouTube video lecture here: https://youtu.be/3r5eNV7WZ6g
In this article, I will teach you how to setup your NVIDIA GPU laptop (or desktop!) for deep learning with NVIDIA’s CUDA and CuDNN libraries.
The main thing to remember before we start is that these steps are always constantly in flux – things change and they change quickly in the field of deep learning. Therefore I remind you of my slogan: “Learn the principles, not the syntax“. We are not doing any coding here so there’s no “syntax” per se, but the general idea is to learn the principles at a high-level, don’t try to memorize details which may change on you and confuse you if you forget about what the principles are.
This article is more like a personal story rather than a strict tutorial. It’s meant to help you understand the many obstacles you may encounter along the way, and what practical strategies you can take to get around them.
There are about 10 different ways to install the things we need. Some will work; some won’t. That’s just how cutting-edge software is. If that makes you uncomfortable, well, stop being a baby. Yes, it’s going to be frustrating. No, I didn’t invent this stuff, it is not within my control. Learn the principles, not the syntax!
This article will be organized into the following sections:
- Why you need this guide
- Choosing your laptop (i.e. a laptop that has an NVIDIA GPU)
- Choosing your Operating System
- Installing CUDA and CuDNN on Ubuntu and similar Linux OSes (Debian, Pop!_OS, Xubuntu, Lubuntu, etc.)
- Installing CUDA and CuDNN on Windows
- Installing GPU-enabled Tensorflow
- Installing GPU-enabled PyTorch
- Installing GPU-enabled Keras
- Installing GPU-enabled Theano
Why you need this guide
If you’ve never setup your laptop for GPU-enabled deep learning before, then you might assume that there’s nothing you need to do beyond buying a laptop with a GPU. WRONG!
You need to have a specific kind of laptop with specific software and drivers installed. Everything must work together.
You can think of all the software on your computer as a “stack” of layers.
At the lowest layer, you have the kernel (very low-level software that interacts with the hardware) and at higher levels you have runtimes and libraries such as SQLite, SSL, etc.
When you write an application, you need to make use of lower-level runtimes and libraries – your code doesn’t just run all by itself.
So, when you install Tensorflow (as an example), that depends on lower-level libraries (such as CUDA and CuDNN) which interact with the GPU (hardware).
If any of the layers in your stack are missing (all the way from the hardware up to high-level libraries), your code will not work.
Low-Level = Hardware
High-Level = Libraries and Frameworks
Choosing your laptop
Not all GPUs are created equal. If you buy a MacBook Pro these days, you’ll get a Radeon Pro Vega GPU. If you buy a Dell laptop, it might come with an Intel UHD GPU.
These are no good for machine learning or deep learning.
You will need a laptop with an NVIDIA GPU.
Some laptops come with a “mobile” NVIDIA GPU, such as the GTX 950m. These are OK, but ideally you want a GPU that doesn’t end with “m”. As always, check performance benchmarks if you want the full story.
I would also recommend at least 4GB of RAM (otherwise, you won’t be able to use larger batch sizes, which will affect training).
In fact, some of the newer neural networks won’t even fit on the RAM to do prediction, never mind training!
One thing you have to consider is if you actually want to do deep learning on your laptop vs. just provisioning a GPU-enabled machine on a service such as AWS (Amazon Web Services).
These will cost you a few cents to a dollar per hour (depending on the machine type), so if you just have a one-off job to run, you may want to consider this option.
I already have a walkthrough tutorial in my course Modern Deep Learning in Python about that, so I assume if you are reading this article, you are rather interested in purchasing your own GPU-enabled computer and installing everything yourself.
Personally, I would recommend Lenovo laptops. The main reason is they always play nice with Linux (we’ll go over why that’s important in the next section). Lenovo is known for their high-quality and sturdy laptops and most professionals who use PCs for work use Thinkpads. They have a long history (decades) of serving the professional community so it’s nearly impossible to go wrong. Other brands generally have lots of issues (e.g. sound not working, WiFi not working, etc.) with Linux.
Here are some good laptops with NVIDIA GPUs:
($694.95)
This one only has an i5 processor and 8GB of RAM, but on the plus side it’s cost-effective. Note that the prices were taken when I wrote this article; they might change.
($964.00)
Same as above but different specs. 16GB RAM with an i7 processor, but only 256GB of SSD space. Same GPU. So there are some tradeoffs to be made.
($998.00)
This is the best option in my opinion. Better or equal specs compared to the previous two. i7 processor, 24GB of RAM (32GB would be ideal!), lots of space (1TB HD + 512GB SSD), and the same GPU. Bonus: it’s nearly the same price as the above (currently).
($1,830.00)
Pricier, but great specs. Same GPU!
($3,472.69)
If you really want to splurge, consider one of these big boys. Thinkpads are classic professional laptops. These come with real beast GPUs – NVIDIA Quadro RTX 5000 with 16GB of VRAM.
You’ve still got the i7 processor, 16GB of RAM, and a 512GB NVMe SSD (basically a faster version of already-super-fast SSDs). Personally, I think if you’re going to splurge, you should opt for 32GB of RAM and a 1TB SSD.
If you’ve watched my videos, you might be wondering: what about a Mac? (I use a Mac for screen recording).
Macs are great in general for development, and they used to come with NVIDIA GPUs (although those GPUs are not as powerful as the ones currently available for PCs). Support for Mac has dropped off in the past few years, so you won’t be able to install say, the latest version of Tensorflow, CUDA, and CuDNN without a significant amount of effort (I spent probably a day and just gave up). And on top of that the GPU won’t even be that great. Overall, not recommended.
Choosing your Operating System
As I mentioned earlier, you probably want to be running Linux (Ubuntu is my favorite).
Why, you might ask?
“Tensorflow works on Windows, so what’s the problem?”
Remember my motto: “Learn the principles, not the syntax“.
What’s the principle here? Many of you probably haven’t been around long enough to know this, but the problem is, many machine learning and deep learning libraries didn’t work with Windows when they first came out.
So, unless you want to wait a year or more after new inventions and software are being made, then try to avoid Windows.
Don’t take my word for it, look at the examples:
- Early on, even installing Numpy, Matplotlib, Pandas, etc. was very difficult on Windows. I’ve spent hours with clients on this. Nowadays you can just use Anaconda, but that’s not always been the case. At the time of this writing, things only started to shape up a few years ago.
- Theano (the original GPU-enabled deep learning library) initially did not work on Windows for many years.
- Tensorflow, Google’s deep learning library and the most popular today, initially did not work on Windows.
- PyTorch, a deep learning library popular with the academic community, initially did not work on Windows.
- OpenAI Gym, the most popular reinforcement learning library, only partially works on Windows. Some environments, such as MuJoCo and Atari, still have no support for Windows.
There are more examples, but these are the major historical “lessons” I point to for why I normally choose Linux over Windows.
One benefit of using Windows is that installing CUDA is very easy, and it’s very likely that your Windows OS (on your Lenovo laptop) will come with it pre-installed. The original use-case for GPUs was gaming, so it’s pretty user-friendly.
If you purchase one of the above laptops and you choose to stick with Windows, then you will not have to worry about installing CUDA – it’s already there. There is a nice user interface so whenever you need to update the CUDA drivers you can do so with just a few clicks.
Installing CuDNN is less trivial, but the instructions are pretty clear (https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#installwindows). Simply download the zip file, unzip it, copy the files to the locations specified in the instructions, and set a few environment variables. Easy!
TO BE CLEAR:
Aside from the Python libraries below (such as Tensorflow / PyTorch) you need to install 2 things from NVIDIA first:
- CUDA (already comes with Windows if you purchase one of the above laptops, Ubuntu instructions below)
- CuDNN (you have to install it yourself, following the instructions on NVIDIA’s website)
DUAL-BOOTING:
I always find it useful to have both Windows and Ubuntu on-hand, and if you get the laptop above that has 2 drives (1TB HD and 512GB SSD) dual-booting is a natural choice.
These days, dual booting is not too difficult. Usually, one starts with Windows. Then, you insert your Ubuntu installer (USB stick), and choose the option to install Ubuntu alongside the existing OS. There are many tutorials online you can follow.
Hint: Upon entering the BIOS, you may have to disable the Secure Boot / Fast Boot options.
INSTALLING PYTHON:
I already have lectures on how to install Python with and without Anaconda. These days, Anaconda works well on Linux, Mac, and Windows, so I recommend it for easy management of your virtual environments.
Environment Setup for UNIX-Like systems (includes Ubuntu and MacOS) without Anaconda
Environment Setup for Windows and/or Anaconda
Installing CUDA and CuDNN on Ubuntu and similar Linux OSes (Debian, Pop!_OS, Xubuntu, Lubuntu, etc.)
Ok, now we get to the hard stuff. You have your laptop and your Ubuntu/Debian OS.
Can you just install Tensorflow and magically start making use of your super powerful GPU? NO!
Now you need to install the “low-level” software that Tensorflow/Theano/PyTorch/etc. make use of – which are CUDA and CuDNN.
This is where things get tricky, because there are many ways to install CUDA and CuDNN, and some of these ways don’t always work (from my experience).
Examples of how things can “randomly go wrong”:
- I installed CUDA on Linux Mint. After this, I was unable to boot the machine and get into the OS.
- Pop!_OS (System76) has their own versions of CUDA and CuDNN that you can install with simple apt commands. Didn’t work. Had to install them the “regular way”.
- Updating CUDA and CuDNN sucks. You may find the nuclear option easier (installing the OS and drivers from scratch)
Here is a method that consistently works for me:
- Go to https://developer.nvidia.com/cuda-downloads and choose the options appropriate for your system. (Linux / x86_64 (64-bit) / Ubuntu / etc.). Note that Pop!_OS is a derivative of Ubuntu, as is Linux Mint.
- You’ll download a .deb file. Do the usual “dpkg -i <filename>.deb” to run the installer. CUDA is installed!
- Next, you’ll want to install CuDNN. Instructions from NVIDIA are here: https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#ubuntu-network-installation
Those instructions are subject to change, but basically you can just copy and paste what they give you (don’t copy the below, check the site to get the latest version):
sudo dpkg -i \ http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb sudo apt-get update && sudo apt-get install libcudnn7 libcudnn7-dev
Installing CUDA and CuDNN on Windows
If you decided you hate reinforcement learning and you’re okay with not being able to use new software until it becomes mainstream, then you may have decided you want to stick with Windows.
Luckily, there’s still lots you can do in deep learning.
As mentioned previously, installing CUDA and CuDNN on Windows is easy.
If you did not get a laptop which has CUDA preinstalled, then you’ll have to install it yourself. Go to https://developer.nvidia.com/cuda-downloads, choose the options appropriate for your system (Windows 10 / x86_64 (64-bit) / etc.)
This will give you a .exe file to download. Simply click on it and follow the onscreen prompts.
As mentioned earlier, installing CuDNN is a little more complicated, but not too troublesome. Just go to https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#installwindows and follow NVIDIA’s instructions for where to put the files and what environment variables to set.
Installing GPU-enabled Tensorflow
Unlike the other libraries we’ll discuss, there are different packages to separate the CPU and GPU versions of Tensorflow.
The Tensorflow website will give you the exact command to run to install Tensorflow (it’s the same whether you are in Anaconda or not).
So you would install it using either:
pip install tensorflow
pip install tensorflow-gpu
Since this article is about GPU-enabled deep learning, you’ll want to install tensorflow-gpu.
UPDATE: Starting with version 2.1, installing “tensorflow” will automatically give you GPU capabilities, so there’s no need to install a GPU-specific version (although the syntax still works).
After installing Tensorflow, you can verify that it is using the GPU:
tf.test.is_gpu_available()
This will return True if Tensorflow is using the GPU.
Installing GPU-enabled PyTorch
Nothing special nowadays! Just do:
pip install torch
as usual.
To check whether PyTorch is using the GPU, you can use the following commands:
In [1]: import torch
In [2]: torch.cuda.current_device()
Out[2]: 0
In [3]: torch.cuda.device(0)
Out[3]: <torch.cuda.device at 0x7efce0b03be0>
In [4]: torch.cuda.device_count()
Out[4]: 1
In [5]: torch.cuda.get_device_name(0)
Out[5]: 'GeForce GTX 950M'
In [6]: torch.cuda.is_available()
Out[6]: True
Installing GPU-enabled Keras
Luckily, Keras is just a wrapper around other libraries such as Tensorflow and Theano. Therefore, there is nothing special you have to do, as long as you already have the GPU-enabled version of the base library.
Therefore, just install Keras as you normally would:
pip install keras
As long as Keras is using Tensorflow as a backend, you can use the same method as above to check whether or not the GPU is being used.
Installing GPU-enabled Theano
For both Ubuntu and Windows, as always I recommend using Anaconda. In this case, the command to install Theano with GPU support is simply:
conda install theano pygpu
If necessary, further details can be found at:
- http://deeplearning.net/software/theano/install_windows.html
- http://deeplearning.net/software/theano/install_ubuntu.html
SIDE NOTE: Unfortunately, I will not provide technical support for your environment setup. You are welcome to schedule a 1-on-1 but availability is limited.
Disclaimer: this post contains Amazon affiliate links.