Skip to Scheduled Dates
Course Overview
This workshop teaches you techniques for data-parallel deep learning training on multiple GPUs to shorten the training time required for data-intensive applications. Working with deep learning tools, frameworks, and workflows to perform neural network training, you’ll learn how to decrease model training time by distributing data to multiple GPUs, while retaining the accuracy of training on a single GPU.
Who Should Attend
Experienced Python Developers
Course Objectives
- Understand how data parallel deep learning training is performed using multiple GPUs
- Achieve maximum throughput when training, for the best use of multiple GPUs
- Distribute training to multiple GPUs using Pytorch
- Distributed Data Parallel
- Understand and utilize algorithmic considerations specific to multi-GPU training performance and accuracy
Course Outline
- Introduction
- Stochastic Gradient Descent and the Effects of Batch Size
- Training on Multiple GPUs with PyTorch Distributed Data Parallel (DDP)
- Maintaining Model Accuracy when Scaling to Multiple GPUs
- Workshop Assessment
- Final Review