Lecture 16

Domain-specific architectures. Tensor Processing Unit.

Lecture

Slides (PDF, PPTX).

Outline

• Performance limitations of modern hardware
• Domain-specific architectures
• Neural networks
• Tensor processing unit
• Systolic arrays

Workshop

• Cloud TPU beginner’s guide
• How to take advantage of GPUs and TPUs for your ML project (Coding TensorFlow)
• TPU Lab in Google Colab

Homework

Play with TPU at Google Colab.

References

• Domain-specific architectures. Chapter 7 in [CAQA].
• TPU (Wikipedia)
• TPU (Google)
• Cloud TPU beginner’s guide
• Dave Patterson. Evaluation of the Tensor Processing Unit. Video. Slides.
• Dave Patterson et al. In-Datacenter Performance Analysis of a Tensor Processing Unit​.
• Onur Mutlu. Computer Architecture - Lecture 27: Systolic Arrays. Video
• A Domain-Specific Architecture for Deep Neural Networks
• Google supercharges machine learning tasks with TPU custom chip
• Systolic array (Wikipedia)
• TensorFlow (Wikipedia)
• Convolutional neural network (Wikipedia)
• Multilayer perceptron (Wikipedia)
• Recurrent neural network (Wikipedia)
• Application-specific integrated circuit (ASIC) (Wikipedia)