Embedded System Design for Machine Learning

Lecturer (assistant)
Number0000002069
Type
Duration3 SWS
TermWintersemester 2022/23
Language of instructionEnglish
Position within curriculaSee TUMonline

Admission information

Objectives

Upon completion of this module, students are able to: * Understand the design flow and design steps for deploying machine learning workloads on embedded devices. * Evaluate the trade-offs involved in executing machine learning workloads such as neural network inference in software and hardware for on embedded processors and dedicated accelerators. * Apply effectively model compression methods to embedded machine learning workloads and understand the theory behind the methods. * Apply hardware acceleration principles (SIMD, Vector, 2D systolic arrays) for accelerating ML workloads and know about the influence of the memory system. * Apply a state-of-the-art machine learning deployment flow to a simple machine learning application such as keyword recognition. * Implement the deployment code on an embedded processor platform (micro-controller board) and to design a simple hardware accelerator for the application that works in simulation.

Description

The lecture will cover the following contents: * Introduction to the design flow and design steps to deploy machine learn-ing workloads on embedded devices * Machine learning theory to understand the typical structure, operatorsand trade-offs in accuracy, memory and performance demands of machinelearning workloads * Neural Network Model Compression Methods: Number systems, Integerand sub-byte Quantization, Quantization aware training, Pruning, RankReduction. * Software Optimization Methods: Memory planning, target-aware operator optimization, operator fusing and tiling * Methods and basic HW blocks for embedded HW-acceleration (SIMD,Vector Instructions, loosely-coupled accelerators, memory systems) The lab part will cover the following contents to transfer the theory into practice: * Introduction to the Machine Learning Deployment Toolchain TVM * Training, model optimization and deployment of a keyword recognition application onto a low-power micro-controller board using TVM * Design of a simple HW accelerator to offload machine learning workload into hardware with test by simulation.

Prerequisites

Basic knowledge on embedded C and micro-controllers (e.g. in the form of a micro-controller programming lab) is assumed to be known. Basic knowledge of Hardware design in VHDL or Verilog (e.g. in the form of a hardware design lab or the lecture "Entwurf Digitaler Systeme mit VHDL und SystemC" (Prof. Ecker) is assumed to be known. Basic knowledge of machine learning algorithms (e.g. in the form of the computational intelligence lecture or the lecture "Machine Learning: Methods and Tools lecture") is recommended.

Teaching and learning methods

The module will consist of two parts, a weekly lecture and a parallel lab with three parts. The lecture will consist of classroom sessions with slide presentation. The exercises will be integrated in the lecture flow in order to apply the learned content directly on example problems. This will be done using activating methods such as group works. The lab will be split into three major tasks, that will be tutored, each being introduced in one classroom lab session. The students will work on these tasks in small groups on their own schedule to also train team work and independent work. The lab tasks will directly put the lecture content into practise, hence, following a problem-oriented learning approach. The course will be taught based on lecture material in the forms of slides and with additional exercises. The lab part will involve the work on a state of the art open-source simulation and deployment flow (TVM) that can be used either on university lab PCs or private machines. Additionally, low-power micro-controller boards will be used to demonstrate the application in real hardware.

Examination

1) The student will hand in code submissions for the three lab parts, which will be graded as passed or fail. All three submissions need to get grades as passed to participate in the final exam. (Coursework) 2) In the final exam (90min written or 30min oral), the students will answer questions from the lecture content and from the lab part to test their understanding of the theoretical and practical aspects of embedded machine learning. (Examination, 100% of grade)

Recommended literature

There exist textbooks covering the content of the lectures so far. It is planned to provide a lecture script based on the lecture contents in the future. The following books cover parts of the lecture’s content and provide related information: * Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, Joel S. Emer;Efficient Processing of Deep Neural Networks; Morgan & Claypool Publishers * David Patterson, John Hennessy;Computer Organization and Design RISC-V Edition - The Hardware Software Interface; Elsevier

Links


All courses

Bachelorbereich: BSc-EI, MSE, BSEDE

 

WS

SS

Diskrete Mathematik für Ingenieure (BSEI, EI00460)

Discrete Mathematics for Engineers (BSEDE ) (Schlichtmann) (Januar)

 

 

WS

SS

Entwurf digitaler Systeme mit VHDL u. System C (BSEI, EI0690) (Ecker)

 

 

SS

Entwurfsverfahren für integrierte Schaltungen (MSE, EI43811) (Schlichtmann)

 

WS

 

Methoden der Unternehmensführung (BSEI, EI0481) (Weigel)

 

WS

 

Praktikum System- und Schaltungstechnik (BSEI, EI0664) (Schlichtmann et al.)

 

 

SS

Schaltungssimulation (BSEI, EI06691) (Gräb/Schlichtmann)

 

 

Masterbereich: MSc-EI, MSCE, ICD

 

SS

Advanced Topics in Communication Electronics (MSCE, MSEI, EI79002)

 

WS

 

Electronic Design Automation (MSCE, MSEI, EI70610) (B. Li, Tseng)

 

WS

 

Design Methodology and Automation (ICD) (Schlichtmann) (Nov)

 

WS

SS

Machine Learning: Methods and Tools (MSCE, MSEI, EI71040) (Ecker)

 

WS

SS

SS

Mathematical Methods of Circuit Design (MSCE, MSEI, EI74042) (Gräb)

Simulation and Optimization of Analog Circuits (ICD) (Gräb) (Mai)

 

 

WS

 

Mixed Integer Programming and Graph Algorithms in Engineering Problems (MSCE, MSEI, EI71059) (Tseng)

 

WS

SS

Numerische Methoden der Elektrotechnik (MSEI, EI70440) (Schlichtmann oder Gräb)

 

WS

WS

SS

Seminar VLSI-Entwurfsverfahren (MSEI, EI7750) (Schlichtmann/Müller-Gritschneder)

Seminar on Topics in Electronic Design Automation (MSCE, EI77502) (Schlichtmann/Müller-Gritschneder)

 

 

WS

SS

Synthesis of Digital Systems (MSCE, MSEI, EI70640) (Müller-Gritschneder)

 

WS

 

Testing Digital Circuits (MSCE, MSEI, EI50141) (Otterstedt)

 

WS

 

Timing of Digital Circuits (MSCE, MSEI, EI70550) (B. Li, Zhang)

 

WS

SS

VHDL System Design Laboratory (MSCE, MSEI, EI7403) (Schlichtmann)

 

MSE: Munich School of Engineering (TUM)

BSEDE: Bachelor of Science in Electronics and Data Engineering (TUM-Asia)

ICD: Master of Science in Integrated Circuit Design (TUM-Asia)

MSCE: Master of Science in Communications Engineering (TUM)

MSEI: Master of Science in Elektrotechnik und Informationstechnik

BSEI: Bachelor of Science in Elektrotechnik und Informationstechnik

 

Aktuelle Infos zur Lehre/Current information on teaching: https://www.tum.de/die-tum/aktuelles/coronavirus/studium/, www.ei.tum.de