Seminar - From Source to Speed: Performance Tools and Techniques in HPC/AI (IN0014, IN2107)
Prof. Dr. Martin Schulz, Urvij Saroliya, Yi Ju, Muhammad Arslan Ansari, Felix Jung
| Pre-course meeting: | Date: 02.02.2026 Time: 15:00 Location: tum-conf.zoom-x.de/j/68297999889 |
| Kick-off meeting: | tbd |
| ECTS: | 5 |
| Language: | English |
| Type: | Seminar, 2S |
| Moodle course: | tbd |
| Registration: | Matching System |
Modern computing systems involves a wide range of architectures — from multi-core CPUs and GPUs to large-scale clusters and specialized AI accelerators — making performance analysis and optimization increasingly challenging. At the same time, high performance computing application and large AI models, especially neural language models, are becoming ever more expensive to train and deploy, both in terms of energy consumption and hardware requirements. These rising costs highlight the need for systematic, predictive approaches to performance analysis that go beyond trial-and-error optimization.
Performance modeling and analysis tools provide the foundation to understand how performance emerges from source code, algorithms, and hardware architectural characteristics. They enable developers and researchers to identify bottlenecks, reason about scalability, and make informed decisions about optimization strategies and hardware selection — crucial for both traditional HPC applications and modern AI workloads.
This seminar introduces and critically discusses performance tools and modeling techniques for high performance computing applications and HPC/AI systems. Students will study classical and modern approaches that explain how performance scales with problem size, parallelism, and hardware capabilities, and how these approaches can be applied to reduce execution time, improve energy efficiency, and efficiently explore architectural design spaces.
Key topics covered in the seminar include:
- Amdahl’s Law and related scaling principles
- Roofline: an insightful visual performance model for multicore architectures
- GPU Performance Analysis Tools and Modeling Techniques
- Scaling laws for neural language models
- Tools and techniques for deep learning performance analysis
- Predictive modeling for efficient architectural design space exploration
- Empirical Performance Models
- MPI Modeling [Postal, LogP family]
- Quantum Software Stack profiling and performance analysis