Semantic image synthesis based only on adversarial supervision has recently made remarkable progress. Different from traditional GAN-based approaches, “OASIS” has redesigned the purpose of the discriminator as an N+1 semantic segmentation network, directly using the given semantic label maps as the ground truth for training. By providing stronger supervision to the discriminator as well as to the generator through spatially- and semantically aware discriminator feedback, OASIS has achieved SOA performance and remains a strong baseline even for powerful diffusion-based models, such as https://arxiv.org/pdf/2207.00050.pdf. Despite its great success, it is well known that semantic image synthesis based only on semantic maps often fails in complex environments where multiple instances occlude each other.

The goal of this work is to investigate whether OASIS can be extended to the panoptic case, both from the perspective of the learning objective and the network architecture.

Your work:

• Literature review of state-of-the-art semantic/ panoptic-aware image synthesis methods, e.g., https://arxiv.org/abs/2012.04781, https://arxiv.org/pdf/2004.10289.pdf.
• Analysis and identification of suitable panoptic segmentation backbones/ learning objectives. Possible starting point: https://arxiv.org/abs/2107.06278.
• Development of a prototype based on https://github.com/boschresearch/OASIS + evaluation thereof and comparison to the current state-of-the-art.
• Optional: opportunity to contribute to publications.

Requirements:

• Strong (proven) background in machine learning, as well as Python and common deep learning libraries such as TensorFlow or PyTorch.
• Self-motivation.

Anomaly detection plays an important role in many application scenarios. In the Industrial Internet of Things (IIoT), anomaly detection enables permanent monitoring and evaluation of machine and process data. In this way, machine failures can be predicted at an early stage, thus avoiding malfunctions, and making maintenance processes efficient.

Traditionally, the raw data is sent to centralized servers where large-scale systems perform analytics on the data gathered from all devices. However, this often leads to high network traffic, latency, and privacy issues. The goal of this work is to analyze the extent to which transformer-based anomaly detection models can be deployed directly to highly resource-constrained devices (MCUs).

Your work:

• Literature review of state-of-the-art anomaly detection methods, e.g. “Deep Learning for Anomaly Detection: A Review” (Pang et al.).
• Analysis and identification of deep learning-based models suitable for the deployment on resource-constrained devices (micro-controllers). Possible starting point: “TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data” (Tuli et al. 2022). GitHub: https://github.com/imperial-qore/TranAD.
• Development of a prototype + evaluation thereof and comparison to the current state-of-the-art using both benchmark and real-world datasets.
• Optional: opportunity to contribute to publications.

Requirements:

• Strong (proven) background in machine learning, as well as Python and common deep learning libraries such as TensorFlow or PyTorch.
• Self-motivation.

Machine Learning on the edge is becoming more and more popular nowadays. Especially for safety critical applications it is not acceptable to have any deviation of model outputs caused by the deployment method while other application might accept differences to the golden reference outputs to some degree.

Our TVM deployment flow is mainly based on the MLonMCU (https://github.com/tum-ei-eda/mlonmcu) tool. While it already supports validating model outputs, this functionality is rather limited.

Task Description:

• Goal: Reimplement the validation feature in MLonMCU to be more flexible and accurate
• Configuration: In additionon to bit-excact equivalences, the user may also allow deviations in some degree (absolute/relative delta)
• Flexibility: Provide an generic interface supporting several target-specific implementations (Offline: data compiled in ROM, Online: data sent via Semihosting, UART,...)
• Accuracy: Allow comparing intermediate values (between layers) in addition to just model outputs.

• Basic Knownledge of Machine Learning
• Experience with Embedded C programming
• Good Python Coding skills
• Ideally experience using TVM Machinne Learning Framework

DXF is short for Drawing Exchange Format or Drawing Interchange Format and is a vector file type. Engineers, designers, and architects often use the DXF format for drawings during product design. DXF is widely used in industrial production. Many devices such as printers, laser cutters, CNC machining, etc. accept DXF format as input. In this project, we would like you to implement a generic DXF library that can be used by different software to produce designs in DXF format. 

• Knowledge and experience in web design and web development or desktop application development
• Good understanding of JavaScript and PHP or Java Spring Boot or the wiliness to learn them
• Basic knowledge of Vue.js (or other frontend frameworks) or object-oriented programming or the cunning to know them
• Solution-oriented thinking

Microfluidic Large Scale Integration (mLSI) refers to the development of microfluidic chips with thousands of integrated micromechanical valves and control components. This technology is used in many areas of biology and chemistry and is a candidate for replacing today's conventional automation paradigm, which consists of Robots for handling liquids.

To enable automated control of mLSI, programming sequence for valve control and fluid inputs is essential. Although, not every researcher, especially biologists or chemists, has programming skills. We want to introduce an interactive but intuitive programming platform for microfluidic researchers that do not require any pre-experience with programming to overcome this problem.

This platform provides a visual programming language that allows users to define their needs by dragging and dropping the command blocks into a canvas. A flow simulation will show up on the side when users click on the run. 

• Knowledge and experience in web design and web development
• Good understanding of HTML5, CSS, JavaScript and PHP (or Java Spring Boot) or the wiliness to learn them
• Basic knowledge of SQL, jQuery and Vue.js (or other frontend frameworks) or the cunning to know them
• Solution-oriented thinking

Machine Learning (ML) on edge devinces (TinyML) can be a challenge in the presence of unreliable computations, e.g., in the presence of soft errors. Hardening the deployed, original inference system with Software Implemented Hardware Fault Tolerance (SIHFT) on algorthmic level can help mitigate such errors,

In this work, we want to look into applying the filter and input checksum (FIC) scheme to a TVM based deployment flow.

Related works:
Making Convolutions Resilient Via Algorithm-Based Error Detection Techniques
https://ieeexplore.ieee.org/abstract/document/9366780

- Good understanding of Linear Algebra
- Basic understanding of computer architectures
- C++, Python, TinyML/TVM

Semantic image synthesis based only on adversarial supervision has recently made remarkable progress. Different from traditional GAN-based approaches, “OASIS” has redesigned the purpose of the discriminator as an N+1 semantic segmentation network, directly using the given semantic label maps as the ground truth for training. By providing stronger supervision to the discriminator as well as to the generator through spatially- and semantically aware discriminator feedback, OASIS has achieved SOA performance and remains a strong baseline even for powerful diffusion-based models, such as https://arxiv.org/pdf/2207.00050.pdf. Despite its great success, it is well known that semantic image synthesis based only on semantic maps often fails in complex environments where multiple instances occlude each other.

The goal of this work is to investigate whether OASIS can be extended to the panoptic case, both from the perspective of the learning objective and the network architecture.

Your work:

• Literature review of state-of-the-art semantic/ panoptic-aware image synthesis methods, e.g., https://arxiv.org/abs/2012.04781, https://arxiv.org/pdf/2004.10289.pdf.
• Analysis and identification of suitable panoptic segmentation backbones/ learning objectives. Possible starting point: https://arxiv.org/abs/2107.06278.
• Development of a prototype based on https://github.com/boschresearch/OASIS + evaluation thereof and comparison to the current state-of-the-art.
• Optional: opportunity to contribute to publications.

Requirements:

• Strong (proven) background in machine learning, as well as Python and common deep learning libraries such as TensorFlow or PyTorch.
• Self-motivation.

Anomaly detection plays an important role in many application scenarios. In the Industrial Internet of Things (IIoT), anomaly detection enables permanent monitoring and evaluation of machine and process data. In this way, machine failures can be predicted at an early stage, thus avoiding malfunctions, and making maintenance processes efficient.

Traditionally, the raw data is sent to centralized servers where large-scale systems perform analytics on the data gathered from all devices. However, this often leads to high network traffic, latency, and privacy issues. The goal of this work is to analyze the extent to which transformer-based anomaly detection models can be deployed directly to highly resource-constrained devices (MCUs).

Your work:

• Literature review of state-of-the-art anomaly detection methods, e.g. “Deep Learning for Anomaly Detection: A Review” (Pang et al.).
• Analysis and identification of deep learning-based models suitable for the deployment on resource-constrained devices (micro-controllers). Possible starting point: “TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data” (Tuli et al. 2022). GitHub: https://github.com/imperial-qore/TranAD.
• Development of a prototype + evaluation thereof and comparison to the current state-of-the-art using both benchmark and real-world datasets.
• Optional: opportunity to contribute to publications.

Requirements:

• Strong (proven) background in machine learning, as well as Python and common deep learning libraries such as TensorFlow or PyTorch.
• Self-motivation.

The thesis will contribute to the advancement of on-device training for TinyML applications and provide insights into the challenges and prospects of the TinyML paradigm.

The neural network models are usually trained on high-performance computers and only the trained models are deployed to edge devices. But the statically trained model cannot adapt dynamically in a real environment and may result in low accuracy for new inputs. On-device training by learning from real-world data after deployment can greatly improve accuracy. Therefore, the goal is to enable the model to adapt to new data collected from sensors by fine-tuning a pre-trained model. 

The thesis will also explore the challenges of TinyML on-device learning. One of the main challenges is the limited hardware resources such as memory and computation that do not allow full backward computation.

Though the high computation cost makes training prohibitive for resource-constrained devices, there are some efficient algorithms like transfer learning for training only the final classifier layer, bias-only update, etc. to reduce the time and the computing power required to train a model.

Despite the challenges, on-device learning is an important area of research in TinyML, as it enables devices to learn and adapt to new data without relying on cloud computing or external servers. 

Your work:

1.    Literature review of existing works. The following references might be helpful to get started:

• https://upcommons.upc.edu/bitstream/handle/2117/353756/160036.pdf
• https://tinyml.mit.edu/on-device-training/
• https://ieeexplore.ieee.org/abstract/document/9211513?casa_token=2xvg7P_Sxs8AAAAA:HcoI_dEeElJDb92HXvjsLbNMpd77WHJd1UQ0KhNF8_xTFWuIwslud_KDa8hGbtJm1mzoXtoOUA
• https://www.politesi.polimi.it/bitstream/10589/187690/6/TinyML_on_device_neural_network_training%20fin.pdf

2.    Experimenting with an efficient algorithm that is a right fit for an ML motor control application

3.    Deployment of a model capable of on-device training

• Good knowledge of machine learning and embedded systems
• Self-motivation and ability to work independently

(all the details are in the PDF)

Software Implemented Hardware Fault Tolerance (SIHFT) aims to improve a digital systems resilience against hardware induced faults, e.g., soft errors due to radiation. One technique that can be deployed is Module Redudancy (MR), here, a computation is conducted from and in multiple hardware resources such that at a checkpoint can be established to verify the isolated results. In software, Dual Module Redundancy (DMR) can be implemented by splitting the architecture's register file and allocating each half to two independent computational threads each.
SIHFT always comes with a performance overhead. With DMR not only due to doubling the computation (>100%), but also because of increased register pressure. Thus, SIHFT is often deployed selectively to vulnerable code sections (Selective Hardening). Since DMR changes the register layout, it also changes an instruction set architecture's properties, e.g., its calling conventions. Mixing DMR and non-DMR code results in needed domain-crossing which are mostly placed at  function calls (coarse-grain).

In this work, we want to study and implement a working solution for selectively applying DMR with fine-grain domain crossing (within functions) .

- very good C++
- good knowledge of at least one assembly language, prefereably RISC-V
- good compiler basics, preferably some experience with LLVM

Siehe beigefügtes pdf

DXF is short for Drawing Exchange Format or Drawing Interchange Format and is a vector file type. Engineers, designers, and architects often use the DXF format for drawings during product design. DXF is widely used in industrial production. Many devices such as printers, laser cutters, CNC machining, etc. accept DXF format as input. In this project, we would like you to implement a generic DXF library that can be used by different software to produce designs in DXF format. 

• Knowledge and experience in web design and web development or desktop application development
• Good understanding of JavaScript and PHP or Java Spring Boot or the wiliness to learn them
• Basic knowledge of Vue.js (or other frontend frameworks) or object-oriented programming or the cunning to know them
• Solution-oriented thinking

Microfluidic Large Scale Integration (mLSI) refers to the development of microfluidic chips with thousands of integrated micromechanical valves and control components. This technology is used in many areas of biology and chemistry and is a candidate for replacing today's conventional automation paradigm, which consists of Robots for handling liquids.

To enable automated control of mLSI, programming sequence for valve control and fluid inputs is essential. Although, not every researcher, especially biologists or chemists, has programming skills. We want to introduce an interactive but intuitive programming platform for microfluidic researchers that do not require any pre-experience with programming to overcome this problem.

This platform provides a visual programming language that allows users to define their needs by dragging and dropping the command blocks into a canvas. A flow simulation will show up on the side when users click on the run. 

• Knowledge and experience in web design and web development
• Good understanding of HTML5, CSS, JavaScript and PHP (or Java Spring Boot) or the wiliness to learn them
• Basic knowledge of SQL, jQuery and Vue.js (or other frontend frameworks) or the cunning to know them
• Solution-oriented thinking

DXF is short for Drawing Exchange Format or Drawing Interchange Format and is a vector file type. Engineers, designers, and architects often use the DXF format for drawings during product design. DXF is widely used in industrial production. Many devices such as printers, laser cutters, CNC machining, etc. accept DXF format as input. In this project, we would like you to implement a generic DXF library that can be used by different software to produce designs in DXF format. 

• Knowledge and experience in web design and web development or desktop application development
• Good understanding of JavaScript and PHP or Java Spring Boot or the wiliness to learn them
• Basic knowledge of Vue.js (or other frontend frameworks) or object-oriented programming or the cunning to know them
• Solution-oriented thinking

The required steps can be described as follows:

• Get used to the MLonMCU Benchmarking flow
• Propose a concept for a remote execution of benchmarks
• Implement remote execution protocol
• Add more features to improve benchmarking throughput
• Add detailed documentation

• Experience with networking protocols
• Good Python programming
• Ideally experience working with UNIX-like operating systems

Microfluidic Large Scale Integration (mLSI) refers to the development of microfluidic chips with thousands of integrated micromechanical valves and control components. This technology is used in many areas of biology and chemistry and is a candidate for replacing today's conventional automation paradigm, which consists of Robots for handling liquids.

To enable automated control of mLSI, programming sequence for valve control and fluid inputs is essential. Although, not every researcher, especially biologists or chemists, has programming skills. We want to introduce an interactive but intuitive programming platform for microfluidic researchers that do not require any pre-experience with programming to overcome this problem.

This platform provides a visual programming language that allows users to define their needs by dragging and dropping the command blocks into a canvas. A flow simulation will show up on the side when users click on the run. 

• Knowledge and experience in web design and web development
• Good understanding of HTML5, CSS, JavaScript and PHP (or Java Spring Boot) or the wiliness to learn them
• Basic knowledge of SQL, jQuery and Vue.js (or other frontend frameworks) or the cunning to know them
• Solution-oriented thinking

Machine Learning (ML) on edge devinces (TinyML) can be a challenge in the presence of unreliable computations, e.g., in the presence of soft errors. Hardening the deployed, original inference system with Software Implemented Hardware Fault Tolerance (SIHFT) on algorthmic level can help mitigate such errors,

In this work, we want to look into applying the filter and input checksum (FIC) scheme to a TVM based deployment flow.

Related works:
Making Convolutions Resilient Via Algorithm-Based Error Detection Techniques
https://ieeexplore.ieee.org/abstract/document/9366780

- Good understanding of Linear Algebra
- Basic understanding of computer architectures
- C++, Python, TinyML/TVM

Siehe beigefügten Aushang.

The ETISS Simulator can be used for simulating programs compiled for a variety of targets on a instruction level. The goal of the work is to make performance analysis (profiling) possible with ETISS on several abstraction levels.

Task Description:

• Survey state-of-the art profiling methods for embedded SW
• Get used to existing set of tools (ETISS, Tracer, RISC-V SW Compiler)
• Generate Instruction Traces of Program Execution with ETISS (already implemented)
• Analyse Trace to count how often specific parts of code are executed on several levels: Instruction, "Basic Block", (For ML workloads: per Layer)
• Visualize Profiling Results
• Convert trace or profiling result to be compatible with existing (GUI-based) profiling tools

• Knownledge of C++ programming language
• Preferably experience with Python Scripting
• Experience with Embedded SW Development (e.g. ARM/RISC-V)

Machine Learning on the edge is becoming more and more popular nowadays. Especially for safety critical applications it is not acceptable to have any deviation of model outputs caused by the deployment method while other application might accept differences to the golden reference outputs to some degree.

Our TVM deployment flow is mainly based on the MLonMCU (https://github.com/tum-ei-eda/mlonmcu) tool. While it already supports validating model outputs, this functionality is rather limited.

Task Description:

• Goal: Reimplement the validation feature in MLonMCU to be more flexible and accurate
• Configuration: In additionon to bit-excact equivalences, the user may also allow deviations in some degree (absolute/relative delta)
• Flexibility: Provide an generic interface supporting several target-specific implementations (Offline: data compiled in ROM, Online: data sent via Semihosting, UART,...)
• Accuracy: Allow comparing intermediate values (between layers) in addition to just model outputs.

• Basic Knownledge of Machine Learning
• Experience with Embedded C programming
• Good Python Coding skills
• Ideally experience using TVM Machinne Learning Framework

The required steps can be described as follows:

• Get used to the MLonMCU Benchmarking flow
• Propose a concept for a remote execution of benchmarks
• Implement remote execution protocol
• Add more features to improve benchmarking throughput
• Add detailed documentation

• Experience with networking protocols
• Good Python programming
• Ideally experience working with UNIX-like operating systems

Microfluidic Large Scale Integration (mLSI) refers to the development of microfluidic chips with thousands of integrated micromechanical valves and control components. This technology is used in many areas of biology and chemistry and is a candidate for replacing today's conventional automation paradigm, which consists of Robots for handling liquids.

To enable automated control of mLSI, programming sequence for valve control and fluid inputs is essential. Although, not every researcher, especially biologists or chemists, has programming skills. We want to introduce an interactive but intuitive programming platform for microfluidic researchers that do not require any pre-experience with programming to overcome this problem.

This platform provides a visual programming language that allows users to define their needs by dragging and dropping the command blocks into a canvas. A flow simulation will show up on the side when users click on the run. 

• Knowledge and experience in web design and web development
• Good understanding of HTML5, CSS, JavaScript and PHP (or Java Spring Boot) or the wiliness to learn them
• Basic knowledge of SQL, jQuery and Vue.js (or other frontend frameworks) or the cunning to know them
• Solution-oriented thinking

Siehe beigefügten Aushang.

DXF is short for Drawing Exchange Format or Drawing Interchange Format and is a vector file type. Engineers, designers, and architects often use the DXF format for drawings during product design. DXF is widely used in industrial production. Many devices such as printers, laser cutters, CNC machining, etc. accept DXF format as input. In this project, we would like you to implement a generic DXF library that can be used by different software to produce designs in DXF format. 

• Knowledge and experience in web design and web development or desktop application development
• Good understanding of JavaScript and PHP or Java Spring Boot or the wiliness to learn them
• Basic knowledge of Vue.js (or other frontend frameworks) or object-oriented programming or the cunning to know them
• Solution-oriented thinking

Microfluidic Large Scale Integration (mLSI) refers to the development of microfluidic chips with thousands of integrated micromechanical valves and control components. This technology is used in many areas of biology and chemistry and is a candidate for replacing today's conventional automation paradigm, which consists of Robots for handling liquids.

To enable automated control of mLSI, programming sequence for valve control and fluid inputs is essential. Although, not every researcher, especially biologists or chemists, has programming skills. We want to introduce an interactive but intuitive programming platform for microfluidic researchers that do not require any pre-experience with programming to overcome this problem.

This platform provides a visual programming language that allows users to define their needs by dragging and dropping the command blocks into a canvas. A flow simulation will show up on the side when users click on the run. 

• Knowledge and experience in web design and web development
• Good understanding of HTML5, CSS, JavaScript and PHP (or Java Spring Boot) or the wiliness to learn them
• Basic knowledge of SQL, jQuery and Vue.js (or other frontend frameworks) or the cunning to know them
• Solution-oriented thinking

Siehe beigefügten Aushang.