TinyML Study Group

@Gueltto9th

Group Members

• Jung Yeon Lee: @curieuxjy
• Seunghyun Oh: @ooshyun
• castleflag: @CastleFlag

MIT 6.5940 Fall 2023 TinyML and Efficient Deep Learning Computing

강의 사이트: https://hanlab.mit.edu/courses/2023-fall-65940

• 목표: 강의 영상 리뷰 및 Lab 실습 완료
• Github 정리 자료 Archiving (참고: GNN Study)

참고강의:

• ECE 5545: Machine Learning Hardware and Systems
  • https://abdelfattah-class.github.io/ece5545/sp23
• MIT 6.S965 Fall 2022 TinyML and Efficient Deep Learning Computing
  • https://hanlab.mit.edu/courses/2022-fall-6s965
  • 2023년도 강의에서 부족한 설명이나 내용 보충할 때 참고

21Lec + 4Lab

• Lec1/2와 Lab0은 제외
• 강의를 듣고 1명씩 돌아가면서 강의 복습 recap 발표
• 다른 사람들은 질문/디스커션 토픽 가져오기
• 주 1회 (약 16주 - 4개월 이내 완료 목표)

---

[Chapter I: Efficient Inference]

• Lecture 3: Pruning and Sparsity (Part I) @ooshyun
• Lecture 4: Pruning and Sparsity (Part II) @curieuxjy
• Lab 1 @CastleFlag
• Lecture 5: Quantization (Part I) @ooshyun
• Lecture 6: Quantization (Part II) @ooshyun
• Lab 2 @curieuxjy
• Lecture 7: Neural Architecture Search (Part I) @CastleFlag
• Lecture 8: Neural Architecture Search (Part II) @CastleFlag
• Lab 3 @ooshyun
• Lecture 9: Knowledge Distillation @ooshyun
• Lecture 10: MCUNet: TinyML on Microcontrollers @curieuxjy
• Lecture 11: TinyEngine and Parallel Processing @curieuxjy

[Chapter II: Domain-Specific Optimization]

• Lecture 12: Transformer and LLM (Part I) @CastleFlag
• Lecture 13: Transformer and LLM (Part II) @CastleFlag
• Lecture 14: Vision Transformer @ooshyun
• Lab 4 @ooshyun
• Lecture 15: GAN, Video, and Point Cloud @curieuxjy
• Lecture 16: Diffusion Model @curieuxjy
• Lecture 17: Distributed Training (Part I) @CastleFlag
• Lecture 18: Distributed Training (Part II) @CastleFlag
• Lab 5 @ooshyun
• Lecture 19: On-Device Training and Transfer Learning @ooshyun
• Lecture 20: Efficient Fine-tuning and Prompt Engineering @curieuxjy

[Chapter IV: Advanced Topics]

• Lecture 21: Basics of Quantum Computing @curieuxjy
• Lecture 22: Quantum Machine Learning @CastleFlag
• Lecture 23: Noise Robust Quantum ML @CastleFlag