Information Theory and Machine Learning Lab
We are a research group led by Prof. Jy-yong Sohn in the Department of Statistics & Data Science at Yonsei University. We study topics across machine learning and AI using mathematical tools from information theory, optimization, learning theory, and probability & statistics. Our current focus is the theoretical and algorithmic aspects of foundation models (FMs).
Internal representations of FMs
What do foundation models encode, and how is it organized inside their weights and activations?
Explore →Training & adaptation of FMs
Principled methods for training, fine-tuning, and continually adapting large models without forgetting.
Explore →Trustworthy & efficient FMs
Reliable evaluation, control, and compression — making foundation models safer and cheaper to run.
Explore →Selected preprints & publications
- ICML '26 Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models
- ICML '26 How to Correctly Report LLM-as-a-Judge Evaluations
- NeurIPS '25 Enhancing Compositional Reasoning in CLIP via Reconstruction and Alignment of Text Descriptions
- NeurIPS '25 Soft Task-Aware Routing of Experts for Equivariant Representation Learning
- ICML '25 On the Similarities of Embeddings in Contrastive Learning
- ICML '25 Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
- AISTATS '25 A Theoretical Framework for Preventing Class Collapse in Supervised Contrastive Learning
- UAI '24 Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks
- AISTATS '24 Analysis of Using Sigmoid Loss for Contrastive Learning
- ICML '23 Looped Transformers as Programmable Computers
Awards, grants & invited talks
Selected awards & grants
- IITP — Next-Generation AI Development 2025–2029
- NRF Korea — Basic Research Laboratory (기초연구실) 2024–2027
- NRF Korea — Outstanding Young Scientist (우수신진) 2024–2027
- Excellence in Teaching Award, Yonsei University 2023
Selected invited talks
- Reasoning Capabilities of Foundation Models — Krafton AI (Dec 2025) , KIAS (Jan 2026)
- Role of Mathematics in the Era of AI — SKIS (Mar 2025) , HYU (Feb 2025)
- Theory of Contrastive Learning — KSS (Jan 2025) , NIMS ICIM (Nov 2024) , Krafton AI (Oct 2024) , KIAS (Sep 2024) , KMS (Oct 2025)
- Theory of Foundation Models — KICS (Nov 2024) , KSIAM (Aug 2024)
- Theory of Efficient Machine Learning — KIAS (Dec 2023)
- Large Language Models (LLMs) — KICS (Feb 2024) , Linq (Jul 2023) , Yonsei Institute of Data Science (Apr 2023)
- Generative Models — KAIA (Jul 2023) , ETRI (Jun 2023)
We are recruiting
Looking for motivated undergraduate and graduate students, as well as postdocs, interested in our research topics. Send your CV and a short research statement to the PI.
Contact PI →