Information Theory and Machine Learning Lab
We are a research group led by Prof. Jy-yong Sohn in the Department of Statistics & Data Science at Yonsei University. We study topics across machine learning and AI using mathematical tools from information theory, optimization, learning theory, and probability & statistics. Our current focus is foundation models (FMs) — studied along their lifecycle, from pre-training through post-training to inference: at each stage we explain what the model is doing (theory) and use that understanding to build better algorithms.
How FMs learn representations
The theory of contrastive & multi-modal pre-training — what embeddings converge to and why — turned into methods that learn better representations.
Explore →How FMs adapt
What is gained and lost when models are fine-tuned, continually updated, aligned, or made to forget — and adaptation methods with provable guarantees.
Explore →How FMs reason & reach deployment
The computations models can carry out at inference, statistically rigorous evaluation, and compression that preserves what matters.
Explore →Selected preprints & publications
- ICML '26 Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models
- ICML '26 How to Correctly Report LLM-as-a-Judge Evaluations
- NeurIPS '25 Enhancing Compositional Reasoning in CLIP via Reconstruction and Alignment of Text Descriptions
- NeurIPS '25 Soft Task-Aware Routing of Experts for Equivariant Representation Learning
- ICML '25 On the Similarities of Embeddings in Contrastive Learning
- ICML '25 Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
- AISTATS '25 A Theoretical Framework for Preventing Class Collapse in Supervised Contrastive Learning
- UAI '24 Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks
- AISTATS '24 Analysis of Using Sigmoid Loss for Contrastive Learning
- ICML '23 Looped Transformers as Programmable Computers
Awards, grants & invited talks
Selected awards & grants
- IITP — Next-Generation AI Development 2025–2029
- NRF Korea — Basic Research Laboratory (기초연구실) 2024–2027
- NRF Korea — Outstanding Young Scientist (우수신진) 2024–2027
- Excellence in Teaching Award, Yonsei University 2023
Selected invited talks
- Reasoning Capabilities of Foundation Models — Krafton AI (Dec 2025) , KIAS (Jan 2026)
- Role of Mathematics in the Era of AI — SKIS (Mar 2025) , HYU (Feb 2025)
- Theory of Contrastive Learning — KSS (Jan 2025) , NIMS ICIM (Nov 2024) , Krafton AI (Oct 2024) , KIAS (Sep 2024) , KMS (Oct 2025)
- Theory of Foundation Models — KICS (Nov 2024) , KSIAM (Aug 2024)
- Theory of Efficient Machine Learning — KIAS (Dec 2023)
- Large Language Models (LLMs) — KICS (Feb 2024) , Linq (Jul 2023) , Yonsei Institute of Data Science (Apr 2023)
- Generative Models — KAIA (Jul 2023) , ETRI (Jun 2023)
We are recruiting
Looking for motivated undergraduate and graduate students, as well as postdocs, interested in our research topics. Send your CV and a short research statement to the PI.
Contact PI →