Foundations of modern machine learning
We build mathematical frameworks that explain the behavior of foundation models — large language models, vision-language models, and the agents built on them — and use that understanding to design better algorithms.
Our research follows the lifecycle of a foundation model — pre-training → post-training → inference — and at every stage we ask the same two questions: what is really happening inside the model? (theory) and how can we do it better? (algorithms).
How foundation models learn representations
A foundation model begins by learning representations from massive multi-modal data. We build the theory of contrastive and multi-modal pre-training — the geometry embeddings converge to, why failures such as class collapse arise, and how concepts are encoded across modalities — and turn that understanding into pre-training methods that learn better representations.
- Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders arXiv · 2026
- Enhancing Compositional Reasoning in CLIP via Reconstruction and Alignment of Text Descriptions NeurIPS · 2025
- Soft Task-Aware Routing of Experts for Equivariant Representation Learning NeurIPS · 2025
- On the Similarities of Embeddings in Contrastive Learning ICML · 2025
- A Theoretical Framework for Preventing Class Collapse in Supervised Contrastive Learning AISTATS · 2025
- Mini-Batch Optimization of Contrastive Loss TMLR · 2024
- Analysis of Using Sigmoid Loss for Contrastive Learning AISTATS · 2024
- Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment Findings of EMNLP · 2022
How foundation models adapt
A pre-trained model is then adapted: fine-tuned to downstream tasks, continually updated, aligned via reinforcement — and sometimes made to unlearn. We characterize exactly what changes at this stage (in-context learning ability, prior knowledge, memorization capacity, representational structure) and design post-training methods with provable guarantees rather than purely empirical recipes.
- Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models ICML · 2026
- Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective ICML · 2025
- Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks UAI · 2024
- LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks NeurIPS · 2022
How foundation models reason & reach deployment
At inference time the model meets the world: it reasons, learns in context, and must be evaluated and served under real budgets. We study the computations transformers can carry out at inference (reasoning, in-context learning), put LLM evaluation on rigorous statistical footing, and compress models — asking what is essential and what is disposable — so capability survives deployment.
- How to Correctly Report LLM-as-a-Judge Evaluations ICML · 2026
- Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning arXiv · 2026
- Looped Transformers as Programmable Computers ICML · 2023
- Can We Find Strong Lottery Tickets in Generative Models? AAAI · 2023
- Rare Gems: Finding Lottery Tickets at Initialization NeurIPS · 2022
- Finding Everything within Random Binary Networks AISTATS · 2022
From theory to practice
Every stage of this pipeline is tested against real-world domains, alongside domain experts. Current applications include healthcare and psychiatry (cognitive behavioral therapy alignment, psychiatric interviews, cognitive-distortion classification), finance (question answering and retrieval-augmented generation), and law (Korean legal QA). These deployments create real impact while surfacing new theoretical questions that feed back into the three pillars.
- Aligning Large Language Models for Cognitive Behavioral Therapy: A Proof-of-Concept Study Frontiers in Psychiatry · 2025
- FinDER: Financial Dataset for Question Answering and Retrieval-Augmented Generation ICLR Workshop · 2025
- Aligning Large Language Models for Enhancing Psychiatric Interviews through Symptom Delineation and Summarization JMIR Formative Research · 2024
- ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification NAACL Workshop · 2024
- Retrieval-based Evaluation for LLMs: A Case Study in Korean Legal QA EMNLP Workshop · 2023
Earlier research directions
Trustworthy ML
Robust, fair, and private machine learning.
Distributed ML
Distributed and federated learning over networks.
- Buffer-based Gradient Projection for Continual Federated Learning TMLR · 2025
- Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks NeurIPS · 2020
- Attack of the Tails: Yes, You Really Can Backdoor Federated Learning NeurIPS · 2020
- Hierarchical Coding for Distributed Computing IEEE ISIT · 2018
Information / Communication Theory
Coding theory for storage, computing, and wireless systems.
- Capacity of Clustered Distributed Storage IEEE Trans. Information Theory · 2019
- Secure Clustered Distributed Storage Against Eavesdropping IEEE Trans. Information Theory · 2019
- On Reusing Pilots Across Interfering Cells in Massive MIMO IEEE Trans. Wireless Comm. · 2017
- Pilot Reuse Strategy Maximizing the Weighted-Sum-Rate in Massive MIMO Systems IEEE JSAC · 2017
Funding
- · IITP — Next-Generation AI development (2025–2029)
- · NRF — Basic Research Laboratory (2024–2027)
- · NRF — Outstanding Young Scientist Award (2024–2027)