Ph.D. Candidate · Seoul National University

Che Hyun Lee

Context-aware guidance
for faithful generation.

Email CV ↗ Google Scholar ↗ GitHub ↗ LinkedIn ↗

I am a Ph.D. candidate in Electrical and Computer Engineering at Seoul National University, advised by Prof. Sungroh Yoon.

My research focuses on spoken dialogue systems and diffusion language models (Diffusion LMs), with the goal of making generated outputs faithful to their context. I study how models use dialogue history, reference text, and language cues, and develop context-aware guidance to shape their influence on generation.

I am particularly interested in which context to emphasize, where to apply guidance, and how strongly to apply it. This connects context-faithful spoken responses, controllable text editing, and language-specific accent guidance in code-switched speech.

Previously, I received my M.S. from the Graduate School of Data Science and my B.S. in Electrical and Computer Engineering at Seoul National University.

Explore my research

Research & Publications

Guiding how context shapes generation.

Spoken dialogue systems and Diffusion LMs, connected through context-aware guidance.

Illustration of a person having a multi-turn spoken conversation with a voice assistant

Spoken Dialogue Systems

Models that understand and respond in speech for natural, multi-turn conversations. Research challenges include remembering dialogue history and using it to generate coherent responses.

Illustration of text generation through iterative denoising of masked tokens

Diffusion Language Model

An alternative to autoregressive generation using continuous or discrete diffusion. Topics include parallel decoding, iterative refinement, and controllable text generation and editing.

Illustration of dialogue, reference text, and language context guiding generation

Context Guidance

Methods that steer text and speech generation using dialogue history, reference text, or language cues, helping outputs remain faithful to the relevant information and constraints.

Publications

11 papers

* Equal contribution. † Corresponding author.

Phrase-Localized Language-Contrastive Guidance: Training-Free Localized Accent Control for Code-Switching Text-to-Speech

Che Hyun Lee, Sangkwon Park, Donghun Kang, Dongwook Lee, Youngho Cho, Heeseung Kim^†, Sungroh Yoon^†

EMNLP 2026OralConference on Empirical Methods in Natural Language Processing (Main)project arXiv code dataset

Training-free control of accent at the phrase level in code-switching text-to-speech.

From Awareness to Adherence: Bridging the Context Gap in Spoken Dialogue Systems via Context-Aware Decoding

Che Hyun Lee, Heeseung Kim^†, Sungroh Yoon^†

INTERSPEECH 2026Conference of the International Speech Communication AssociationarXiv code

Strengthens the influence of relevant dialogue history through context-aware decoding.

HEAR Who Said What: Unlocking Speaker-Attributed Reasoning via Counterfactual Voice Grounding

Dongwook Lee, Sangkwon Park, Eunwoo Song, Che Hyun Lee, Youngho Cho, Junho Kim, June Young Yi, Heeseung Kim^†, Sungroh Yoon^†

EMNLP 2026Conference on Empirical Methods in Natural Language Processing (Main)arXiv project code dataset

Teaches audio LLMs to reason about who said what by grounding answers in counterfactual voices.

Still Between Us? Evaluating and Improving Voice Assistant Robustness to Third-Party Interruptions

Dongwook Lee, Eunwoo Song, Che Hyun Lee, Heeseung Kim^†, Sungroh Yoon^†

ACL 2026Annual Meeting of the Association for Computational LinguisticsarXiv project code dataset

Trains and evaluates voice assistants to handle interruptions from third-party speakers.

Beyond Language-Specific Neurons: The Challenge of Identifying Speech-Specific Neurons in Multimodal LLMs

Nohil Park, Che Hyun Lee, Jiheum Yeom, Heeseung Kim, Sungroh Yoon^†

IEEE JSTSP 2026IEEE Journal of Selected Topics in Signal Processing, vol. 20, no. 1, pp. 90-98, January 2026paper

Shows why speech-specific neurons are hard to isolate in multimodal LLMs.

EdiText: Controllable Coarse-to-Fine Text Editing with Diffusion Language Models

Che Hyun Lee, Heeseung Kim, Jiheum Yeom, Sungroh Yoon^†

ACL 2025Annual Meeting of the Association for Computational LinguisticsarXiv

Controls the degree of text editing in a Diffusion LM, from coarse changes to fine adjustments.

Does Your Voice Assistant Remember? Analyzing Conversational Context Recall and Utilization in Voice Interaction Models

Heeseung Kim*, Che Hyun Lee*, Sangkwon Park, Jiheum Yeom, Nohil Park, Sangwon Yu, Sungroh Yoon^†

ACL Findings 2025Findings of the Association for Computational LinguisticsarXiv demo dataset

Benchmarks how voice interaction models recall and use information from earlier dialogue turns.

NanoVoice: Efficient Speaker-Adaptive Text-to-Speech for Multiple Speakers

Nohil Park, Heeseung Kim, Che Hyun Lee, Jooyoung Choi, Jiheum Yeom, Sungroh Yoon^†

ICASSP 2025OralIEEE International Conference on Acoustics, Speech, and Signal Processingproject arXiv

Efficiently personalizes voice adapters for multiple speakers simultaneously.

VoiceGuider: Enhancing Out-of-Domain Performance in Parameter-Efficient Speaker-Adaptive Text-to-Speech via Autoguidance

Jiheum Yeom, Heeseung Kim, Jooyoung Choi, Che Hyun Lee, Nohil Park, Sungroh Yoon^†

ICASSP 2025IEEE International Conference on Acoustics, Speech, and Signal Processingproject arXiv

An inference-time guidance method that keeps speaker adaptation robust for unseen voices.

VoiceTailor: Lightweight Plug-In Adapter for Diffusion-Based Personalized Text-to-Speech

Heeseung Kim, Sang-gil Lee, Jiheum Yeom, Che Hyun Lee, Sungwon Kim, Sungroh Yoon^†

INTERSPEECH 2024Annual Conference of the International Speech Communication Associationproject arXiv

One-shot personalized text-to-speech using lightweight low-rank adapters.

Edit-A-Video: Single Video Editing with Object-Aware Consistency

Chaehun Shin*, Heeseung Kim*, Che Hyun Lee, Sang-gil Lee, Sungroh Yoon^†

ACML 2023Oral · Best Paper AwardAsian Conference on Machine Learningproject arXiv

One-shot video editing that keeps backgrounds consistent via masked blending.

Background

Education & Research Experience

Education

Seoul National University

Ph.D. Candidate in Electrical and Computer Engineering

Spoken dialogue systems, Diffusion LMs, and context-aware guidance

Mar 2022 – Present

Seoul National University

M.S., Graduate School of Data Science

Machine learning, natural language processing, and data science

Mar 2020 – Feb 2022

Seoul National University

B.S. in Electrical and Computer Engineering

Machine learning and deep learning

Mar 2014 – Feb 2020

Research Experience

SNU–Samsung Electronics System LSI Collaboration

Streaming Personalized Text-to-Speech

Trained TTS models with causal modules and developed chunk-level caching for low-latency streaming synthesis. Worked on efficient multi-speaker personalization, with related research on localized accent guidance for code-switching TTS.

Dec 2023 – Present

SNU–Samsung Electronics DS Collaboration

Parameter-Efficient Fine-Tuning in Multimodal AI Models

Inference-time guidance for robust out-of-domain TTS personalization; co-authored VoiceGuider.

Sep 2023 – Sep 2024

Undergraduate Research Student · SNU

Electrical and Computer Engineering

Implemented a CNN-based prediction model for CRISPR-Cpf1 protein sequence data, supervised by Prof. Sungroh Yoon.

Mar 2019 – Aug 2020

Awards & Teaching

Honors, Awards & Teaching

Honors & Awards

Best Paper Award

Asian Conference on Machine Learning (ACML)

Edit-A-Video: Single Video Editing with Object-Aware Consistency

2023

Cum Laude

Seoul National University

B.S. in Electrical and Computer Engineering

2020

Academic Performance Scholarship

Seoul National University

Spring and Fall 2015; Spring 2019

2015, 2019

Teaching Experience

Samsung AI Expert Program

Teaching Assistant · Samsung Electronics

Supported a three-day lecture series on speech processing, speech recognition, and speech synthesis.

Apr – May 2026

Topics in Computers and VLSI

Teaching Assistant · Seoul National University

Course 430.659(005) · Graduate course on industry AI development and commercialization, offered in collaboration with SK Telecom.

2022, 2024, 2025