Phrase-Localized Language-Contrastive Guidance:
Training-Free Localized Accent Control for Code-Switching Text-to-Speech

Che Hyun Lee1, Sangkwon Park1, Donghun Kang1, Dongwook Lee2,
Youngho Cho2, Heeseung Kim3, Sungroh Yoon1,2,4
1Department of Electrical and Computer Engineering, Seoul National University
2Interdisciplinary Program in Artificial Intelligence, Seoul National University
3Department of Artificial Intelligence, University of Seoul
4AIIS, ASRI, INMC, and ISRC, Seoul National University

EMNLP 2026 · Main Conference

📄 arXiv ♪ Listen to Demos ⌨ Code ▤ Benchmark

Code, the 1,200-utterance synthetic code-switching benchmark, and reproduction instructions are available at github.com/saga1214/PhraseLocalizedLCG.

⏱ Hear the difference in 10 seconds — cafe pickup call (EN→KO)

In a Korean cafe, you will hear 주문하신 음료 나왔습니다 when your drink is ready.
Vanillathe phrase comes out as English-accented "Jumunhashin..."
Ours — LCGthe same phrase, spoken like a native Korean barista

What is LCG?

  1. The problem. When a sentence embeds a foreign phrase ("This station is 홍대입구"), zero-shot TTS speaks the phrase with the carrier language's accent — because classifier-free guidance applies one language condition to the whole utterance.
  2. The idea. Add a third guidance branch conditioned on the phrase's own language, and apply it only inside the phrase region.
  3. The trick. The phrase region is located by the model itself: we read its self-attention to find which audio frames align to the embedded text — no fine-tuning, no external aligner.

Overview

The challenge

Global guidance flattens the embedded phrase into the carrier accent (left); LCG steers only the phrase region toward its own language (right).

Cross-lingual accent leakage and localized control

The method in three steps

  1. Probe — read decoder self-attention (layers 8 & 12) to find the frames aligned to the embedded phrase.
  2. Refine — union the masks from both language tags, then dilate for recall; the diffusion process absorbs the slack.
  3. Guide — add a phrase-language guidance term, gated by the mask, with its own strength λ.
Method overview: attention probing, mask refinement, phrase-localized LCG

Real-World Scenarios

Everyday situations where a carrier sentence embeds a foreign phrase. Vanilla vs. our phrase-localized LCG (λ=7), paper-final configuration.

The last tab embeds French and German phrases in English carriers — the training-free control covers all five languages of the paper.

Cafe pickup call EN→KO

In a Korean cafe, you will hear 주문하신 음료 나왔습니다 when your drink is ready.
» 주문하신 음료 나왔습니다 — your drink is ready
Vanilla
Ours λ7

Restaurant service phrase EN→JA

Your server will say ごゆっくりお召し上がりください, so please take your time.
» ごゆっくりお召し上がりください — please take your time and enjoy your meal
Vanilla
Ours λ7

Restaurant menu EN→JA

Tonight's special is えびと野菜の天ぷら盛り合わせ, served with matcha salt.
» えびと野菜の天ぷら盛り合わせ — assorted shrimp and vegetable tempura
Vanilla
Ours λ7

After a meal EN→JA

At the end of a meal, you can say ごちそうさまでした.
» ごちそうさまでした — thank you for the meal
Vanilla
Ours λ7

Shop greeting EN→KO

As you walk in, the staff will greet you with 어서 오세요, 찾으시는 거 있으세요?
» 어서 오세요, 찾으시는 거 있으세요? — welcome — are you looking for anything?
Vanilla
Ours λ7

Benchmark Samples

Samples drawn directly from our released 1,200-utterance synthetic code-switching benchmark, comparing all three systems from the paper (unguided baseline, coupled Swap, and phrase-localized LCG).

Isolated phrase segments

Korean carrier sentences with embedded English phrases, drawn from the LA=1 subset: all systems synthesize the correct words, so the difference you hear is residual accent quality on the isolated segment.

Example 1 — “practical workshops aligned with our buying committee” KO→EN

지난달 우리 팀이 도입한 기업 간 영업 컨설팅사는 기대보다 훨씬 실무적이었고, 리뷰를 남길 만큼 인상이 좋았다. 첫 진단 미팅에서 a clear account-based sales playbook 으로 논의를 정리해 주니, 영업대표와 마케팅 담당자가 같은 언어로 문제를 보게 됐다. 워크숍은 practical workshops aligned with our buying committee 와 실제 고객 미팅 사례를 엮어 진행돼, 강의보다 공동 설계에 가까웠다. 특히 actionable pipeline diagnostics grounded in real CRM data 가 가장 도움이 됐는데, 막연했던 리드 품질 문제와 딜 지연 원인이 숫자로 보였다. 마지막 제안서에 담긴 a lightweight operating model for cross-functional execution 과 transparent ROI assumptions for the first six months 까지 설득력 있어, 비용은 높지만 재계약은 긍정적으로 검토 중이다.
Vanilla
Swap λ3
Ours λ7

Example 2 — “lean household operations with zero emotional overhead” KO→EN

퇴근하고 집에 오자마자 운동복을 세탁기에 넣었는데, 오늘은 minimum viable weekday morning routine 으로 내일 입을 양말이랑 머그컵까지 미리 챙겼다. 싱크대에 컵 두 개랑 접시 하나만 남겨두기 싫어서 lean household operations with zero emotional overhead 처럼 바로 치웠고, 냉장고 앞 메모에는 data driven grocery budget tracking system 의 잔액을 작게 적어놨다. 베란다 화분에 물 주는 사이 고양이는 새 택배 박스에 들어가서 weekly user research session with a very picky stakeholder 에 참여하는 표정으로 나를 봤다. 별거 아닌 집안일도 이렇게 이름을 붙이면 묘하게 게임 같고, 잠들기 전 마음이 조금 정리된다.
Vanilla
Swap λ3
Ours λ7

Full sentences

Complete utterances with three embedded phrases each, comparing global naturalness and phrase nativeness.

English carrier · Japanese phrases EN→JA

After the rain, I ducked into a tiny old-school cafe by the station, and the 深煎り豆をネルで落とす香り made my whole jacket smell like comfort. The owner gave me a smile that said ごゆっくりお召し上がりください, not fussy, just warm. I took the corner seat and watched steam blur the window, feeling a sort of 雨上がりの縁側で飲むほうじ茶 settle into the evening. Next time, I’m bringing a book and pretending I don’t have notifications. Honestly, it beat any fancy latte art.
» 深煎り豆をネルで落とす香り — the aroma of dark-roasted beans brewed through a flannel filter
» ごゆっくりお召し上がりください — please take your time and enjoy it
» 雨上がりの縁側で飲むほうじ茶 — roasted green tea sipped on a veranda after the rain
Vanilla
Swap λ3
Ours λ7

English carrier · Korean phrases EN→KO

Han’s slim novel reads like a review of everything we keep misplacing: names, rooms, the voices of the dead. The 첫눈 녹듯 사라지는 어머니의 마지막 손길 gives the opening its ache, while the 제사상에 올린 식어 가는 흰밥 한 그릇 turns grief into something almost domestic. By the final chapter, even 봄비에 젖은 벚꽃잎이 흩어진 골목 feels less symbolic than inevitable. I admired its restraint; it left me quiet, not devastated, and somehow that was worse.
» 첫눈 녹듯 사라지는 어머니의 마지막 손길 — last trace of a mother’s touch fading like first snow
» 제사상에 올린 식어 가는 흰밥 한 그릇 — cooling bowl of white rice set out for the ancestors
» 봄비에 젖은 벚꽃잎이 흩어진 골목 — an alley strewn with cherry petals wet from spring rain
Vanilla
Swap λ3
Ours λ7

Evaluation Corpus

All audio above is synthesized from our 1,200-utterance balanced synthetic code-switching benchmark, generated with a large language model and spanning five languages across twelve directional configurations. The corpus is released with the code and documented in benchmark/.

1,200
utterances
12
directions
5
languages (EN/FR/DE/JA/KO)
~3.6
avg phrases/utt
18
topical domains
6
stylistic registers

Each utterance contains 3–5 dense, technical or literary embedded phrases (~6 words, ~35 characters per phrase). The corpus is balanced by direction (100 utterances per direction) to enable per-direction evaluation.

Citation

@misc{lee2026phraselocalizedlanguagecontrastiveguidancetrainingfree,
  title         = {Phrase-Localized Language-Contrastive Guidance: Training-Free Localized Accent Control for Code-Switching Text-to-Speech},
  author        = {Che Hyun Lee and Sangkwon Park and Donghun Kang and Dongwook Lee and Youngho Cho and Heeseung Kim and Sungroh Yoon},
  year          = {2026},
  eprint        = {2609.01016},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2609.01016},
}