Linkai Peng

Aiming to build Her.

I'm a PhD student in Language & Cognition at the University of Connecticut, working with James S. Magnuson. I approach human-level speech intelligence from the human side: I build computational models of how people perceive and understand speech as it unfolds, and use them to measure and improve how machines listen. Before my PhD, I was a senior machine learning engineer on the speech team at NetEase in China. Some human behaviors I keep coming back to: , , , and .

I'm looking for summer 2027 research internships in speech LLMs, informed by cognitive science.

Visual summaries of my studies:

Full-Duplex Speech Models Take the Floor When Asked, Not When Needed, arXiv 2026, under review

L. Peng, B. Nuchged, K. Fu, Y. Yao

Animation: the user mentions a sparking charger and the model stays silent, then the user asks where to eat and the model replies right away Table of speech-onset rates for seven full-duplex models across trigger types

Full-duplex speech models need to actively offer help to users, which matters for home assistants, tutoring, and virtual interviews. However, today's models almost never speak up when needed. They answer questions and talk after pauses, but a hazard or a false claim draws no more speech than an ordinary sentence.

arXiv · code

Listen, Diagnose, and Interrupt: Coupling Mispronunciation Diagnostics into a Full-Duplex Speech LLM, preprint 2026, under review

L. Peng, B. Nuchged, K. Fu, Y. Yao, J. Zhang

This full-duplex language-learning tutor feeds a WavLM detector's mispronunciation diagnosis into a frozen 11B full-duplex model (VoiceChat) through a zero-initialized residual channel. It keeps the detector's accuracy, with sentence-level F1 nearly unchanged, and works in full-duplex mode: the correction starts a median 516 ms after the mispronounced word, while the learner is still talking.

PDF

Beyond the Transcript: Measuring and Improving the Use of Acoustic Information in Audio Language Models, preprint 2026, under review

L. Peng, B. Nuchged, Y. Yao

Study overview: data, models, recording gain, probe readout and repair Accuracy of 19 open audio models and Gemini 3.1 Pro on every task, with the two repaired models and their gains over the base models

PDF

Do Machines Listen Like Humans? A Temporal Benchmark for Phonological Competition in End-to-End ASR, Interspeech 2026

L. Peng, C. Brodbeck, S. Luthra, K. Brown, J. Rueckl, M. Escabí, D. Gow, J. S. Magnuson

🧠 Causality shapes a model's cognitive dynamics and lays the groundwork for real-time spoken interaction.

paper · code

A Study on Fine-Tuning wav2vec2.0 Model for the Task of Mispronunciation Detection and Diagnosis, Interspeech 2021

L. Peng, K. Fu, B. Lin, D. Ke, J. Zhang

Fine-tuning wav2vec 2.0 reaches 60.44% F1 on L2-ARCTIC, and 55.52% with 3× less labelled data.

paper

All publications on Google Scholar

Experience:

Honors & Service: