# Two Heads Are Better Than One: Async Knowledge Injection for Speech AI with Tandem Architecture **So Kuroki** **Yotaro Kubo** **Takuya Akiba** **Yujin Tang** **Manato Yaguchi (intern)** **The University of Tokyo** **APRIL 2026** We’re excited to introduce KAME: Tandem Architecture for Enhancing Knowledge in Real-Time Speech-to-Speech Conversational AI. KAME means turtle in Japanese. - **Paper (arxiv):** [https://arxiv.org/abs/2510.02327](https://arxiv.org/abs/2510.02327) (Accepted at ICASSP 2026) - **Inference code:** [https://github.com/SakanaAI/kame](https://github.com/SakanaAI/kame) - **Finetune code:** [https://github.com/SakanaAI/kame_finetune](https://github.com/SakanaAI/kame_finetune) - **Model:** [https://huggingface.co/SakanaAI/kame](https://huggingface.co/SakanaAI/kame) ## Overview A conversation with AI can typically be realized in two ways: Speech-to-Speech (S2S) [1, 2, 3] systems and cascaded systems that connect Speech-to-Text (STT), LLM, Text-to-Speech (TTS) modules [4]. S2S systems are responsive but tend to be less knowledgeable, while cascaded systems can suffer from some lag. KAME connects an S2S model with a backend LLM to combine their strengths. The S2S model handles the fast response loop, while the backend LLM runs asynchronously on a slower cycle and injects “oracle” signals as they become available. More concretely, as the user’s speech grows, the backend LLM is repeatedly called on the expanding partial transcript to provide progressively refined response signals. KAME enables the S2S model to start responding immediately, while the LLM runs in parallel to continuously reason about the response. This shifts the paradigm from “think then speak” to “speak while thinking.” ## Comparison Between KAME and Moshi KAME is compared with Moshi [1] to demonstrate its responsiveness and knowledge. Moshi itself is a remarkable system: building a full-duplex S2S model with near human-level response latency is a long-standing challenge, requiring the model to listen, think, and speak simultaneously under tight temporal constraints, and Moshi is a model whose architecture makes this possible. Yet low latency creates an unavoidable trade-off. The faster a model must respond, the less time it has to think, which limits the knowledge it can deliver. KAME is designed to break this trade-off by letting the system “speak while thinking”: it allows a backend LLM to continue reasoning in parallel without sacrificing responsiveness, so the system can respond quickly and knowledgeably. In these examples, KAME uses the Moshi-based S2S front-end and `gpt-4.1` as the backend LLM. The question audio is generated by converting MT-Bench [5] text prompts into speech using TTS. Scores are obtained by transcribing each spoken answer and evaluating the transcription in text space with an LLM judge, following MT-Bench evaluation. ### Reasoning **Question** David has three sisters. Each of them has one brother. How many brothers does David have? **KAME Response** David has three sisters. Each of his brothers has that brother. So all three sisters have three brothers, and each sister has one brother. That brother is David himself. So David is the only brother. **Moshi Response** Since each sister has one brother and David has three sisters, he has three brothers. **STEM** **Question** In the field of quantum physics, what is superposition, and how does it relate to the phenomenon of quantum entanglement? **KAME Response** Superposition in quantum physics means that a particle, like an electron or a photon, can exist in more than one state at the same time. **Moshi Response** Superposition is a concept in quantum physics that mentions the act of quantum entanglement as a background process. **Humanities** **Question** What are some business etiquette norms when doing business in Japan? **KAME Response** When doing business in Japan, there are several important etiquette norms to keep in mind. First, bow is a common way to greet and show respect. A handshake may also be used, but bowing is more traditional. **Moshi Response** One common etiquette norm in Japan is considering the other person’s status when making offers or expressions of gratitude. ## Backend LLM Swapping KAME allows easy swapping between backend LLMs such as `gpt-4.1`, `claude-opus-4-1`, and `gemini-2.5-flash`. This makes it possible to choose the most suitable model for each use case. In our examples, `claude-opus-4-1` tends to score higher on reasoning tasks, while `gpt-4.1` tends to score higher on humanities tasks. ### Reasoning **Question** David has three sisters. Each of them has one brother. How many brothers does David have? **Claude Response** Let’s start by finding the total. David has a total of three sisters, and each of his sisters have one brother. **Gemini Response** Okay, David has zero brothers. He is the one brother to all three of his sisters. **Humanities** **Question** Describe five key principles in evaluating an argument in analytical writing. **KAME Response** Certainly, here are five main principles to keep in mind when evaluating an argument in analytic writing. **Claude Response** I need to determine when an analytic considered the American analytical writing, all Prissions write. **Gemini Response** There are five key principles to consider. First, the rarity of the argument is clear. ## Conclusion KAME combines the responsiveness of a speech-to-speech system with the stronger knowledge access of a backend LLM. Across the examples above, it maintains low latency while delivering more informed responses than a standalone S2S baseline. The tandem design also makes the backend flexible. By swapping the backend model based on the task, KAME can benefit from the strengths of different LLMs without changing the overall interaction pattern. ### References 1. **Moshi: a speech-text foundation model for realtime dialogue** Defossez, A., Mazare, L., Orsini, M., Royer, A., Perez, P., Jegou, H., Grave, E. and Zeghidour, N., 2024. KyutAI. 2. **LLaMA-Omni: seamless speech interaction with large language models** Fang, Q., Guo, S., Zhou, Y., Ma, Z., Zhang, S. and Feng, Y., 2025. International Conference on Learning Representations (ICLR). 3. **SpeechGPT: Empowering large language models with intrinsic cross-modal conversational abilities** Zhang, D., Li, S., Zhang, X., Zhan, J., Wang, P., Zhou, Y. and Qiu, X., 2023. Findings of the Association for Computational Linguistics: EMNLP, pp. 15757—15773. 4. **Unmute.sh by KyutAI** [link](https://unmute.sh/) KyutAI, ., 2025. KyutAI. 5. **Judging LLM-as-a-judge with MT-Bench and Chatbot Arena** Zheng, L., Chiang, W., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D. and Xing, E., 2023. Advances in Neural Information Processing Systems (NeurIPS), Vol 36, pp. 46595—46623.