CharaTalk.
The joy of having a character in the conversation.
Text from an LLM becomes speech, which drives a VRM character's expressions and mouth movements. An iOS project that takes AI conversation beyond typing.
What problem does it solve?
Can we create the sense of presence and closeness with a character that chat alone can't convey? I'm working on connecting speech synthesis, 3D rendering, and a conversation model on the iPhone.
What I chose, and how I built it.
Generate the voice on-device
On-device speech synthesis using Irodori TTS and speech.cpp (Metal).
Link voice and expression
Analyze the volume of playback audio and map it to the VRM model's mouth movements.
Choose your model
Designed to work with local LLMs or external compatible APIs.
Keep 3D rendering independent
Avatars are rendered with RealityKit / VRMKit, with support for loading VRM files.
The big picture.
For technical details, the README and source code in the public repository are the primary source.
Read the README and code on GitHub ↗Status and caveats.
The README on GitHub explicitly asks users to mind the rights and commercial-use terms of third-party models, avatars, and assets. This page does not guarantee on-device performance or indicate App Store availability.
Want to try it?
This page is an introduction. Installation steps, requirements, license, and the latest status are in the public repository.
Check out CharaTalk on GitHub ↗