Skip to content
Aryan Deshmukh

App · Python

Valorant Voice AI

Text to speech in Valorant agent voices: dataset pipeline, XTTS-v2 fine-tune per agent, measured evaluation, typed API (Kaggle GPU)

Source on GitHubruns locallyCI passing

Why it exists

Speech ML is mostly data work and honest measurement. The goal is one project that shows the whole loop end to end on a problem people care about, with every claim backed by a result file that names the model and the data it came from.

Measured results

Valorant Voice AI: speaker similarity against word error rate for each fine-tuning run, next to the zero-shot baselineValorant Voice AI: speaker similarity against word error rate for each fine-tuning run, next to the zero-shot baseline

What it does

  1. 01

    Agent list is fixed. the 29 playable agents. There is no upload of arbitrary reference audio, so the service cannot be used to clone a real person.

  2. 02

    Dataset pipeline. downloads the public voice-line dataset of 26 agents on Kaggle, resamples to 22.05 kHz mono, trims silence, normalises loudness, drops clips outside 1.5 to 12 seconds, drops empty and repeated lines, and keeps only lines whose text Whisper hears in the clip. A per-agent audit table shows what was kept and dropped, and why.

  3. 03

    Leak-free split. a line's train, validation or test assignment is a hash of its text, so test lines are never trained on, a repeated line such as "Okay." never sits in both train and test, and nothing moves when more data is added.

  4. 04

    Every agent with data, by measurement. 26 agents are served by zero-shot cloning, because none of eight fine-tunes beat it. GET /v1/agents returns each agent's measured speaker similarity and word error rate, and the page lists only agents that have voice data and a measured result. Miks, Veto and Waylay stay unlisted: the public dataset has no lines for them. Table: docs/per-agent-results.md.

  5. 05

    API. typed JSON request, text length cap, per-client rate limit, every response labelled X-Generated-Audio: ai.

  6. 06

    Evaluation harness. speaker similarity (ECAPA-TDNN cosine), word error rate (Whisper hears the output), and real-time factor, written with the checkpoint and manifest hash.

Next

Node Connect