Convert Figma logo to code with AI

OpenMOSS logoMOSS-TTS

MOSS‑TTS Family is an open‑source speech and sound generation model family from MOSI.AI and the OpenMOSS team. It is designed for high‑fidelity, high‑expressiveness, and complex real‑world scenarios, covering stable long‑form speech, multi‑speaker dialogue, voice/character design, environmental sound effects, and real‑time streaming TTS.

3,866
344
3,866
13
View on GitHub

Quick Overview

Error generating quick overview

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

MOSS-TTS Family


    

OpenMOSS%2FMOSS-TTS | Trendshift
OpenClaw

WeChat Lark

English | 简体中文

MOSS‑TTS Family is an open‑source speech and sound generation model family from MOSI.AI and the OpenMOSS team. It is designed for high‑fidelity, high‑expressiveness, and complex real‑world scenarios, covering stable long‑form speech, multi‑speaker dialogue, voice/character design, environmental sound effects, and real‑time streaming TTS.

News

  • 2026.6.18: 🚀 MOSS-TTS-Local-Transformer-v1.5 receives Day-0 support in SGLang-Omni — the first inference backend to support the MossTTSLocal architecture, with an OpenAI-compatible /v1/audio/speech endpoint, streaming, and voice cloning. See the cookbooks: moss_tts_local, moss_tts.
  • 2026.6.18: 🚀 Released MOSS-TTS-Local-Transformer-v1.5, a 4B MossTTSLocal checkpoint that inherits all v1.5 features (language tags, stable cloning, explicit pause control, etc.), scales the backbone from Qwen3-1.7B to Qwen3-4B, and uses MOSS-Audio-Tokenizer-v2 for native 48 kHz stereo output.
  • 2026.6.7: 🚀 Released MOSS-Audio-Tokenizer-v2, natively supporting 48 kHz stereo input and output. Check out the MOSS-Audio-Tokenizer repository for more details!
  • 2026.6.2: 🚀 vLLM-Omni now supports the full MOSS-TTS series (MossTTSDelay, MossTTSRealtime, and MossTTSNano architectures), including MOSS-TTS-v1.5, MOSS-TTS, MOSS-TTSD, MOSS-SoundEffect, MOSS-VoiceGenerator, MOSS-TTS-Realtime, and MOSS-TTS-Nano. See the recipe and examples.
  • 2026.5.26: 🚀 Released MOSS-SoundEffect-v2.0, a new text-to-audio model using a DiT backbone with the Flow Matching objective, generating 48 kHz bilingual sound effects up to 30 seconds — see moss_soundeffect_v2/.
  • 2026.5.26: 🚀 Released MOSS-TTS-v1.5, with stronger multilingual synthesis when language tags are provided, more stable voice cloning, better long-reference short-text cloning, punctuation-following prosody, and explicit pause control via [pause X.Ys].
  • 2026.5.6: 🚀 MOSS-TTS and MOSS-Audio-Tokenizer now support mlx-audio. Visit the mlx-audio GitHub repository for details.
  • 2026.4.29: 📝 MOSS-TTS 2.0 is coming soon! We are collecting TTS feedback, suggestions, and feature requests via the requirements collection form.
  • 2026.4.13: 🚀 MOSS-TTS-Nano, our ~100M-parameter model, is now available! It supports multilingual voice cloning, 48 kHz stereo input/output, and streaming output on just 4 CPU cores. Check the GitHub repository and our blog for more details.
  • 2026.3.31: 📄 Our technical reports for MOSS-TTSD and MOSS-VoiceGenerator are now available on arXiv!
  • 2026.3.26: 📘 Added a tutorial on fine-tuning the MOSS-TTS-Realtime!
  • 2026.3.20: 📄 Our technical report is now available on arXiv!
  • 2026.3.18: 🚀 Added a first-class MOSS-TTS llama.cpp implementation in the companion repository OpenMOSS/llama.cpp, including end-to-end docs and a runnable pipeline for GGUF backbone inference plus ONNX audio codec decoding. See the first-class e2e guide.
  • 2026.3.16: 📘 Added a tutorial on fine-tuning the MossTTSLocal architecture, suitable for MOSS-TTS-Local-Transformer!
  • 2026.3.12: 🚀 Added SGLang backend support for the MossTTSDelay architecture, enabling efficient inference for MOSS-TTS (Delay) and MOSS-SoundEffect, with around 3× faster generation throughput!
  • 2026.3.11: 📘 Added a tutorial on fine-tuning the MossTTSDelay architecture, suitable for MOSS-TTS(Delay), MOSS-TTSD, MOSS-VoiceGenerator, and MOSS-SoundEffect!
  • 2026.3.10: ⚡️ Significantly optimized the VRAM usage of llama.cpp inference pipeline. Now 8B model fits onto 8GB GPUs!
  • 2026.3.4: 🚀 Added PyTorch-free inference support — enabling lightweight on-device deployment via llama.cpp + ONNX Runtime. Quantized GGUF weights are released at OpenMOSS-Team/MOSS-TTS-GGUF, and the ONNX audio tokenizer is available at OpenMOSS-Team/MOSS-Audio-Tokenizer-ONNX. See the llama.cpp backend for details.
  • 2026.3.4: 🎉 We add MOSS-TTS skills in ClawHub of 🦞 OpenClaw: feishu-voice-tts and moss-tts-voice.
  • 2026.2.10: 🎉🎉🎉 We have released MOSS-TTS Family. Check our Blog for more details! Our Huggingface Space is here: MOSS-TTS, MOSS-TTSD-v1.0, MOSS-VoiceGenerator.

Demo

Contents

Introduction

When a single piece of audio needs to sound like a real person, pronounce every word accurately, switch speaking styles across content, remain stable over tens of minutes, and support dialogue, role‑play, and real‑time interaction, a single TTS model is often not enough. The MOSS‑TTS Family breaks the workflow into five production‑ready models that can be used independently or composed into a complete pipeline.

  • MOSS‑TTS: The flagship production model featuring high fidelity and optimal zero-shot voice cloning. It supports long-speech generation, fine-grained control over Pinyin, phonemes, and duration, as well as multilingual/code-switched synthesis.
  • MOSS‑TTSD: A spoken dialogue generation model for expressive, multi-speaker, and ultra-long dialogues. The new v1.0 version achieves industry-leading performance on objective metrics and outperformed top closed-source models like Doubao and Gemini 2.5-pro in subjective evaluations. You can visit the MOSS-TTSD repository for details.
  • MOSS‑VoiceGenerator: An open-source voice design model capable of generating diverse voices and styles directly from text prompts, without any reference speech. It unifies voice design, style control, and synthesis, functioning independently or as a design layer for downstream TTS. Its performance surpasses other top-tier voice design models in arena ratings.
  • MOSS‑TTS‑Realtime: A multi-turn context-aware model for real-time voice agents. It uses incremental synthesis to ensure natural and coherent replies, making it ideal for building low-latency voice agents when paired with text models. The TTFB (Time To First Byte) of MOSS-TTS-Realtime reaches 180 ms, and the $T_{\text{LLM-first-sentence}} + T_{\text{MOSS-TTS-Realtime-TTFB}}$ is 377 ms.
  • MOSS‑SoundEffect: A content creation model specialized in sound effect generation with wide category coverage and controllable duration. It generates audio for natural environments, urban scenes, biological sounds, human actions, and musical fragments, suitable for film, games, and interactive experiences.

Model Architecture

We train MossTTSDelay and MossTTSLocal as complementary baselines under one training/evaluation setup: Delay emphasizes long-context stability, inference speed, and production readiness, while Local emphasizes lightweight flexibility and strong objective performance for streaming-oriented systems. Together they provide reproducible references for deployment and research.

MossTTSRealtime is not a third comparison baseline; it is a capability-driven design for voice agents. By modeling multi-turn context from both prior text and user acoustics, it delivers low-latency streaming speech that stays coherent and voice-consistent across turns.

ArchitectureCore MechanismArch Details
MossTTSDelayMulti‑head parallel RVQ prediction with delay‑pattern schedulingArch Details
MossTTSLocalTime‑synchronous RVQ blocks with a depth transformerArch Details
MossTTSRealtimeHierarchical text–audio inputs for realtime synthesisArch Details

Released Models

ModelArchitectureSizeModel CardHugging FaceModelScope
MOSS-TTS-v1.5MossTTSDelay8BModel CardHugging FaceModelScope
MOSS-TTS 1.0MossTTSDelay8BModel CardHugging FaceModelScope
MOSS-TTS-Local-Transformer-v1.5MossTTSLocal4BModel CardHugging FaceModelScope
MOSS-TTS-Local-TransformerMossTTSLocal1.7BModel CardHugging FaceModelScope
MOSS‑TTSD‑V1.0MossTTSDelay8BModel CardHugging FaceModelScope
MOSS‑VoiceGeneratorMossTTSDelay1.7BModel CardHugging FaceModelScope
MOSS‑SoundEffectMossTTSDelay8BModel CardHugging FaceModelScope
MOSS‑SoundEffect‑v2.0MossSoundEffectPipeline1.3B DiTModel CardHugging FaceModelScope
MOSS‑TTS‑RealtimeMossTTSRealtime1.7BModel CardHugging FaceModelScope

Supported Languages

MOSS-TTS-v1.5 and MOSS-TTS-Local-Transformer-v1.5 currently support 31 languages. They keep the 20 languages supported by MOSS-TTS 1.0 and extend multilingual continued training to Cantonese, Dutch, Finnish, Hindi, Macedonian, Malay, Romanian, Swahili, Tagalog, Thai, and Vietnamese.

MOSS-TTSD and MOSS-TTS-Realtime follow their own model cards for supported language coverage.

LanguageCodeFlagLanguageCodeFlagLanguageCodeFlag
Chinesezh🇨🇳Cantoneseyue🇭🇰Englishen🇺🇸
Arabicar🇸🇦Czechcs🇨🇿Danishda🇩🇰
Dutchnl🇳🇱Finnishfi🇫🇮Frenchfr🇫🇷
Germande🇩🇪Greekel🇬🇷Hebrewhe🇮🇱
Hindihi🇮🇳Hungarianhu🇭🇺Italianit🇮🇹
Japaneseja🇯🇵Koreanko🇰🇷Macedonianmk🇲🇰
Malayms🇲🇾Persian (Farsi)fa🇮🇷Polishpl🇵🇱
Portuguesept🇵🇹Romanianro🇷🇴Russianru🇷🇺
Spanishes🇪🇸Swahilisw🇹🇿Swedishsv🇸🇪
Tagalogtl🇵🇭Thaith🇹🇭Turkishtr🇹🇷
Vietnamesevi🇻🇳

MOSS-TTS-v1.5

MOSS-TTS-v1.5 is continued from MOSS-TTS 1.0. It preserves the main 1.0 capabilities, including zero-shot voice cloning, long-form speech generation, token-level duration control, Pinyin/IPA pronunciation control, multilingual synthesis, and code-switching.

Compared with MOSS-TTS 1.0, v1.5 focuses on these improvements:

  • Stronger multilingual synthesis with language tags: when the language is known, set it in processor.build_user_message(text=text, language="French") or the equivalent API field.
  • More stable voice cloning: v1.5 improves speaker similarity and reduces cloning variance across repeated generations.
  • Better long-reference, short-text cloning: v1.5 handles reference audio that is much longer than the target text more reliably.
  • More stable punctuation-following prosody: v1.5 follows punctuation-driven pauses more closely, especially in long sentences.
  • Explicit pause control: use inline pause markers such as [pause 3.2s], for example 我今天学习了一首中国的古诗，它的名字是[pause 3.2s]静夜思！.

MOSS-TTS-Local-Transformer-v1.5

MOSS-TTS-Local-Transformer-v1.5 is the 48 kHz stereo local-transformer release. It uses MOSS-Audio-Tokenizer-v2 as the audio tokenizer and adds realtime streaming decode examples in moss_tts_local_v1.5/.

Compared with MOSS-TTS-Local-Transformer-v1.0, v1.5 focuses on these improvements:

  • Higher-fidelity stereo audio modeling: v1.5 uses MOSS-Audio-Tokenizer-v2 as the audio tokenizer, supporting native 48 kHz stereo input and output for richer spatial detail and more natural perceived audio quality.
  • Stronger multilingual synthesis with language tags: when the language is known, set it in processor.build_user_message(text=text, language="French") or the equivalent API field.
  • More stable voice cloning: v1.5 improves speaker similarity and reduces cloning variance across repeated generations.
  • Better long-reference, short-text cloning: v1.5 handles reference audio that is much longer than the target text more reliably.
  • More stable punctuation-following prosody: v1.5 follows punctuation-driven pauses more closely, especially in long sentences.
  • Explicit pause control: use inline pause markers such as [pause 3.2s], for example 我今天学习了一首中国的古诗，它的名字是[pause 3.2s]静夜思！.

Quickstart

OpenClaw API Skills

We add MOSS-TTS skills in ClawHub of 🦞 OpenClaw. You can get your API key from MOSI AI Studio.

SkillDescriptionInstall
feishu-voice-ttsSend voice messages in Feishuclawhub install feishu-voice-tts
moss-tts-voiceCall MOSS-TTS API to generate speechclawhub install moss-tts-voice

Environment Setup

We recommend a clean, isolated Python environment with Transformers 5.0.0 to avoid dependency conflicts.

Using Conda

conda create -n moss-tts python=3.12 -y
conda activate moss-tts

Install the FFmpeg system dependency (required by torchcodec for audio I/O):

# Debian/Ubuntu
sudo apt-get install -y ffmpeg
# macOS
brew install ffmpeg

Install all required Python dependencies:

git clone https://github.com/OpenMOSS/MOSS-TTS.git
cd MOSS-TTS
pip install --extra-index-url https://download.pytorch.org/whl/cu128 -e ".[torch-runtime]"

Using uv

# Install uv first: https://docs.astral.sh/uv/getting-started/installation/
git clone https://github.com/OpenMOSS/MOSS-TTS.git
cd MOSS-TTS
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install --torch-backend cu128 -e ".[torch-runtime]"

(Optional) Install FlashAttention 2

For better speed and lower GPU memory usage, you can install FlashAttention 2 if your hardware supports it.

If you use Conda/pip:

pip install --extra-index-url https://download.pytorch.org/whl/cu128 -e ".[torch-runtime,flash-attn]"

If your machine has limited RAM and many CPU cores, you can cap build parallelism:

MAX_JOBS=4 pip install --extra-index-url https://download.pytorch.org/whl/cu128 -e ".[torch-runtime,flash-attn]"

If you use uv:

uv pip install --torch-backend cu128 -e ".[torch-runtime,flash-attn]"

If your machine has limited RAM and many CPU cores, you can cap build parallelism:

MAX_JOBS=4 uv pip install --torch-backend cu128 -e ".[torch-runtime,flash-attn]"

Notes:

  • Dependencies are managed in pyproject.toml, which currently pins torch==2.9.1+cu128 and torchaudio==2.9.1+cu128.
  • In uv, --torch-backend cu128 lets uv fetch compatible PyTorch CUDA wheels and resolve the rest from PyPI with the default safe index strategy.
  • If you need another backend, replace cu128 with your target (for example, cpu, cu126).
  • If FlashAttention 2 fails to build on your machine, you can skip it and use the default attention backend.
  • FlashAttention 2 is only available on supported GPUs and is typically used with torch.float16 or torch.bfloat16.

MOSS‑TTS Basic Usage

If you prefer browser demos, we provide scripts for the main models:

ModelScript
MOSS-TTSclis/moss_tts_app.py
MOSS-TTS-Local-Transformer-v1.5clis/moss_tts_local_v1.5_app.py
MOSS-TTSDclis/moss_ttsd_app.py
MOSS-VoiceGeneratorclis/moss_voice_generator_app.py
MOSS-SoundEffectclis/moss_sound_effect_app.py

For the MOSS-TTS-Realtime Gradio demo, please refer to MOSS-TTS-Realtime Model Card

Tip: MOSS-TTS-v1.5 uses the same generation API as the 1.0 MossTTSDelay-8B checkpoint. For multilingual inputs, set language whenever the language is known.

MOSS-TTS provides a convenient generate interface for rapid usage. The examples below cover:

  1. Direct generation (Chinese / English / multilingual text with language tags / Pinyin / IPA)
  2. Voice cloning
  3. Duration control
  4. Explicit pause control with [pause X.Ys]
from pathlib import Path
import importlib.util
import torch
import torchaudio
from transformers import AutoModel, AutoProcessor
# Disable the broken cuDNN SDPA backend
torch.backends.cuda.enable_cudnn_sdp(False)
# Keep these enabled as fallbacks
torch.backends.cuda.enable_flash_sdp(True)
torch.backends.cuda.enable_mem_efficient_sdp(True)
torch.backends.cuda.enable_math_sdp(True)


pretrained_model_name_or_path = "OpenMOSS-Team/MOSS-TTS-v1.5"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32

def resolve_attn_implementation() -> str:
    # Prefer FlashAttention 2 when package + device conditions are met.
    if (
        device == "cuda"
        and importlib.util.find_spec("flash_attn") is not None
        and dtype in {torch.float16, torch.bfloat16}
    ):
        major, _ = torch.cuda.get_device_capability()
        if major >= 8:
            return "flash_attention_2"

    # CUDA fallback: use PyTorch SDPA kernels.
    if device == "cuda":
        return "sdpa"

    # CPU fallback.
    return "eager"


attn_implementation = resolve_attn_implementation()
print(f"[INFO] Using attn_implementation={attn_implementation}")

processor = AutoProcessor.from_pretrained(
    pretrained_model_name_or_path,
    trust_remote_code=True,
)
processor.audio_tokenizer = processor.audio_tokenizer.to(device)

text_1 = "亲爱的你，\n你好呀。\n\n今天，我想用最认真、最温柔的声音，对你说一些重要的话。\n这些话，像一颗小小的星星，希望能在你的心里慢慢发光。\n\n首先，我想祝你——\n每天都能平平安安、快快乐乐。\n\n希望你早上醒来的时候，\n窗外有光，屋子里很安静，\n你的心是轻轻的，没有着急，也没有害怕。\n\n希望你吃饭的时候胃口很好，\n走路的时候脚步稳稳，\n晚上睡觉的时候，能做一个又一个甜甜的梦。\n\n我希望你能一直保持好奇心。\n对世界充满问题，\n对天空、星星、花草、书本和故事感兴趣。\n当你问“为什么”的时候，\n希望总有人愿意认真地听你说话。\n\n我也希望你学会温柔。\n温柔地对待朋友，\n温柔地对待小动物，\n也温柔地对待自己。\n\n如果有一天你犯了错，\n请不要太快责怪自己，\n因为每一个认真成长的人，\n都会在路上慢慢学会更好的方法。\n\n愿你拥有勇气。\n当你站在陌生的地方时，\n当你第一次举手发言时，\n当你遇到困难、感到害怕的时候，\n希望你能轻轻地告诉自己：\n“我可以试一试。”\n\n就算没有一次成功，也没有关系。\n失败不是坏事，\n它只是告诉你，你正在努力。\n\n我希望你学会分享快乐。\n把开心的事情告诉别人，\n把笑声送给身边的人，\n因为快乐被分享的时候，\n会变得更大、更亮。\n\n如果有一天你感到难过，\n我希望你知道——\n难过并不丢脸，\n哭泣也不是软弱。\n\n愿你能找到一个安全的地方，\n慢慢把心里的话说出来，\n然后再一次抬起头，看见希望。\n\n我还希望你能拥有梦想。\n这个梦想也许很大，\n也许很小，\n也许现在还说不清楚。\n\n没关系。\n梦想会和你一起长大，\n在时间里慢慢变得清楚。\n\n最后，我想送你一个最最重要的祝福：\n\n愿你被世界温柔对待，\n也愿你成为一个温柔的人。\n\n愿你的每一天，\n都值得被记住，\n都值得被珍惜。\n\n亲爱的你，\n请记住，\n你是独一无二的，\n你已经很棒了，\n而你的未来，\n一定会慢慢变得闪闪发光。\n\n祝你健康、勇敢、幸福，\n祝你永远带着笑容向前走。"
text_2 = "We stand on the threshold of the AI era.\nArtificial intelligence is no longer just a concept in laboratories, but is entering every industry, every creative endeavor, and every decision. It has learned to see, hear, speak, and think, and is beginning to become an extension of human capabilities. AI is not about replacing humans, but about amplifying human creativity, making knowledge more equitable, more efficient, and allowing imagination to reach further. A new era, jointly shaped by humans and intelligent systems, has arrived."
text_3 = "nin2 hao3，qing3 wen4 nin2 lai2 zi4 na3 zuo4 cheng2 shi4？"
text_4 = "nin2 hao3，qing4 wen3 nin2 lai2 zi4 na4 zuo3 cheng4 shi3？"
text_5 = "您好，请问您来自哪 zuo4 cheng2 shi4？"
text_6 = "/həloʊ, meɪ aɪ æsk wɪtʃ sɪti juː ɑːr frʌm?/"
text_7 = "Bonjour, je voudrais essayer une voix française naturelle et stable."
text_8 = "我今天学习了一首中国的古诗，它的名字是[pause 3.2s]静夜思！"

# Use audio from ./assets/audio to avoid downloading from the cloud.
ref_audio_1 = "https://speech-demo.oss-cn-shanghai.aliyuncs.com/moss_tts_demo/tts_readme_demo/reference_zh.wav"
ref_audio_2 = "https://speech-demo.oss-cn-shanghai.aliyuncs.com/moss_tts_demo/tts_readme_demo/reference_en.m4a"

conversations = [
    # Direct TTS (no reference). Language tags are recommended in v1.5.
    [processor.build_user_message(text=text_1)],
    [processor.build_user_message(text=text_2)],
    # Direct TTS (no reference). For languages other than Chinese and English,
    # set the language tag whenever it is known.
    [processor.build_user_message(text=text_7, language="French")],
    # Pinyin or IPA input
    [processor.build_user_message(text=text_3)],
    [processor.build_user_message(text=text_4)],
    [processor.build_user_message(text=text_5)],
    [processor.build_user_message(text=text_6)],
    # Explicit pause control. Use [pause X.Ys], such as [pause 3.2s].
    [processor.build_user_message(text=text_8)],
    # Voice cloning (with reference)
    [processor.build_user_message(text=text_1, reference=[ref_audio_1])],
    [processor.build_user_message(text=text_2, reference=[ref_audio_2])],
    # Duration control
    [processor.build_user_message(text=text_2, tokens=325)],
    [processor.build_user_message(text=text_2, tokens=600)],
]

model = AutoModel.from_pretrained(
    pretrained_model_name_or_path,
    trust_remote_code=True,
    attn_implementation=attn_implementation,
    dtype=dtype,
).to(device)
model.eval()

batch_size = 1

save_dir = Path("inference_root")
save_dir.mkdir(exist_ok=True, parents=True)
sample_idx = 0
with torch.no_grad():
    for start in range(0, len(conversations), batch_size):
        batch_conversations = conversations[start : start + batch_size]
        batch = processor(batch_conversations, mode="generation")
        input_ids = batch["input_ids"].to(device)
        attention_mask = batch["attention_mask"].to(device)

        outputs = model.generate(
            input_ids=input_ids,
            attention_mask=attention_mask,
            max_new_tokens=4096,
        )

        for message in processor.decode(outputs):
            audio = message.audio_codes_list[0]
            out_path = save_dir / f"sample{sample_idx}.wav"
            sample_idx += 1
            if audio.ndim == 1:
                audio = audio.unsqueeze(0)
            torchaudio.save(
                str(out_path),
                audio.detach().cpu().to(torch.float32),
                processor.model_config.sampling_rate,
            )

For each model's full usage, please refer to its corresponding model card. For the 48 kHz stereo local-transformer v1.5 checkpoint and realtime streaming decode, see moss_tts_local_v1.5/.

Fine-Tuning

Finetuning tutorials are organized by architecture.

Currently available:

Additional architecture-specific finetuning tutorials will be added under their corresponding directories.

llama.cpp Backend (Torch-Free Inference)

For lightweight or edge deployment, MOSS-TTS supports a torch-free inference path using llama.cpp for the Qwen3 backbone and ONNX Runtime / TensorRT for the audio tokenizer. No PyTorch installation required.

We also maintain a newer first-class MOSS-TTS path in the companion repository OpenMOSS/llama.cpp. Unlike the legacy bridge backend documented below, it moves multi-channel embeddings, multi-head outputs, and delay-pattern decoding directly into llama.cpp.

For that path, start from the first-class e2e guide.

Quick Start

# 1. Install (torch-free)
pip install -e ".[llama-cpp-onnx]"

# 2. Download pre-quantized backbone + embedding/lm_head weights
huggingface-cli download OpenMOSS-Team/MOSS-TTS-GGUF --local-dir weights/MOSS-TTS-GGUF

# 3. Download ONNX audio tokenizer
huggingface-cli download OpenMOSS-Team/MOSS-Audio-Tokenizer-ONNX --local-dir weights/MOSS-Audio-Tokenizer-ONNX

# 4. Build the C bridge (one-time, requires llama.cpp compiled from source)
cd moss_tts_delay/llama_cpp && bash build_bridge.sh /path/to/llama.cpp && cd ../..

# 5. Run inference
python -m moss_tts_delay.llama_cpp \
    --config configs/llama_cpp/default.yaml \
    --text "Hello, world!" --output output.wav

# 6. (Optional) Low-memory mode for 8 GB GPUs — loads/unloads components per stage
python -m moss_tts_delay.llama_cpp \
    --config configs/llama_cpp/trt-8gb.yaml \
    --text "Hello, world!" --output output.wav

Installation Profiles

ProfileInstall CommandDependenciesUse Case
Torch-free (ONNX)pip install -e ".[llama-cpp-onnx]"numpy, onnxruntime-gpu, tokenizersRecommended starting point
Torch-free (TRT)pip install -e ".[llama-cpp-trt]"numpy, tensorrt, cuda-pythonMaximum audio tokenizer speed (build engines yourself)
Torch-acceleratedpip install -e ".[llama-cpp-onnx,llama-cpp-torch]"+ torchGPU-accelerated LM heads (~30x faster)

Want to convert weights yourself? See the conversion guide for step-by-step instructions on extracting, converting, and quantizing MOSS-TTS weights with llama.cpp.

Model Weights

RepositoryContentsDownload
OpenMOSS-Team/MOSS-TTS-GGUFQ4_K_M backbone .gguf, embeddings/ (.npy), lm_heads/ (.npy), tokenizerhuggingface-cli download OpenMOSS-Team/MOSS-TTS-GGUF --local-dir weights/MOSS-TTS-GGUF
OpenMOSS-Team/MOSS-Audio-Tokenizer-ONNXEncoder & decoder ONNX modelshuggingface-cli download OpenMOSS-Team/MOSS-Audio-Tokenizer-ONNX --local-dir weights/MOSS-Audio-Tokenizer-ONNX

Note: We do not provide pre-built TensorRT engines, as they are tied to your specific GPU and TensorRT version. To use TRT, build engines from the ONNX models yourself — see moss_audio_tokenizer/trt/build_engine.sh.

Configuration

Four pre-built configs are provided in configs/llama_cpp/:

  • default.yaml — ONNX audio + GGUF backbone (recommended start)
  • trt.yaml — TensorRT audio + GGUF backbone (max throughput, user-built engines)
  • trt-8gb.yaml — Low-memory mode for 8 GB GPUs (staged loading, TRT audio)
  • cpu-only.yaml — fully CPU-based (no GPU required)

Key config options:

  • heads_backend: auto | numpy | torch — LM heads computation backend
  • audio_backend: onnx | trt | torch — audio tokenizer backend
  • low_memory: true | false — staged loading for limited VRAM (loads/unloads encoder, backbone, decoder per stage)
  • kv_cache_type_k / kv_cache_type_v — KV cache quantization (e.g. q8_0, q4_0) to reduce VRAM
  • flash_attn: auto | enabled | disabled — flash attention for lower peak VRAM during prefill

For full documentation, see moss_tts_delay/llama_cpp/README.md.

Accelerated Inference Backends

MOSS-TTS models can be served with high-throughput inference backends for production deployment. Both SGLang-Omni and vLLM-Omni provide OpenAI-compatible serving with streaming and voice cloning support.

SGLang-Omni

SGLang-Omni supports the MossTTSDelay and MossTTSLocal architectures. It is currently the only inference backend that supports the MossTTSLocal architecture, providing Day-0 support for MOSS-TTS-Local-Transformer-v1.5.

For setup and usage, see the cookbooks:

vLLM-Omni

vLLM-Omni supports the MossTTSDelay, MossTTSRealtime, and MossTTSNano architectures, covering MOSS-TTS, MOSS-TTS-v1.5, MOSS-TTSD, MOSS-SoundEffect, MOSS-VoiceGenerator, MOSS-TTS-Realtime, and MOSS-TTS-Nano.

For setup and usage, see:

Evaluation

This section summarizes the family‑level evaluation highlights for MOSS‑TTS, MOSS-TTSD and MOSS‑VoiceGenerator. For full details, see each model’s model card.

MOSS‑TTS

MOSS‑TTS achieved state‑of‑the‑art results on the open‑source zero‑shot TTS benchmark Seed‑TTS‑eval, surpassing all open‑source models and rivaling leading closed‑source systems.

ModelParamsOpen‑sourceEN WER (%) ↓EN SIM (%) ↑ZH CER (%) ↓ZH SIM (%) ↑
DiTAR0.6B❌1.6973.51.0275.3
FishAudio‑S14B❌1.7262.571.2272.1
CosyVoice31.5B❌2.22721.1278.1
Seed‑TTS❌2.2576.21.1279.6
MiniMax‑Speech❌1.6569.20.8378.3
CosyVoice0.3B✅4.2960.93.6372.3
CosyVoice20.5B✅3.0965.91.3875.7
CosyVoice30.5B✅2.0271.81.1678
F5‑TTS0.3B✅2671.5376
SparkTTS0.5B✅3.1457.31.5466
FireRedTTS0.5B✅3.82461.5163.5
FireRedTTS‑21.5B✅1.9566.51.1473.6
Qwen2.5‑Omni7B✅2.7263.21.775.2
FishAudio‑S1‑mini0.5B✅1.94551.1868.5
IndexTTS21.5B✅2.2370.61.0376.5
VibeVoice1.5B✅3.0468.91.1674.4
HiggsAudio‑v23B✅2.4467.71.574
GLM-TTS1.5B✅2.2367.21.0376.1
GLM-TTS-RL1.5B✅1.9168.10.8976.4
VoxCPM0.5B✅1.8572.90.9377.2
Qwen3‑TTS0.6B✅1.6870.391.2376.4
Qwen3‑TTS1.7B✅1.571.451.3376.72
MossTTSDelay8B✅1.8470.861.3776.98
MossTTSLocal1.7B✅1.9373.281.4479.62

MOSS‑TTSD

Objective Evaluation

We evaluate MOSS‑TTSD-v1.0 using three objective metrics: Speaker Attribution Accuracy (ACC), Speaker Similarity (SIM), and Word Error Rate (WER). Benchmarked against multiple open-source and closed-source models, the results show that MOSS‑TTSD-v1.0 consistently achieves either the best or second-best performance.

ModelZH - SIMZH - ACCZH - WEREN - SIMEN - ACCEN - WER
Comparison with Open-Source Models
MOSS-TTSD-v1.00.79490.95870.04850.73260.96260.0988
MOSS-TTSD-v0.70.74230.93910.05170.67430.92660.1612
Vibevoice 7B0.75900.92220.05700.71400.95540.0946
Vibevoice 1.5 B0.74150.87980.08180.69610.93530.1133
FireRedTTS20.73830.90220.0768---
Higgs Audio V2---0.68600.90250.2131
Comparison with Proprietary Models
MOSS-TTSD-v1.0 (elevenlabs_voice)0.81650.97360.03910.73040.95650.1005
Eleven V30.69700.96530.03630.67300.94980.0824
MOSS-TTSD-v1.0 (gemini_voice)---0.78930.96550.0984
gemini-2.5-pro-preview-tts---0.67860.95370.0859
gemini-2.5-flash-preview-tts---0.71940.95110.0871
MOSS-TTSD-v1.0 (doubao_voice)0.82260.96300.0571---
Doubao_Podcast0.80340.96060.0472---

Subjective Evaluation

For open-source models, annotators are asked to score each sample pair in terms of speaker attribution accuracy, voice similarity, prosody, and overall quality. Following the methodology of the LMSYS Chatbot Arena, we compute Elo ratings and confidence intervals for each dimension. alt text

For closed-source models, annotators are only asked to choose the overall preferred one in each pair, and we compute the win rate accordingly. alt text

MOSS‑VoiceGenerator

MOSS‑VoiceGenerator demonstrates strong subjective preference across overall preference, instruction following, and naturalness.

MOSS‑TTS-Realtime

We evaluated the TTFB (Time To First Byte) and RTF (Real-Time Factor) of MOSS-TTS-Realtime.

Note: SDPA + torch.compile were enabled during testing. The following results are tested on a single L20 GPU.

ModelTTFB (ms)RTF
MOSS-TTS-Realtime180（After warm up）0.51

We deployed Qwen3.5-9B using vLLM to measure $T_{\text{LLM-first-sentence}}$. The time required to generate 12 tokens (the TTS prefill length) was 197 ms.

$T_{\text{LLM-first-sentence}} + T_{\text{MOSS-TTS-Realtime-TTFB}} = 197ms + 180ms = 377ms$

MOSS-TTS-Nano

Introduction

MOSS-TTS-Nano is our lightweight TTS model designed for CPU-first realtime deployment. It focuses on the part of speech generation that matters most in practical products: a small footprint, low-latency streaming generation, and voice-cloning quality that is strong enough for local demos, web services, and lightweight production integration. Built on a pure autoregressive Audio Tokenizer + LLM pipeline, it keeps the deployment stack simple while making realtime speech generation accessible even without a GPU.

Its main features include:

  • 0.1B parameters: a compact model size that keeps memory usage and deployment cost low, making local deployment and lightweight serving much easier.
  • Realtime generation on just 4 CPU cores: streaming speech generation can run efficiently on CPU-only environments, which is practical for local applications and cost-sensitive deployment scenarios.
  • Multilingual voice cloning: supports multilingual voice cloning workflows, making it suitable for cross-language synthesis with a single reference speaker.
  • 48 kHz stereo input/output: natively supports high-quality stereo audio, improving both reference-audio fidelity and final listening quality.

To learn more about setup, advanced usage, and evaluation metrics, please visit the MOSS-TTS-Nano repository.

MOSS TTS Nano architecture

Architecture of MOSS-TTS-Nano

Model Weights

ModelHugging FaceModelScope
MOSS-TTS-NanoHugging FaceModelScope

MOSS-Audio-Tokenizer

Introduction

MOSS-Audio-Tokenizer serves as the unified discrete audio interface for the entire MOSS-TTS Family. It is based on the Cat (Causal Audio Tokenizer with Transformer) architecture—a 1.6-billion-parameter, "CNN-free" homogeneous audio tokenizer built entirely from Causal Transformer blocks.

  • Unified Discrete Bridge: It acts as the shared backbone for MOSS-TTS, MOSS-TTSD, MOSS-VoiceGenerator, MOSS-SoundEffect, and MOSS-TTS-Realtime, providing a consistent audio representation across the family.
  • Extreme Compression & High Fidelity: It compresses 24kHz raw audio into a remarkably low frame rate of 12.5Hz. Utilizing a 32-layer Residual Vector Quantizer (RVQ), it supports high-fidelity reconstruction across variable bitrates from 0.125kbps to 4kbps.
  • Massive-Scale General Audio Training: Trained from scratch on 3 million hours of diverse data (speech, sound effects, and music), the model achieves state-of-the-art reconstruction among open source audio tokenizers.
  • Native Streaming Design: The pure Causal Transformer architecture is specifically designed for scalability and low-latency streaming inference, enabling real-time production workflows.

To learn more about setup, advanced usage, and evaluation metrics, please visit the MOSS-Audio-Tokenizer Repository

MOSS Audio Tokenizer architecture Architecture of MOSS Audio Tokenizer

Model Weights

ModelSampling RateChannelsHugging FaceModelScope
MOSS-Audio-Tokenizer24kHz1 channel (mono)HFMS
MOSS-Audio-Tokenizer-v248kHz2 channels (stereo)Hugging FaceModelScope
MOSS-Audio-Tokenizer-Nano48kHz2 channels (stereo)Hugging FaceModelScope

Objective Reconstruction Evaluation

We compare MOSS Audio Tokenizer with open-source audio tokenizers on the LibriSpeech test-clean subset using SIM, STOI, PESQ-NB, and PESQ-WB. Bitrate is controlled by varying the number of RVQ codebooks during decoding, and MOSS Audio Tokenizer leads reconstruction quality among open-source audio tokenizers at comparable 0–4 kbps bitrates.

LibriSpeech objective metrics for audio tokenizers

📚 More Information

🌟 Community Projects

The MOSS-TTS community has been growing rapidly, and we’re delighted to showcase some outstanding projects and features built by community members:

LICENSE

Models in MOSS-TTS Family are licensed under the Apache License 2.0.

Citation

@misc{gong2026mossttstechnicalreport,
      title={MOSS-TTS Technical Report}, 
      author={Yitian Gong and Botian Jiang and Yiwei Zhao and Yucheng Yuan and Kuangwei Chen and Yaozhou Jiang and Cheng Chang and Dong Hong and Mingshu Chen and Ruixiao Li and Yiyang Zhang and Yang Gao and Hanfu Chen and Ke Chen and Songlin Wang and Xiaogui Yang and Yuqian Zhang and Kexin Huang and ZhengYuan Lin and Kang Yu and Ziqi Chen and Jin Wang and Zhaoye Fei and Qinyuan Cheng and Shimin Li and Xipeng Qiu},
      year={2026},
      eprint={2603.18090},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2603.18090}, 
}

@misc{zhang2026mossttsdtextspokendialogue,
      title={MOSS-TTSD: Text to Spoken Dialogue Generation}, 
      author={Yuqian Zhang and Donghua Yu and Zhengyuan Lin and Botian Jiang and Mingshu Chen and Yaozhou Jiang and Yiwei Zhao and Yiyang Zhang and Yucheng Yuan and Hanfu Chen and Kexin Huang and Jun Zhan and Cheng Chang and Zhaoye Fei and Shimin Li and Xiaogui Yang and Qinyuan Cheng and Xipeng Qiu},
      year={2026},
      eprint={2603.19739},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2603.19739}, 
}

@misc{huang2026mossvoicegeneratorcreaterealisticvoices,
      title={MOSS-VoiceGenerator: Create Realistic Voices with Natural Language Descriptions}, 
      author={Kexin Huang and Liwei Fan and Botian Jiang and Yaozhou Jiang and Qian Tu and Jie Zhu and Yuqian Zhang and Yiwei Zhao and Chenchen Yang and Zhaoye Fei and Shimin Li and Xiaogui Yang and Qinyuan Cheng and Xipeng Qiu},
      year={2026},
      eprint={2603.28086},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2603.28086}, 
}

Star History

Star History Chart