Teaching a model how to speak

A voice interface is experienced in time. Pronunciation, pacing, silence, and continuity all contribute to whether a response feels coherent. A lower training loss is only one part of that story.

A concrete starting point

The public English CSM podcast model card documents a fine-tune of sesame/csm-1b on Vyvo/en-podcast-1. Its reported setup provides a useful anchor for discussing adaptation.

Training samples

68,891

LoRA rank / alpha

32 / 32

Epochs

4

Configuration reported by the model card; these values are not independent quality measurements.

What LoRA changes

Low-rank adaptation keeps the original weights frozen and learns an update through two smaller matrices. In the usual formulation, A projects into a rank-r space, and B projects back into the output space.

The attraction is practical: fewer trainable parameters and portable adapters. Where supported, the learned update can be merged into the base weights for deployment.

Make the parameter budget tangible

For one projection, the adapter has r(din + dout) parameters. A small calculation makes the difference visible without loading a model.

def lora_parameters(in_features, out_features, rank):
    return rank * (in_features + out_features)


# An illustrative 4096 × 4096 projection, without biases.
base = 4096 * 4096
adapter = lora_parameters(4096, 4096, rank=32)

print(f"Base weights: {base:,}")
print(f"Adapter weights: {adapter:,}")
print(f"Adapter / base: {adapter / base:.2%}")

Base weights: 16,777,216
Adapter weights: 262,144
Adapter / base: 1.56%

The 4,096-dimensional layer is an illustrative example, not a claim about this CSM model’s layer dimensions. The calculation excludes biases and counts the added adapter parameters.

The metric is not the experience

The card reports training loss falling from 7.56 to 6.77. That describes the optimization process. It does not, by itself, establish better intelligibility, voice similarity, or a preferred listening experience.

Think in systems

For an interactive product, the model is one component. Preparation, generation, buffering, and playback all shape the response.

Make the listening experience the target, and use training metrics to investigate how to get there.