Project 03 / 05 · Expressive synthesis
Implemented and testedText-to-Speech & Prosody Pipeline
I built an utterance segmentation and prosody-planning pipeline with per-chunk speaking rates and sentence-type F0 targets, connected to real offline macOS text-to-speech rendering.
Readable text is not automatically natural speech. Questions, emphasis, phrasing, pauses and speaker consistency affect how a caller experiences an answer, especially under strict turn-taking deadlines.
What I implemented
- Implemented sentence segmentation and reproducible declarative prosody plans for questions, statements and emphatic responses.
- Implemented per-segment speaking-rate decisions and target pitch contours, represented separately from the synthesis engine.
- Integrated offline macOS speech rendering to AIFF files and validated output creation without external TTS accounts or API costs.
Data and execution path
- 01
Text input and sentence boundaries
- 02
Dialogue function: question / statement / emphasis
- 03
Per-chunk rate and target pitch-contour planning
- 04
Local TTS adapter invokes the macOS synthesis engine
- 05
AIFF audio output per chunk
- 06
Review intelligibility, pauses and future controllable-TTS adapter contract
Experiment and results
The local pipeline renders actual speech through macOS `say`. The rate setting is applied per chunk; the F0 curves are separate planning metadata and do not control the system synthesizer's pitch. The distinction matters because the control interface defines what expressive synthesis can actually deliver.
These results are from the specified local dataset or offline rendering pipeline. I keep them separate from real speech-corpus and live-call metrics because the populations and failure modes differ.
System-level constraints
- Text normalization and dialogue intent belong outside the acoustic renderer; pronunciation, pause placement and question inflection need separate evaluation.
- Time-to-first-audio, chunk boundaries, barge-in, speaker stability and multilingual pronunciation affect conversation quality differently.
- Objective acoustic measures do not replace human listening studies when evaluating naturalness or expressive speech.
- Neural TTS deployment couples model version, speaker conditioning, quantization and accelerator scheduling to acoustic behavior.
What the measurements establish
I did not train a neural TTS foundation model or claim emotion-, prosody- or multi-speaker-controllable synthesis with macOS `say`. Target pitch contours are prepared for a future controllable model adapter, not applied to current rendered speech.
Punctuation does not come with an acoustic renderer. A question mark deserves more than being pronounced as a period with ambition.