Voice Type Classifier (VTC)¶

The Voice Type Classifier is a free, open-source tool that listens to long audio recordings made around young children and labels who is speaking when — the child wearing the recorder, other children, adult women, or adult men.

Quick Start¶

VTC takes long, naturalistic audio recordings collected by a small device worn by a child (typically aged 0–5) and produces a timeline indicating when speech occurs and which broad category of speaker is talking: the key child wearing the recorder, other children nearby, an adult female voice, or an adult male voice. The input is one or more WAV audio files at 16 kHz, mono; the output is a spreadsheet (CSV) and a set of timestamp files (RTTM) listing every detected speech segment with a start time, duration, and speaker label.

VTC does not transcribe what was said, and it does not distinguish between two different speakers of the same type (e.g., it cannot tell two adult women apart). Accuracy varies by speaker category — detections of the key child and adult female speech are most reliable, while detections of other children are the weakest and should be interpreted with care. VTC is not a replacement for LENA: it is a different system with similar but non-identical categories, and the two are not interchangeable in analyses.

If this matches what you need, head to the Getting Started page to install VTC and run it on your first recording.

What the speaker labels mean¶

The four categories VTC uses, in plain terms.

VTC sorts every moment of detected speech into one of four buckets:

These categories are speaker types, not individual people. VTC will tell you "an adult female spoke between 3.30s and 5.28s," but it cannot tell you which adult female if more than one is present. If two siblings are both in the room, both will be labelled OCH and VTC won't separate them.

What VTC is not¶

Three common misconceptions, addressed up front, so you don't build an analysis on the wrong assumption.

• VTC is not a speech recognizer. It identifies who is speaking when. It does not produce any transcript of what was said.
• VTC is not a speaker diarizer. Diarization tools try to track individual people across a recording (e.g., "Speaker A," "Speaker B"). VTC only assigns a category (KCHI / OCH / MAL / FEM). Two different adult females will both be labelled FEM with no distinction between them.
• VTC is not a LENA replacement. Both VTC and the commercial LENA system label speakers in child-centred recordings, but they are different systems with different categories, different recorders, and different licensing. VTC is free, open-source, runs on any Unix machine, accepts ordinary WAV files, and can be fine-tuned on your own data. LENA is a commercial product that requires its own hardware recorder. Their speaker categories are similar but not identical (VTC uses KCHI/OCH/MAL/FEM, LENA uses CHN/CXN/MAN/FAN among others). For detailed accuracy comparisons, see the ExELang book, and a side-by-side comparison is on the Version History page.

How accurate is VTC?¶

VTC's labels are estimates, not ground truth — and accuracy is uneven across the four speaker categories. Always interpret results with the error rates in mind.

The table below shows how often VTC agrees with human annotators across the four categories. Higher numbers are better; the arrows (↑) indicate this in each column.

The score reported is an F1 score, which is a single number summarising two kinds of mistake: missing speech that was actually there ("false negatives") and labelling speech that wasn't there or was the wrong category ("false positives"). An F1 of 100% would mean perfect agreement with the human annotator; 0% would mean no agreement at all. The Average F1 column is the simple mean across the four speaker types.

The bottom row, Human 2, shows how well a second human annotator agrees with the first human annotator on the same recordings. This is included as a reference point: even two careful human listeners do not perfectly agree, so this row is the practical ceiling we can hope for from any automated system.

Reading the table in plain terms:

• Adult female (FEM) and the key child (KCHI) are detected most reliably — roughly comparable to human-vs-human agreement.
• Adult male (MAL) detection has improved substantially across versions but still lags behind FEM. This may partly reflect the relative scarcity of adult male speech in many child-centred recordings.
• Other children (OCH) is the weakest category. Even the best version of VTC misses or mislabels a substantial share of these segments. Treat OCH counts as rough estimates only, especially when comparing recordings or aggregating across long sessions.

Related tools¶

VTC fits into a small ecosystem of tools for processing child-centred recordings; you may want some of these for your full analysis pipeline.

Technical Reference¶

This section is intended for engineers and ML practitioners. Non-technical readers can stop here.

Evaluation methodology¶

• Benchmark: BabyTrain-2025 validation set.
• Metric: Per-class F1 over speaker activity detection, with the standard pyannote-style frame-level scoring. The Average F1 column is the unweighted macro-average over the four classes (KCHI, OCH, MAL, FEM).
• "Human 2" row: Inter-annotator agreement, computed by treating a second human annotator's labels as predictions against the first annotator's labels on the same evaluation portion of BabyTrain-2025. It is intended as a soft upper bound for what any model can be expected to achieve under the same labelling protocol.
• Default decision thresholds are tuned to maximise per-class F1 on the BabyTrain-2025 validation split (f1.toml); a high-precision threshold set (hp.toml) is also shipped and selected by maximising precision under a recall ≥ 0.5 constraint per class.

Lineage¶

See the Version History page for label-set differences and migration notes from VTC 1.0.