Measuring an Author's Voice With Code, Not Vibes

"Write in my style" is an instruction a model cannot check. It has no copy of your style and no definition of it, so it cannot say whether the paragraph it just wrote is closer to your voice than the one before. What it has is your sample compressed into an impression, and an impression is not something you can test. The instruction sounds precise and changes nothing you can verify.

The alternative is to measure voice before asking anyone to match it. Take a body of the author's own edited prose, compute a fixed set of features from it with a script, and store the result as a profile. Then score every draft against that profile. The output is a table of numbers and a distance, not an opinion.

Why "match my style" fails as an instruction

A checkable instruction needs three parts: a value, a tolerance and a way to fail. "Write shorter sentences" has none of them. Shorter than what, by how much, and how would you know it did not? The model cannot answer, so it does what language models do with vague constraints: it produces the average of everything it has seen that looks like the request.

Asking the model to judge its own output does not help. It is agreeable by training, and it has no reference text to compare against beyond the context window. Even when you paste samples, attention favours what is distinctive and recent. One long lyrical paragraph in a sample can pull a whole chapter toward its register.

A script has none of these problems. It counts. The same text gives the same numbers whoever runs it, and a draft either falls inside the profile's range or it does not. That is what makes the instruction checkable: it becomes a measurement with a threshold.

What you can extract from a corpus

Everything below is a count over text. None of it requires understanding what the sentences mean, which is exactly why it can be computed deterministically.

Sentence length and its distribution

Record the mean, the median, the 10th and 90th percentiles, and the share of sentences longer than 40 words. The mean on its own hides the thing readers actually notice. An author who alternates six-word sentences with forty-five-word ones can have the same average as one who writes every sentence at twenty-five words, and the two read nothing alike. Store the distribution.

Split on terminal punctuation with a regular expression, then repair the splits that abbreviations cause: Mr., Dr., etc., initials and ellipses. Count words with a split on whitespace after normalising apostrophes. In Python, statistics.quantiles(lengths, n=10) gives the deciles in one call.

Paragraph length

Words per paragraph and sentences per paragraph. In fiction this is where scene-setting and dialogue separate; in non-fiction it is the most visible rhythm after sentence length. Count paragraphs on blank lines rather than indentation, because indentation rarely survives conversion between formats.

Dialogue against narration

Two ratios are worth keeping. The share of paragraphs that contain quoted speech measures how often the text cuts to dialogue. The share of words inside quotation marks measures how much of the text is dialogue. They move independently: a chapter can have many short exchanges and still be mostly narration. Normalise quote characters first, because curly quotes, straight quotes and the low-high quotes used in Romanian are different characters to a regular expression.

Lexical range

The type-token ratio is the number of distinct words divided by the total number of words. Lexical diversity falls as a text gets longer, because any writer repeats words, so a raw ratio compares length as much as vocabulary. Compare windows of equal length instead, say 1,000 words, and average the windows. A count of words used exactly once, the hapax share, is a useful companion.

Adverb density

Adverbs per 1,000 words. This one is language-specific. In English, the -ly suffix catches many adverbs and misses others such as fast, well and very, while overcounting words like only and early. A part-of-speech tagger is more accurate: spaCy labels the coarse tag ADV. In Romanian the suffix trick fails badly, because -mente is not productive the way -ly is; you need a tagger or a curated list.

Punctuation habits

Counts per 1,000 words for semicolons, colons, em dashes, ellipses, exclamation marks, question marks and parentheses. Count code points, not appearances: an em dash is one character, an en dash another, a hyphen another. Convert the corpus to one encoding and one quote style before counting, or the numbers measure your file conversion instead of the author.

Sentence openers

Take the first word of every sentence and the first part of speech, then keep the top twenty. Writers have tics that survive editing: opening with And, with a participial phrase, with a pronoun, with a subordinate clause. The distribution of openers is more distinctive than vocabulary, because it is harder to fake consciously.

Tense and person

Measure the share of finite verbs in past and present tense, and the rate of first-person singular, first-person plural, second person and third person. Do it per chapter. Point of view and tense are usually constant within a chapter and vary between books, so a whole-corpus average can describe a voice that no single page uses.

Computing the profile with a script

The pipeline is five steps. Store the corpus as plain UTF-8 text, one file per chapter. Normalise quotes and dashes, and write down what you normalised. Run the feature functions over each chapter and keep the per-chapter values, not only the corpus average. Aggregate into mean, standard deviation and a few quantiles. Save the result as JSON next to the preprocessing rules and the corpus size.

Most of the features need only the Python standard library: re for splitting, statistics for the aggregates, collections.Counter for the frequency tables. Adverb density and tense need a tagger. The profile file should look something like this, with illustrative numbers:

{"sentence_length": {"mean": 17.4, "sd": 6.1, "p10": 8, "p90": 29}, "dialogue_paragraph_share": 0.31, "semicolons_per_1k": 2.4}

Two properties matter more than the exact feature list. The first is determinism: no model in the loop, so the profile can be rebuilt and compared. The second is provenance: record which files went in, which were excluded, and how the text was cleaned. A profile without that record cannot be compared with another, and in a year nobody will remember why one chapter was dropped.

Scoring a draft so the answer is a number

For each feature, compute how far the draft sits from the profile in units of the profile's own spread: subtract the corpus mean and divide by the standard deviation. That gives a z-score per feature. Report the features separately.

Do not collapse them into one average too early. A draft that sits at zero on nine features and six standard deviations away on dialogue ratio is not "mostly in voice"; it is a draft with a structural problem that an average would hide.

The threshold should come from the author, not from your taste. Hold out a few chapters when you build the profile, then score those chapters against it. Their spread is the author's own variation. A new draft inside that band is as close to the profile as the author's own work is. A draft outside it is measurably different. Without this step, any tolerance you choose is arbitrary, and you will spend the argument defending the number instead of the prose.

Two controls make the score trustworthy. Score a chapter by a different author: if the scorer cannot separate it from the target, the profile is too weak to use. Score an early unedited draft by the same author: if it scores the same as the published chapter, the features are not capturing what editing changed.

What the numbers cannot tell you

Metrics capture surface rhythm, not judgement. They can tell you that a draft has the author's sentence-length distribution, adverb rate and punctuation habits. They cannot tell you that the metaphor is tired, that the scene does not earn its place, or that the argument collapses in the third paragraph. A page can be perfectly in voice and still be bad.

An author who writes novels, essays and email has at least three voices. One profile averaged across all of them describes none of them. Build a profile per register, and inside a novel consider separating dialogue-heavy scenes from descriptive ones if the gap is large.

There is also a Goodhart problem. A model told to hit every metric will hit every metric. It will insert semicolons at the right rate, cut sentences to the median and open with the favoured words, and the result will read like something assembled to pass an inspection. The metrics are a floor. They catch drift, and they catch a ghostwriter who has quietly changed register. They do not produce quality, and they should never be the only reader.

The rules are only as good as their edge cases, too. Sentence splitting fails on abbreviations, on unmarked dialogue and on languages with different punctuation. Check a sample by hand before you trust the counts, and recheck when you add a book to the corpus.

Building the corpus

The corpus decides the result more than the code does.

• Published, edited text over drafts. A draft contains the author's process, including everything an editor later removed. If you want the voice readers know, use the version that was published.
• Strip front matter. Title pages, copyright notices, dedications, acknowledgements, about-the-author pages, publisher advertising. Public-domain files carry a header and a footer; remove both, and check the first and last pages by hand, because the boilerplate differs between sources.
• Keep chapter boundaries. One file per chapter, in order. Boundaries let you measure variance per chapter, hold chapters out for calibration, and avoid averaging a first-person chapter into a third-person one.
• Deduplicate. Collected editions repeat stories. A duplicated chapter gets double weight in every average.
• Compare equal windows for lexical measures. Type-token ratio falls with length, so use fixed 1,000-word windows rather than whole books of different sizes.
• Respect rights. Use public-domain text or files the author gave you. Project Gutenberg's terms point to its licence and warn that copyright differs outside the United States, so downloadable does not mean free to reuse.

Size matters in one direction: enough that no single chapter dominates the mean. A few dozen chapters is a reasonable start for a novel, and long chapters need fewer.

Where this fits in practice

The shape is worth copying: a profile built from published pieces, a scorer run on every draft, and a human read on anything the scorer flags or clears. It does not replace editing. It makes the first pass cheaper, because the obvious drift is already visible as a number.

Vellum, the Claude Code plugin we publish for novelists, measures style from the author's own manuscript with a Python engine, alongside its continuity checks and export. It is one implementation of the idea above.

If you have a corpus and want a profile from it, or a check that a ghostwritten draft stays inside one, see our AI systems work or book a free intro call. The measurement is the easy part. Deciding which chapters belong in the corpus is where the work is.