Repeated-token titles fell by 37%.
The rate moved from 16.0% to 10.1% while automatic teacher similarity changed by less than 0.2%. That observation is genuine for those two checkpoints.
PorkiCoder Research
The Sniff TestA seed lottery, dissected.
A literary initialization appeared to reduce repeated words in one tiny language-model checkpoint. A deterministic 31-run matched control found that the favorable direction did not recur consistently across starts.
12.7Mstudent-model parameters
31deterministic control runs
907development-evaluation tasks
What survived
A real checkpoint difference, not a proven literary mechanism.
The paper separates what happened in the original pair from what the matched controls can support.
Sequence order had no stable advantage.
Ordered Verne averaged 14.75% repeated titles versus 13.79% for frequency-matched shuffled Verne and was more repetitive in six of nine matches.
This is a seed-lottery autopsy.
One architecture and nine body-donor matches do not establish a universal law about small models or prove that seed always dominates training signal.
Evidence boundary
Read the result with its constraints attached.
- The original comparison changed both the transplanted embeddings and the random transformer-body draw.
- The later 0-to-10 judge was unblinded and evaluated dependent records, so its score difference remains descriptive.
- The random donor used an fp32 initialize-and-save path, while trained donors used bf16 before fp32 export. Ordered versus shuffled is the narrower sequence-order comparison.
- Public artifacts contain aggregate outcomes and provenance hashes, not production task text, source identifiers, or row-level predictions.
Audit trail
Aggregate evidence and provenance.
The files below pin the completed control matrix, the prospective continuation rule, the recorded runtime, and the exact scientific source revision.
Complete control summary31 validated runs, aggregate metrics, validation, and hashes · JSON
Prospective interim ruleRule and recorded continuation decision · Markdown
Recorded runtime environmentGPU, CUDA, Python, and package versions · Text
Published checksumsSHA-256 pins for the paper, payload, and aggregate artifacts · Text
Scientific source commitb5aea82acf2f24003750565bcde7c9d2d8d6251c · Text
Exact retraining and row-level recomputation require controlled prompt-derived records that are not distributed here for privacy. The interactive paper remains readable without JavaScript, and its public data payload rejects incompatible or row-level schemas.