BENCH 02·DOCX·2026
neurotic_docx_bench
An open benchmark that scores DOCX converters and redline engines against Microsoft Word's own output: 6,427 documents, 3,502 compare pairs, failures count as zero.

/ Overview
Lawyers live in Word. A tool that converts a contract to PDF, or compares two drafts into tracked changes, is only useful if the result matches what Word itself would have produced, and almost nobody measures that. This benchmark does: it renders the same documents with Word and with each tool, and scores how far apart they land.
It is deliberately strict. A tool that crashes on a file scores zero for that file rather than being dropped from the average.
/ What it does
- Scores DOCX-to-PDF converters on 6,427 Word-exported documents, page by page, against Word’s own render.
- Scores redline engines on 3,502 document pairs against Word’s built-in compare, plus accept-all and reject-all round trips.
- Publishes every table, every tool version, and a viewer that puts Word’s page beside each engine’s with the differences overlaid.
/ Disclosure
I also write Jubarte, one of the tools ranked here; its rows are marked as author-affiliated and follow the same rules as everyone else’s. The score is pixel similarity to Word, which is a proxy: it has not yet been validated against lawyers’ judgment of which redline is better. Adapters for other tools are mine, so a low score can be the adapter’s fault rather than the tool’s; maintainers who think so are invited to dispute it on the repository.
/ Approach
Word is the oracle because Word is what the other side will open the file in. Results are generated, not hand-edited, and carry the versions and dates of every tool, so a maintainer who disagrees with a number can reproduce it or dispute it in the open.