The Reference uses mdBook to convert its source files from Markdown to HTML. See Building the Reference for more information. Additionally, there is an mdBook extension called mdbook-spec that adds a variety of custom features used by the Reference.
In addition to the book tooling, there is a variety of testing tools documented in the tests chapter.