Google Git
Sign in
rust / rust-lang / reference / refs/heads/TC/parse-without-regex
commit1a7304b8a42a37366946de00f449b1b6a041ebee[log] [tgz]
authorTravis Cross <tc@traviscross.com>Thu May 15 15:00:56 2025 +0000
committerTravis Cross <tc@traviscross.com>Fri May 16 07:27:34 2025 +0000
tree361dc8b08d9fa1f78a0bd70d02d425a4a468fa2b
parentc703c8d13456d4a6fdc83c9b39b370af3df12d00 [diff]
Parse grammar without regexes

We'd been parsing the grammar with a combination of recursive descent
and regular expression matchers.  This combination has its merits, and
it's done tastefully here, but it seems maybe more straightforward to
do the parsing entirely with recursive descent.  Among other things,
doing it this way allows us to provide more precise error reporting on
malformed inputs.

The cost, in terms of lines of code, of doing this is rather modest,
and the result seems at least as clear -- there's some mental cost to
code switching between the two worlds.  So let's make the switch and
parse the grammar without regular expressions.

We verified that the rendered output of the Reference is byte
identical before and after this change.
1 file changed
tree: 361dc8b08d9fa1f78a0bd70d02d425a4a468fa2b
  1. .cargo/
  2. .github/
  3. docs/
  4. mdbook-spec/
  5. src/
  6. style-check/
  7. theme/
  8. xtask/
  9. .gitattributes
  10. .gitignore
  11. book.toml
  12. CONTRIBUTING.md
  13. LICENSE-APACHE
  14. LICENSE-MIT
  15. README.md
  16. reference.md
  17. rust-toolchain.toml
  18. rustfmt.toml
  19. STYLE.md
  20. triagebot.toml
README.md

The Rust Language Reference

This document is the primary reference for the Rust programming language.

Dependencies

Installing dependencies

First, ensure that you have a recent copy of the nightly Rust compiler installed, as this is needed in order to run the tests:

rustup toolchain install nightly

Now, ensure you have mdbook installed, as this is needed in order to build the Reference:

cargo install --locked mdbook

Building

To build the Reference, first clone the project:

git clone https://github.com/rust-lang/reference.git

(Alternatively, if you don't want to use git, download a ZIP file of the project, extract it using your preferred tool, and rename the top-level directory to reference.)

Now change your current directory to the working directory:

cd reference

To test all of the code examples in the Reference, run:

mdbook test

For authors, consider using the server functionality which supports automatic reload.

To build the Reference locally (in build/) and open it in a web browser, run:

SPEC_RELATIVE=0 mdbook build --open

This will open a browser with a websocket live-link to automatically reload whenever the source is updated.

You can also use mdbook's live webserver option, which will automatically rebuild the book and reload your web browser whenever a source file is modified:

SPEC_RELATIVE=0 mdbook serve --open

SPEC_RELATIVE

The SPEC_RELATIVE=0 environment variable makes links to the standard library go to https://doc.rust-lang.org/ instead of being relative, which is useful when viewing locally since you normally don't have a copy of the standard library.

The published site at https://doc.rust-lang.org/reference/ (or local docs using rustup doc) does not set this, which means it will use relative links which supports offline viewing and links to the correct version (for example, links in https://doc.rust-lang.org/1.81.0/reference/ will stay within the 1.81.0 directory).

SPEC_DENY_WARNINGS

The SPEC_DENY_WARNINGS=1 environment variable will turn all warnings generated by mdbook-spec to errors. This is used in CI to ensure that there aren't any problems with the book content.

SPEC_RUST_ROOT

The SPEC_RUST_ROOT can be used to point to the directory of a checkout of https://github.com/rust-lang/rust. This is used by the test-linking feature so that it can find tests linked to reference rules. If this is not set, then the tests won't be linked.