Merge ref '9f32ccf35fb8' from rust-lang/rust
Pull recent changes from https://github.com/rust-lang/rust via Josh.
Upstream ref: 9f32ccf35fb877270bc44a86a126440f04d676d0
Filtered ref: 87b13773969f65eec6762cfe4194954e7513f59b
Upstream diff: https://github.com/rust-lang/rust/compare/2f3f27bf79ec147fec9d2e7980605307a74067f4...9f32ccf35fb877270bc44a86a126440f04d676d0
This merge was created using https://github.com/rust-lang/josh-sync.
diff --git a/rust-version b/rust-version
index df2bac8..0dc9ce8 100644
--- a/rust-version
+++ b/rust-version
@@ -1 +1 @@
-2f3f27bf79ec147fec9d2e7980605307a74067f4
+9f32ccf35fb877270bc44a86a126440f04d676d0
diff --git a/src/SUMMARY.md b/src/SUMMARY.md
index a161273..2491409 100644
--- a/src/SUMMARY.md
+++ b/src/SUMMARY.md
@@ -108,6 +108,7 @@
- [Installation](./autodiff/installation.md)
- [How to debug](./autodiff/debugging.md)
- [Autodiff flags](./autodiff/flags.md)
+ - [Type Trees](./autodiff/type-trees.md)
# Source Code Representation
diff --git a/src/about-this-guide.md b/src/about-this-guide.md
index f1a406a..4f5733a 100644
--- a/src/about-this-guide.md
+++ b/src/about-this-guide.md
@@ -48,9 +48,9 @@
that are not fully realized yet.
All this makes keeping this guide completely up to date on everything very hard!
-The Guide itself is of course open-source as well,
-and the sources can be found at the [GitHub repository].
-If you find any mistakes in the guide, please file an issue about it.
+The guide itself is of course open source as well,
+and the sources are hosted on [a GitHub repository].
+If you find any mistakes in the guide, please file an issue.
Even better, open a PR with a correction!
If you do contribute to the guide,
@@ -105,7 +105,7 @@
[cheatsheet]: https://bors.rust-lang.org/
[Miri]: https://github.com/rust-lang/miri
[@bors]: https://github.com/bors
-[GitHub repository]: https://github.com/rust-lang/rustc-dev-guide/
+[a GitHub repository]: https://github.com/rust-lang/rustc-dev-guide/
[rustc API docs]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle
[Forge]: https://forge.rust-lang.org/
[compiler-team]: https://github.com/rust-lang/compiler-team/
diff --git a/src/appendix/glossary.md b/src/appendix/glossary.md
index 1837b59..21162f8 100644
--- a/src/appendix/glossary.md
+++ b/src/appendix/glossary.md
@@ -68,6 +68,7 @@
<span id="rib">rib</span> | A data structure in the name resolver that keeps track of a single scope for names. ([see more](../name-resolution.md))
<span id="rpit">RPIT</span> | A return-position `impl Trait`. ([see the reference](https://doc.rust-lang.org/reference/types/impl-trait.html#abstract-return-types)).
<span id="rpitit">RPITIT</span> | A return-position `impl Trait` in trait. Unlike RPIT, this is desugared to a generic associated type (GAT). Introduced in [RFC 3425](https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html). ([see more](../return-position-impl-trait-in-trait.md))
+<span id="rustbuild">rustbuild</span> | A deprecated term for the part of bootstrap that is written in Rust
<span id="scrutinee">scrutinee</span> | A scrutinee is the expression that is matched on in `match` expressions and similar pattern matching constructs. For example, in `match x { A => 1, B => 2 }`, the expression `x` is the scrutinee.
<span id="sess">`sess`</span> | The compiler _session_, which stores global data used throughout compilation
<span id="side-tables">side tables</span> | Because the [AST](#ast) and HIR are immutable once created, we often carry extra information about them in the form of hashtables, indexed by the id of a particular node.
diff --git a/src/autodiff/type-trees.md b/src/autodiff/type-trees.md
new file mode 100644
index 0000000..68cb786
--- /dev/null
+++ b/src/autodiff/type-trees.md
@@ -0,0 +1,193 @@
+# TypeTrees for Autodiff
+
+## What are TypeTrees?
+Memory layout descriptors for Enzyme. Tell Enzyme exactly how types are structured in memory so it can compute derivatives efficiently.
+
+## Structure
+```rust
+TypeTree(Vec<Type>)
+
+Type {
+ offset: isize, // byte offset (-1 = everywhere)
+ size: usize, // size in bytes
+ kind: Kind, // Float, Integer, Pointer, etc.
+ child: TypeTree // nested structure
+}
+```
+
+## Example: `fn compute(x: &f32, data: &[f32]) -> f32`
+
+**Input 0: `x: &f32`**
+```rust
+TypeTree(vec![Type {
+ offset: -1, size: 8, kind: Pointer,
+ child: TypeTree(vec![Type {
+ offset: 0, size: 4, kind: Float, // Single value: use offset 0
+ child: TypeTree::new()
+ }])
+}])
+```
+
+**Input 1: `data: &[f32]`**
+```rust
+TypeTree(vec![Type {
+ offset: -1, size: 8, kind: Pointer,
+ child: TypeTree(vec![Type {
+ offset: -1, size: 4, kind: Float, // -1 = all elements
+ child: TypeTree::new()
+ }])
+}])
+```
+
+**Output: `f32`**
+```rust
+TypeTree(vec![Type {
+ offset: 0, size: 4, kind: Float, // Single scalar: use offset 0
+ child: TypeTree::new()
+}])
+```
+
+## Why Needed?
+- Enzyme can't deduce complex type layouts from LLVM IR
+- Prevents slow memory pattern analysis
+- Enables correct derivative computation for nested structures
+- Tells Enzyme which bytes are differentiable vs metadata
+
+## What Enzyme Does With This Information:
+
+Without TypeTrees:
+```llvm
+; Enzyme sees generic LLVM IR:
+define float @distance(ptr %p1, ptr %p2) {
+; Has to guess what these pointers point to
+; Slow analysis of all memory operations
+; May miss optimization opportunities
+}
+```
+
+With TypeTrees:
+```llvm
+define "enzyme_type"="{[-1]:Float@float}" float @distance(
+ ptr "enzyme_type"="{[-1]:Pointer, [-1,0]:Float@float}" %p1,
+ ptr "enzyme_type"="{[-1]:Pointer, [-1,0]:Float@float}" %p2
+) {
+; Enzyme knows exact type layout
+; Can generate efficient derivative code directly
+}
+```
+
+# TypeTrees - Offset and -1 Explained
+
+## Type Structure
+
+```rust
+Type {
+ offset: isize, // WHERE this type starts
+ size: usize, // HOW BIG this type is
+ kind: Kind, // WHAT KIND of data (Float, Int, Pointer)
+ child: TypeTree // WHAT'S INSIDE (for pointers/containers)
+}
+```
+
+## Offset Values
+
+### Regular Offset (0, 4, 8, etc.)
+**Specific byte position within a structure**
+
+```rust
+struct Point {
+ x: f32, // offset 0, size 4
+ y: f32, // offset 4, size 4
+ id: i32, // offset 8, size 4
+}
+```
+
+TypeTree for `&Point` (internal representation):
+```rust
+TypeTree(vec![
+ Type { offset: 0, size: 4, kind: Float }, // x at byte 0
+ Type { offset: 4, size: 4, kind: Float }, // y at byte 4
+ Type { offset: 8, size: 4, kind: Integer } // id at byte 8
+])
+```
+
+Generates LLVM
+```llvm
+"enzyme_type"="{[-1]:Pointer, [-1,0]:Float@float, [-1,4]:Float@float, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer}"
+```
+
+### Offset -1 (Special: "Everywhere")
+**Means "this pattern repeats for ALL elements"**
+
+#### Example 1: Direct Array `[f32; 100]` (no pointer indirection)
+```rust
+TypeTree(vec![Type {
+ offset: -1, // ALL positions
+ size: 4, // each f32 is 4 bytes
+ kind: Float, // every element is float
+}])
+```
+
+Generates LLVM: `"enzyme_type"="{[-1]:Float@float}"`
+
+#### Example 1b: Array Reference `&[f32; 100]` (with pointer indirection)
+```rust
+TypeTree(vec![Type {
+ offset: -1, size: 8, kind: Pointer,
+ child: TypeTree(vec![Type {
+ offset: -1, // ALL array elements
+ size: 4, // each f32 is 4 bytes
+ kind: Float, // every element is float
+ }])
+}])
+```
+
+Generates LLVM: `"enzyme_type"="{[-1]:Pointer, [-1,-1]:Float@float}"`
+
+Instead of listing 100 separate Types with offsets `0,4,8,12...396`
+
+#### Example 2: Slice `&[i32]`
+```rust
+// Pointer to slice data
+TypeTree(vec![Type {
+ offset: -1, size: 8, kind: Pointer,
+ child: TypeTree(vec![Type {
+ offset: -1, // ALL slice elements
+ size: 4, // each i32 is 4 bytes
+ kind: Integer
+ }])
+}])
+```
+
+Generates LLVM: `"enzyme_type"="{[-1]:Pointer, [-1,-1]:Integer}"`
+
+#### Example 3: Mixed Structure
+```rust
+struct Container {
+ header: i64, // offset 0
+ data: [f32; 1000], // offset 8, but elements use -1
+}
+```
+
+```rust
+TypeTree(vec![
+ Type { offset: 0, size: 8, kind: Integer }, // header
+ Type { offset: 8, size: 4000, kind: Pointer,
+ child: TypeTree(vec![Type {
+ offset: -1, size: 4, kind: Float // ALL array elements
+ }])
+ }
+])
+```
+
+## Key Distinction: Single Values vs Arrays
+
+**Single Values** use offset `0` for precision:
+- `&f32` has exactly one f32 value at offset 0
+- More precise than using -1 ("everywhere")
+- Generates: `{[-1]:Pointer, [-1,0]:Float@float}`
+
+**Arrays** use offset `-1` for efficiency:
+- `&[f32; 100]` has the same pattern repeated 100 times
+- Using -1 avoids listing 100 separate offsets
+- Generates: `{[-1]:Pointer, [-1,-1]:Float@float}`
\ No newline at end of file
diff --git a/src/building/bootstrapping/writing-tools-in-bootstrap.md b/src/building/bootstrapping/writing-tools-in-bootstrap.md
index c3660e2..8250a6f 100644
--- a/src/building/bootstrapping/writing-tools-in-bootstrap.md
+++ b/src/building/bootstrapping/writing-tools-in-bootstrap.md
@@ -3,17 +3,26 @@
There are three types of tools you can write in bootstrap:
- **`Mode::ToolBootstrap`**
+
Use this for tools that don’t need anything from the in-tree compiler and can run with the stage0 `rustc`.
- The output is placed in the "bootstrap-tools" directory. This mode is for general-purpose tools built
- entirely with the stage0 compiler, including target libraries and only works for stage 0.
+ The output is placed in the "bootstrap-tools" directory.
+ This mode is for general-purpose tools built entirely with the stage0 compiler,
+ including target libraries, and it only works for stage 0.
- **`Mode::ToolStd`**
- Use this for tools that rely on the locally built std. The output goes into the "stageN-tools" directory.
+
+ Use this for tools that rely on the locally built std.
+ The output goes into the "stageN-tools" directory.
This mode is rarely used, mainly for `compiletest` which requires `libtest`.
- **`Mode::ToolRustcPrivate`**
- Use this for tools that use the `rustc_private` mechanism, and thus depend on the locally built `rustc` and its rlib artifacts. This is more complex than the other modes because the tool must be built with the same compiler used for `rustc` and placed in the "stageN-tools" directory. When you choose `Mode::ToolRustcPrivate`, `ToolBuild` implementation takes care of this automatically. If you need to use the builder’s compiler for something specific, you can get it from `ToolBuildResult`, which is
- returned by the tool's [`Step`].
+
+ Use this for tools that use the `rustc_private` mechanism,
+ and thus depend on the locally built `rustc` and its rlib artifacts.
+ This is more complex than the other modes because the tool must be built with the same compiler used for `rustc` and placed in the "stageN-tools" directory.
+ When you choose `Mode::ToolRustcPrivate`, `ToolBuild` implementation takes care of this automatically.
+ If you need to use the builder’s compiler for something specific,
+ you can get it from `ToolBuildResult`, which is returned by the tool's [`Step`].
Regardless of the tool type you must return `ToolBuildResult` from the tool’s [`Step`] implementation and use `ToolBuild` inside it.
diff --git a/src/building/how-to-build-and-run.md b/src/building/how-to-build-and-run.md
index b07d353..36610f2 100644
--- a/src/building/how-to-build-and-run.md
+++ b/src/building/how-to-build-and-run.md
@@ -149,7 +149,7 @@
```
PS C:\Users\vboxuser\rust> ./x
./x : File C:\Users\vboxuser\rust\x.ps1 cannot be loaded because running scripts is disabled on this system. For more
-information, see about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170.
+information, see about_Execution_Policies at https://go.microsoft.com/fwlink/?LinkID=135170.
At line:1 char:1
+ ./x
+ ~~~
diff --git a/src/compiler-team.md b/src/compiler-team.md
index 6be5283..896d9e6 100644
--- a/src/compiler-team.md
+++ b/src/compiler-team.md
@@ -1,10 +1,14 @@
# About the compiler team
+> NOTE:
+> There exists much detail about the team [on Forge], making most of the following obsolete.
+
rustc is maintained by the [Rust compiler team][team]. The people who belong to
this team collectively work to track regressions and implement new features.
Members of the Rust compiler team are people who have made significant
contributions to rustc and its design.
+[on Forge]: https://forge.rust-lang.org/compiler
[team]: https://www.rust-lang.org/governance/teams/compiler
## Discussion
diff --git a/src/fuzzing.md b/src/fuzzing.md
index 3000537..cc98b49 100644
--- a/src/fuzzing.md
+++ b/src/fuzzing.md
@@ -90,14 +90,15 @@
triggering the ICE, such as syntax errors or borrow-checking errors
- Minimize the test case (see below). If successful, you can label the
issue with `S-has-mcve`. Otherwise, you can apply `E-needs-mcve`.
-- Add the minimal test case to the rust-lang/rust repo as a [crashes test].
+- Add the minimal test case to the rust-lang/rust repo as a [crash test].
While you're at it, consider including other "untracked" crashes in your PR.
- Please don't forget to mark your issue with `S-bug-has-test` afterwards.
+ Please don't forget to mark all relevant issues with `S-bug-has-test` once
+ your PR is merged.
See also [applying and removing labels][labeling].
[bisect]: https://rust-lang.github.io/cargo-bisect-rustc/
-[crashes test]: tests/compiletest.html#crashes-tests
+[crash test]: tests/compiletest.html#crash-tests
[labeling]: https://forge.rust-lang.org/release/issue-triaging.html#applying-and-removing-labels
## Minimization
diff --git a/src/profiling/with_perf.md b/src/profiling/with_perf.md
index 0d4f23b..e452dde 100644
--- a/src/profiling/with_perf.md
+++ b/src/profiling/with_perf.md
@@ -4,8 +4,7 @@
## Initial steps
-- Get a clean checkout of rust-lang/master, or whatever it is you want
- to profile.
+- Get a clean checkout of rust-lang/rust
- Set the following settings in your `bootstrap.toml`:
- `rust.debuginfo-level = 1` - enables line debuginfo
- `rust.jemalloc = false` - lets you do memory use profiling with valgrind
diff --git a/src/tests/best-practices.md b/src/tests/best-practices.md
index efc6260..10372c3 100644
--- a/src/tests/best-practices.md
+++ b/src/tests/best-practices.md
@@ -71,7 +71,7 @@
> //! Regression test for <https://github.com/rust-lang/rust/issues/123456>.
> ```
>
-> One exception to this rule is [crashes tests]: there it is canonical that
+> One exception to this rule is [crash tests]: there it is canonical that
> tests are named only after issue numbers because its purpose is to track
> snippets from which issues no longer ICE/crash, and they would either be
> removed or converted into proper ui/other tests in the fix PRs.
@@ -199,4 +199,4 @@
[compiletest directives]: ./directives.md
[`run-make`]: ./compiletest.md#run-make-tests
[FileCheck]: https://llvm.org/docs/CommandGuide/FileCheck.html
-[crashes tests]: ./compiletest.md#crashes-tests
+[crash tests]: ./compiletest.md#crash-tests
diff --git a/src/tests/ci.md b/src/tests/ci.md
index a8cc959..1488103 100644
--- a/src/tests/ci.md
+++ b/src/tests/ci.md
@@ -1,7 +1,7 @@
# Testing with CI
The primary goal of our CI system is to ensure that the `master` branch of
-`rust-lang/rust` is always in a valid state and passes our test suite.
+`rust-lang/rust` is always in a valid state by passing our test suite.
From a high-level point of view, when you open a pull request at
`rust-lang/rust`, the following will happen:
@@ -151,8 +151,9 @@
Each job pattern can either be an exact name of a job or a glob pattern that matches multiple jobs,
for example `*msvc*` or `*-alt`. You can start at most 20 jobs in a single try build. When using
-glob patterns, you might want to wrap them in backticks (`` ` ``) to avoid GitHub rendering
-the pattern as Markdown.
+glob patterns in the PR description, you can (but do not have to) wrap them in backticks (`` ` ``) to avoid GitHub rendering
+the pattern as Markdown if it contains e.g. an asterisk. Note that this escaping will not work when using
+the `@bors jobs=` parameter.
The job pattern needs to match one or more jobs defined in the `auto` or `optional` sections
of [`jobs.yml`]:
diff --git a/src/tests/codegen-backend-tests/cg_gcc.md b/src/tests/codegen-backend-tests/cg_gcc.md
index 4caf4c0..74dbfb7 100644
--- a/src/tests/codegen-backend-tests/cg_gcc.md
+++ b/src/tests/codegen-backend-tests/cg_gcc.md
@@ -1,3 +1,56 @@
-# GCC codegen backend tests
+# GCC codegen backend
-TODO: please add some more information to this page.
+If you ran into an error related to tests executed with the GCC codegen backend on CI,
+you can use the following command to run tests locally using the GCC backend:
+
+```bash
+./x test tests/ui --set 'rust.codegen-backends = ["llvm", "gcc"]' --test-codegen-backend gcc
+```
+
+Below, you can find more information about how to configure the GCC backend in bootstrap.
+
+## Choosing which codegen backends are built
+
+The `rust.codegen-backends = [...]` bootstrap option affects which codegen backends will be built and
+included in the sysroot of the produced `rustc`. To use the GCC codegen backend, `"gcc"` has to
+be included in this array in `bootstrap.toml`:
+
+```toml
+rust.codegen-backends = ["llvm", "gcc"]
+```
+
+If you don't want to change your `bootstrap.toml` file, you can alternatively run your `x`
+commands with `--set rust.codegen-backends=["llvm", "gcc"]'`. For example:
+
+```bash
+./x build --set 'rust.codegen-backends=["llvm", "gcc"]'
+```
+
+The first backend in the `codegen-backends` array will determine which backend will be used as the
+*default backend* of the built `rustc`. This also determines which backend will be used to compile the
+stage 1 standard library (or anything built in stage 2+). To produce `rustc` that uses the GCC backend
+by default, you can thus put `"gcc"` as the first element of this array:
+
+```bash
+./x build --set 'rust.codegen-backends=["gcc"]' library
+```
+
+## Choosing the codegen backend used in tests
+
+To run compiler tests with the GCC codegen backend being used to build the test Rust programs, you can use the
+`--test-codegen-backend` flag:
+
+```bash
+./x test tests/ui --test-codegen-backend gcc
+```
+
+Note that in order for this to work, the tested compiler must have the GCC codegen backend available in its sysroot
+directory. You can achieve that using the [instructions above](#choosing-which-codegen-backends-are-built).
+
+## Downloading GCC from CI
+
+The `gcc.download-ci-gcc` bootstrap option controls if GCC (which is a dependency of the GCC codegen backend)
+will be downloaded from CI or built locally. The default value is `true`, which will download GCC from CI
+if there are no local changes to the GCC sources and the given host target is available on CI.
+
+Note that GCC can currently only be downloaded from CI for the `x86_64-unknown-linux-gnu` target.
diff --git a/src/tests/compiletest.md b/src/tests/compiletest.md
index a4a7299..0234b39 100644
--- a/src/tests/compiletest.md
+++ b/src/tests/compiletest.md
@@ -72,7 +72,7 @@
| [`mir-opt`](#mir-opt-tests) | Check MIR generation and optimizations |
| [`coverage`](#coverage-tests) | Check coverage instrumentation |
| [`coverage-run-rustdoc`](#coverage-tests) | `coverage` tests that also run instrumented doctests |
-| [`crashes`](#crashes-tests) | Check that the compiler ICEs/panics/crashes on certain inputs to catch accidental fixes |
+| [`crashes`](#crash-tests) | Check that the compiler ICEs/panics/crashes on certain inputs to catch accidental fixes |
### General purpose test suite
@@ -557,7 +557,7 @@
[`src/tools/coverage-dump`]: https://github.com/rust-lang/rust/tree/master/src/tools/coverage-dump
[`tests/coverage-run-rustdoc`]: https://github.com/rust-lang/rust/tree/master/tests/coverage-run-rustdoc
-### Crashes tests
+### Crash tests
[`tests/crashes`] serve as a collection of tests that are expected to cause the
compiler to ICE, panic or crash in some other way, so that accidental fixes are
@@ -580,13 +580,13 @@
When you do so, each issue number should be noted in the file name (`12345.rs`
should suffice) and also inside the file by means of a `//@ known-bug: #12345`
directive. Please [label][labeling] the relevant issues with `S-bug-has-test`
-afterwards.
+once your PR is merged.
If you happen to fix one of the crashes, please move it to a fitting
subdirectory in `tests/ui` and give it a meaningful name. Please add a doc
comment at the top of the file explaining why this test exists, even better if
you can briefly explain how the example causes rustc to crash previously and
-what was done to prevent rustc to ICE/panic/crash.
+what was done to prevent rustc to ICE / panic / crash.
Adding
diff --git a/src/tracing.md b/src/tracing.md
index 5e5b81f..a7cdab7 100644
--- a/src/tracing.md
+++ b/src/tracing.md
@@ -109,7 +109,7 @@
See the [`tracing`] crate's docs, and specifically the docs for [`debug!`] to
see the full syntax you can use. (Note: unlike the compiler, the [`tracing`]
-crate and its examples use the `RUST_LOG` environment variable. rustc, rustdoc,
+crate and its examples use the `RUSTC_LOG` environment variable. rustc, rustdoc,
and other tools set custom environment variables.)
**Note that unless you use a very strict filter, the logger will emit a lot of
