Fortran bindings for Enzyme

Source files in this subdirectory provides Fortran bindings for Enzyme, as detailed in the following.

The custom derivatives section registers reverse-mode and forward-mode rules through a derived type of Fortran procedure pointers.

Note on compilers

Before providing details on the Fortran bindings, it is worth noting that Enzyme only supports the 2023.0.0 and 2023.2.4 versions of the Intel IFX Fortran compiler. We strongly recommend using the Flang compiler, which is available as part of the LLVM project.

Running Enzyme from flang

Configuring Enzyme with -DENZYME_FLANG=ON builds FlangEnzyme-<LLVM version>, a pass plugin that flang can load with -fpass-plugin. Enzyme then runs as part of the flang optimization pipeline, so a single command differentiates and compiles:

$ flang -fpass-plugin=/path/to/FlangEnzyme-21.so -I /path/to/enzyme/modules program.f90 -o program

The -I flag points at the directory holding the enzyme.mod module file, which is built by -DENZYME_FORTRAN=ON (see the sections below).

The same build installs FlangEnzyme-<LLVM version>.cfg next to the plugin. Passing it with --config replaces the plugin flag and also covers link-time optimization:

$ flang --config=/path/to/FlangEnzyme-21.cfg -I /path/to/enzyme/modules -flto -c a.f90 b.f90
$ flang --config=/path/to/FlangEnzyme-21.cfg -flto a.o b.o -o program

Under -flto (full or thin) the compile step leaves __enzyme_autodiff calls alone, because their callee may be compiled in another file; lld differentiates the linked program instead. The config file selects lld and loads LLDEnzyme-<LLVM version> into it, so the same flags work with and without -flto. With -flto=thin, lld differentiates each module on its own, after ThinLTO imported functions into it. The compile step asks ThinLTO to import the function passed to __enzyme_autodiff and the functions it calls. ThinLTO still applies its size limit to functions further down the call graph, and never imports functions marked noinline (which includes everything compiled at -O0). When such a function is missing, Enzyme reports that it has no derivative for it; compile it in the same file as the call, raise the limit with -Wl,-mllvm,-import-instr-limit=<N>, or use full -flto.

Without the plugin the derivative has to be produced out of line, by emitting LLVM IR from flang and running the Enzyme pass over it with opt:

$ flang -flto -c -I /path/to/enzyme/modules program.f90 -o program.bc
$ opt -load-pass-plugin=/path/to/LLVMEnzyme-21.so \
    -passes='preserve-nvvm,enzyme,preserve-nvvm-end' program.bc -o program-enzyme.bc
$ flang -flto program-enzyme.bc -o program

preserve-nvvm consumes the registration markers described below (custom derivatives and enzyme_function_like procedure pointers) before enzyme differentiates, and preserve-nvvm-end restores the linkage and inlining attributes it changed. -passes=enzyme alone skips registration.

Both routes are exercised by the tests in enzyme/test/Fortran. The plugin route is flang-only; with ifx use the opt pipeline above.

Function hooks for differentiation

We provide bindings for the __enzyme_fwddiff and __enzyme_autodiff function hooks using implicit interfaces. Some Fortran compilers disallow procedure names starting with an underscore so we rename the function hooks to remove the leading double underscore.

To make use of the enzyme_autodiff function hook in your code, import via

use enzyme, only: enzyme_autodiff

and call it as a subroutine or function as appropriate. For example, if you have a function

  real function square(x)
    real, intent(in) :: x
    square = x**2
  end function

then you can compute its derivative with reverse mode with the call

  call enzyme_autodiff(square, x, dx)

Similarly for enzyme_fwddiff. Thanks to the implicit interface, arbitrary signatures are supported, with the following caveats.

[!NOTE] A limitation of the implicit interfacing is that it only works for arguments that are passed by reference - the default in Fortran. If you want to pass any arguments by value using the value attribute then you will need to write an explicit interface block to the function hook yourself.

[!WARNING] The implicit interfacing approach is not supported by the Intel Fortran compiler ifx when running without optimizations, i.e., running with -O0. If you want to use ifx with -O0 then you will need to write an explicit interface block, even if you are only passing arguments by reference.

[!WARNING] Differentiation with respect to procedures with assumed shape arrays is not currently supported when compiling with Flang. It should work with ifx, however.

Activity descriptors

We provide bindings for the activity descriptors enzyme_const, enzyme_dup, enzyme_dupnoneed, and enzyme_out, as well as the descriptors enzyme_scalar, enzyme_width, and enzyme_vector. To make use of these in your code, import via

use enzyme, only: enzyme_const, enzyme_dup

and then include them in calls to function hooks as you would in C or C++. For example, if you have a subroutine

  subroutine my_subroutine(n, x, y)
    integer, intent(in) :: n
    real, dimension(n), intent(in) :: x
    real, dimension(n), intent(out) :: y
    ! ...
  end subroutine my_subroutine

then you can make use of activity descriptors like so:

  call enzyme_autodiff(my_subroutine, enzyme_const, n, &
                       enzyme_dup, x, dx, enzyme_dup, y, dy)

Function hook for batching

We do not currently provide bindings for the __enzyme_batch function hook because it requires enzyme_width to be passed-by-value as an integer and this is not supported by the implicit interfacing approach used for the other function hooks. As such, you will need to write your own explicit interface block to handle the batching. See the Fortran batching test for an example.

[!NOTE] In C, the batched output is provided using a simple struct. The required syntax is different in Fortran - you should instead provide each entry of the output batch individually.

[!NOTE] You will likely find that batching works more straightforwardly with subroutines than with Fortran functions.

Function-like hooks

The enzyme_function_like hook tells Enzyme to differentiate a function as if it were a known mathematical function. For example, Enzyme can use the derivative of log1p for double_value, regardless of its implementation. The examples below deliberately compute 2*x while requesting the derivative of log1p: at x = 2, Enzyme returns 1/3 instead of 2. This illustrates a derivative override; the two functions are not mathematically equivalent.

Choose a registration form

Registration with a subroutine call

Call enzyme_function_like as a subroutine with the target function and the symbolic name of the mathematical function:

module enzyme_math_names
  use, intrinsic :: iso_c_binding, only: c_int
  implicit none
  private
  integer(c_int), public, bind(C, name="enzyme_math_log1p") :: enzyme_log1p
end module enzyme_math_names

Import the binding in the program or procedure that registers the target:

use enzyme, only: enzyme_function_like
use enzyme_math_names, only: enzyme_log1p

! Put this call after all declarations.
call enzyme_function_like(double_value, enzyme_log1p)

Put the registration call in executable code, after declarations. For an internal function, you must use this form instead of an initialized procedure pointer. An internal function follows contains inside a program or another procedure. The compiler supplies its explicit interface.

The call-style test shows this placement. Its registration call is in the main program. Its target function, double_value, is inside that program, after contains.

[!WARNING] Call registration does not support all internal functions. An internal function can access variables from its containing program or procedure. Fortran calls this access host association. For example, double_value could calculate factor * x, where factor is a variable in the containing procedure.

Flang can then generate an adapter that gives the function access to those variables. The current registration code requires a direct function reference. It cannot process this adapter, and compilation can fail with First argument of enzyme_function_like must be a constant function.

The example above uses only the argument x and does not need this adapter.

Here enzyme_log1p supplies the symbolic function name log1p; its value is not used. Functions passed to enzyme_function_like must have an LLVM-level signature compatible with the selected mathematical function. Scalar arguments must use the value attribute so that Flang lowers them as LLVM values rather than using Fortran's usual by-reference calling convention. This binding is currently supported with Flang.

When running Enzyme separately with opt, preserve-nvvm must process the enzyme_function_like hook before differentiation:

$ opt -load-pass-plugin=/path/to/LLVMEnzyme-21.so \
    -passes='preserve-nvvm,enzyme,preserve-nvvm-end' input.bc -o output.bc

[!WARNING] When using this separate opt workflow, compile the Fortran source to LLVM with -O0. Otherwise, Flang may inline calls to the function before preserve-nvvm processes the enzyme_function_like hook.

The FlangEnzyme compiler plugin runs preserve-nvvm at the start of Flang's LLVM optimization pipeline and does not require this separate opt step.

Use this hook to assign a mathematical rule to a custom function.

For call registration, declare the required symbolic names in a user module. Use integer(c_int) with bind(C, name="enzyme_math_<function>"). Replace <function> with a mathematical rule name that Enzyme supports. The variable value is not used. The binding name selects the rule.

For example, declare a binding for the sin rule:

module enzyme_math_names
  use, intrinsic :: iso_c_binding, only: c_int
  implicit none
  private

  integer(c_int), public, bind(C, name="enzyme_math_sin") :: enzyme_sin
end module enzyme_math_names

Import this binding where the registration call occurs:

use enzyme, only: enzyme_function_like
use enzyme_math_names, only: enzyme_sin

! Put this call after all declarations.
call enzyme_function_like(function_similar_to_sin, enzyme_sin)

Procedure-pointer registration

Alternatively, a statically initialized procedure pointer can register the same relationship without a hook call or symbolic-name binding. Enzyme reads and removes the registration marker at compile time. Do not call through the registration pointer. Enzyme replaces remaining references to the marker with null pointers. Call the target function directly, for example, double_value(x). Use the same FlangEnzyme plugin or separate opt pipeline described above for call-style registration.

Register a module function

Put the pointer declaration in the declaration section of a module, program, function, or subroutine where the module function is accessible. Put it before executable statements or contains. Omit private when the declaration is outside a module‘s declaration section. The example below puts the declaration before the module’s contains statement.

module function_like_example
  implicit none

  procedure(double_value), pointer, private :: &
    fn__enzyme_function_like__log1p => double_value

contains

  function double_value(x) result(y)
    real, value :: x
    real :: y

    y = 2.0 * x
  end function double_value

  function test(x) result(y)
    real, intent(in) :: x
    real :: y

    y = double_value(x)
  end function test

end module function_like_example

program main
  use enzyme, only: enzyme_autodiff
  use function_like_example, only: test
  implicit none
  real :: x, dx

  x = 2.0
  dx = 0.0
  call enzyme_autodiff(test, x, dx)
  write(*,"(f6.4)") dx ! Prints 0.3333
end program main

Here, procedure(double_value) gives the pointer the target‘s interface, and => double_value initializes it with the target. PreserveNVVM reads the mathematical name after the exact __enzyme_function_like__ delimiter, so this example registers the target as log1p. The prefix before the delimiter can be any valid name but must be unique in its scope. private is optional in a module; it keeps the registration marker out of the module’s public API.

The test wrapper takes its argument by reference for the enzyme_autodiff binding, while double_value takes its argument by value to match the scalar log1p rule.

Custom derivatives

Enzyme can use custom derivative routines written in Fortran, in reverse mode and in forward mode. Register them with one module variable of a derived type whose components are procedure pointers initialized to the routines. No C registration file is needed. The reverse-mode walkthrough below uses three components; forward mode uses two.

The complete example calculates 2*x but uses the derivative of log1p(x), the natural logarithm of 1+x. This deliberate difference shows that Enzyme uses the custom rule. At x = 2:

1. Define these three routines in one module.

   Routine	Operation
   double_value(x, y)	Calculates y = 2*x.
   augment_double_value(x, dx, y, dy)	Calls double_value. Leaves dy unchanged.
   reverse_double_value(x, dx, y, dy)	Adds the custom derivative to dx. Clears dy.

   The augmented forward routine calculates the original result. It can also save values for the reverse routine. This is the forward part of reverse-mode differentiation, not forward-mode differentiation.

   Each original argument is followed by its derivative storage: x, dx, y, dy. Keep all arguments, including unused arguments. This example uses scalars passed by reference. It needs no tape, which is storage for intermediate values from the forward calculation. Other interfaces must also match Enzyme's custom derivative requirements.

   The reverse routine contains:

   dx = dx + dy / (1.0 + x)
   dy = 0.0
   

   Enzyme supplies dy, the derivative of the final result with respect to y. The first statement adds the contribution for x to dx. Clearing dy prevents an earlier write to y from adding the contribution again. The forward routine must preserve dy because Enzyme can supply it before that call.

2. Put this declaration before contains in the same module.

   type :: gradient_registration
     procedure(double_value), pointer, nopass :: primal => double_value
     procedure(augment_double_value), pointer, nopass :: augmented => augment_double_value
     procedure(reverse_double_value), pointer, nopass :: reverse => reverse_double_value
   end type gradient_registration
   
   type(gradient_registration) :: f__enzyme_register_gradient_double_value
   

   Keep the pointer order: original routine, forward routine, reverse routine. procedure(...) specifies the interface; => sets the default target. nopass prevents Fortran from passing the record as an additional argument.

   Give each registration variable a unique name containing __enzyme_register_gradient. The leading f makes this example a valid Fortran identifier. Enzyme reads the static initializer during compilation, then removes the registration variable. Therefore, do not access the registration variable during program execution. Do not replace the initializers with assignments during program execution.

3. Differentiate a function that calls the original routine.

   In the example, wrapper calls double_value and returns y.

   x = 2.0
   dx = 0.0
   call enzyme_autodiff(wrapper, x, dx)
   

   Initialize dx to zero for a fresh derivative calculation. Use an existing derivative value only when you want to accumulate contributions. Enzyme starts the derivative of the scalar result at one. Thus, the custom rule adds 1/(1+x) to dx.

4. Compile and run the complete example.

   Keep all three routines and the registration in one source file for this example. Use a plugin built for your Flang and LLVM version. Replace the include path with the directory containing enzyme.mod. Replace the plugin path with your plugin‘s path. Run these commands from the example’s directory:

   flang -O2 -I /path/to/enzyme/modules \
     -fpass-plugin=/path/to/FlangEnzyme-21.so \
     custom_derivative.f90 -o custom_derivative
   ./custom_derivative
   

   The plugin registers the custom rule before optimization. Expected output:

   y = 4.0000
   dx = 0.3333
   

Alternatively, use the opt route from Running Enzyme from flang with matching Flang, LLVM, and Enzyme versions. Replace the paths as above:

flang -O0 -flto -c -I /path/to/enzyme/modules \
  custom_derivative.f90 -o custom_derivative.bc
opt -load-pass-plugin=/path/to/LLVMEnzyme-21.so \
  -passes='preserve-nvvm,enzyme,preserve-nvvm-end' \
  custom_derivative.bc -o custom_derivative-enzyme.bc
flang -O2 -flto custom_derivative-enzyme.bc -o custom_derivative
./custom_derivative

Use -O0 before opt to prevent optimization from removing calls before registration. The passes register the custom routines, differentiate the code, and restore temporary linkage and inlining attributes. The final compilation can use -O2.

The regression test checks registration, custom routine calls, and derivative accumulation when two calls overwrite the same output.

Forward-mode custom derivatives

Forward mode uses the same mechanism with two components, the original routine and the forward-mode routine, and a registration variable whose name contains __enzyme_register_derivative:

type :: derivative_registration
  procedure(double_value), pointer, nopass :: primal => double_value
  procedure(forward_double_value), pointer, nopass :: forward => forward_double_value
end type derivative_registration

type(derivative_registration) :: f__enzyme_register_derivative_double_value

The forward routine takes each original argument followed by its tangent, x, dx, y, dy. It calculates the original result and sets dy from dx:

subroutine forward_double_value(x, dx, y, dy)
  real, intent(in) :: x, dx
  real, intent(out) :: y, dy

  call double_value(x, y)
  dy = dx / (1.0 + x)
end subroutine forward_double_value

Unlike the reverse routine, the forward routine overwrites dy instead of adding to it. Differentiate a subroutine wrapper(x, y) that calls double_value with enzyme_fwddiff:

x = 2.0
dx = 1.0
call enzyme_fwddiff(wrapper, enzyme_dup, x, dx, enzyme_dup, y, dy)

The forward-mode example prints dy = 0.3333. The seed test checks that dy scales with dx and that the rule replaces a previous value of dy.