From 064707ff2d68d71d8a19c7208adb5f3bc7038109 Mon Sep 17 00:00:00 2001 From: Lukas Larsson Date: Mon, 17 Apr 2017 16:26:31 +0200 Subject: [PATCH] Expand text on beam_loader with syntax description (#62) * Expand text on beam_loader with syntax description * Add a general description about how ops.tab does it transformation rules * Change listings to be names tags * beam_loader: Add a more complex transformation example * beam_loader: Add details about %macro flags * beam_loader: Fix review comments * beam_loader: Add links to various parts of Erlang/OTP source * fixup! beam_loader: Fix review comments --- chapters/beam_loader.asciidoc | 238 ++++++++++++++++++++++++++++++++++ 1 file changed, 238 insertions(+) diff --git a/chapters/beam_loader.asciidoc b/chapters/beam_loader.asciidoc index ae11e8a..bf0c222 100644 --- a/chapters/beam_loader.asciidoc +++ b/chapters/beam_loader.asciidoc @@ -49,3 +49,241 @@ e.g. +erts/emulator/x86_64-unknown-linux-gnu/opt/smp/+). The same program (beam_makeops) also generates the Erlang code for the compiler back end +beam_opcodes.erl+. + +=== Understanding ops.tab + +// Missing description about build specific options, i.e. the %unless construct + +The transformations in +https://github.com/erlang/otp/blob/OTP-19.3/erts/emulator/beam/ops.tab[`ops.tab`] +are executed in the order that they are written in the file. So just like in +Erlang pattern matching, the different rules are triggered from top to bottom. + +The types that `ops.tab` uses for arguments in instructions can be found in +xref:AP-Instructions[]. + +==== Transformations + +// This section should be expanded with explanations about +// what goes on when a more complex tranformation is done. +// * bif specific transformations should be mentioned. + +Most of the rules in `ops.tab` are transformations between different +instructions. A simple transformation looks like this: + +.... +move S x==0 | return => move_return S +.... + +This combines a `move` from any location to `x(0)` and return into a single +instruction called `move_return`. Let's break the tranformation apart to +see what the different parts do. + +move:: is the instruction that the pattern first has to match. This can be either +a generic instruction that the compiler has emitted, or a temporary instruction +that `ops.tab` has emitted to help with transformations. + +S:: is a variable binding any type of value. Any value in the pattern (left hand side or `=>`) +that is used in the generator (right hand side of `=>`) has to be bound to a variable. + +x==0:: is a guard that says that we only apply the transformation if the target +location is an `x` register with the value `0`. It is possible to chain multiple +types and also bind a variable here. For instance `D=xy==0` would allow both +`x` and `y` registers with a value of `0` and also bind the argument to the variable `D`. + +|:: signifies the end of this instruction and the beginning of another instruction +that is part of the same pattern. + +return:: is the second instruction to match in this pattern. + +`=>`:: signifies the end of the pattern and the start of the code that is to be +generated. + +move_return S:: is the name of the generated instruction together with the name of +the variable on the lhs. It is possible to generate +https://github.com/erlang/otp/blob/OTP-19.3/erts/emulator/beam/ops.tab#L625[multiple instructions] +as part of a transformation by using the `|` symbol. + +[[complex_example]] +.A more complex example + +More complex translations can be done in `ops.tab`. For instance take the +https://github.com/erlang/otp/blob/OTP-19.3/erts/emulator/beam/ops.tab#L127-L182[`select_val`] +instruction. It will be translated by the loader into either a jump table, a linear +search array or a binary search array depending on the input values. + +.... +is_integer Fail=f S | select_val S=s Fail=f Size=u Rest=* | \ + use_jump_tab(Size, Rest) => gen_jump_tab(S, Fail, Size, Rest) +.... + +The above transformation creates a jump table if possible of the `select_val`. +There are a bunch of new techniques used in the transformations. + +S:: is used in both `is_integer` and `select_val`. This means that both the +values have to be of the same type and have the same value. Futhermore the `S=s` guard +limits the type to a be a source register. +Rest=*:: allows a variable number of arguments in the instruction and binds them to +variable `Rest`. +use_jump_tab(Size, Rest):: calls a the +https://github.com/erlang/otp/blob/OTP-19.3/erts/emulator/beam/beam_load.c#L2707[use_jump_tab] +C function in `beam_load.c` that decides whether the arguments in the `select_val` +can be transformed into a jump table. +\:: signifies that the transformation rule continues on the next line. +gen_jump_tab(S, Fail, Size, Rest):: calls the +https://github.com/erlang/otp/blob/OTP-19.3/erts/emulator/beam/beam_load.c#L3692[gen_jump_tab] +C function in `beam_load.c` that takes care of generating the approprioate instruction. + +==== Specific instruction + +When all transformations are done, we have to decide how the specifc instruction should +look like. Let's continue to look at `move_return`: + +.... +%macro: move_return MoveReturn -nonext +move_return x +move_return c +move_return n +.... + +This will generate three different instructions that will use the +https://github.com/erlang/otp/blob/OTP-19.3/erts/emulator/beam/beam_emu.c#L636[`MoveReturn`] +macro in `beam_emu.c` to do the work. + +%macro: move_return:: this tells `ops.tab` to generate the code for `move_return`. If there +is no `%macro` line, the instruction has to be implemented by hand in beam_emu.c. The code +for the instruction will be places in `beam_hot.h` or `beam_cold.h` depending on if the +`%hot` or `%cold` directive is active. + +MoveReturn:: tells the code generator to that the name of the c-macro in beam_emu.c to use +is MoveReturn. This macro has to be implemented manually. + +-nonext:: tells the code generator that it should not generate a dispatch to the next +instruction, the `MoveReturn` macro will take care of that. + +move_return x:: tells the code generator to generate a specific instruction for when the +instruction argument is an x register. `c` for when it is a constant, `n` when it is `NIL`. +No instructions are in this case generated for when the argument is a y register as the +compiler will never generate such code. + +The resulting code in `beam_hot.h` will look like this: + +[source, C] +----------------------------- +OpCase(move_return_c): + { + MoveReturn(Arg(0)); + } + +OpCase(move_return_n): + { + MoveReturn(NIL); + } + +OpCase(move_return_x): + { + MoveReturn(xb(Arg(0))); + } +----------------------------- + +All the implementor has to do is to define the `MoveReturn` macro in `beam_emu.c` and +the instruction is complete. + +[[macro_arguments]] +.Macro flags + +The `%macro` rules can take multiple different flags to modify the code that +gets generated. + +The examples below assume that there is a specific instructions looking like this: + +.... +%macro move_call MoveCall +move_call x f +.... + +without any flags to the `%macro` we the following code will be generated: + +[source, C] +BeamInstr* next; +PreFetch(2, next); +MoveCall(Arg(0)); +NextPF(2, next); + +-nonext:: Don't emit a dispatch for this instructions. This is used for instructions +that are known to not continue with the next instructions, i.e. return, call, jump. + +`%macro move_call MoveCall -nonext` +[source, C] +MoveCall(xb(Arg(0))); + +-arg_*:: Include the arguments of type * as arguments to the c-macro. Not all argument +types are included by default in the c-macro. For instance the type `f` used for fail +labels and local function calls is not included. So giving the option `-arg_f` will +include that as an argument to the c-macro. + +`%macro move_call MoveCall -arg_f` +[source, C] +MoveCall(xb(Arg(0)), Arg(1)); + +-size:: Include the size of the instruction as an argument to the c-macro. + +`%macro move_call MoveCall -size` +[source, C] +MoveCall(xb(Arg(0)), 2); + +-pack:: Pack any arguments if possible. This places multiple register arguments in +the same word if possible. As register arguments can only be 0-1024, we only need +10 bits to store them + 2 for tagging. So on a 32-bit system we can put 2 registers +in one word, while on a 64-bit we can put 4 registers in one word. Packing instruction +can greatly decrease the memory used for a single instruction. However there is +also a small cost to unpack the instruction, which is why it is not enabled +for all instructions. + +The example with the call cannot do any packing as `f` cannot be packed and only one +other argument exists. So let's look at the +https://github.com/erlang/otp/blob/OTP-19.3/erts/emulator/beam/ops.tab#L539[put_list] +instruction as an example instead. + +.... +%macro:put_list PutList -pack +put_list x x x +.... + +[source, C] +BeamInstr tmp_packed1; +BeamInstr* next; +PreFetch(1, next); +tmp_packed1 = Arg(0); +PutList(xb(tmp_packed1&BEAM_TIGHT_MASK), + xb((tmp_packed1>>BEAM_TIGHT_SHIFT)&BEAM_TIGHT_MASK), + xb((tmp_packed1>>(2*BEAM_TIGHT_SHIFT)))); +NextPF(1, next); + +This packs the 3 arguments into 1 machine word, which halves the required memory +for this instruction. + +-fail_action:: Include a fail action as an argument to the c-macro. Note that the +https://github.com/erlang/otp/blob/OTP-19.3/erts/emulator/beam/beam_emu.c#L2996-L2998[`ClauseFail()`] +macro assumes the fail label is in the first argument of the instructions, +so in order to use this in the above example we should transform +the `move_call x f` to `move_call f x`. + +`%macro move_call MoveCall -fail_action` +[source, C] +MoveCall(xb(Arg(0)), ClauseFail()); + +-gen_dest:: Include a +https://github.com/erlang/otp/blob/OTP-19.3/erts/emulator/beam/beam_emu.c#L166-L174[store function] +as an argument to the c-macro. + +`%macro move_call MoveCall -gen_dest` +[source, C] +MoveCall(xb(Arg(0)), StoreSimpleDest); + +-goto:: Replace the normal next dispatch with a jump to a c-label inside beam_emu.c + +`%macro move_call MoveCall -goto:do_call` +[source, C] +MoveCall(xb(Arg(0))); +goto do_call;