Date: March 23, 2020
: Original whitepaper, PDF
Abstract
The aim of this text is to provide a description of the Telegram Open Network Virtual Machine (TON VM or TVM), used to execute smart contracts in the TON Blockchain.Introduction
The primary purpose of the Telegram Open Network Virtual Machine (TON VM or TVM) is to execute smart-contract code in the TON Blockchain. TVM must support all operations required to parse incoming messages and persistent data, and to create new messages and modify persistent data. Additionally, TVM must meet the following requirements:- It must provide for possible future extensions and improvements while retaining backward compatibility and interoperability, because the code of a smart contract, once committed into the blockchain, must continue working in a predictable manner regardless of any future modifications to the VM.
- It must strive to attain high “(virtual) machine code” density, so that the code of a typical smart contract occupies as little persistent blockchain storage as possible.
- It must be completely deterministic. In other words, each run of the same code with the same input data must produce the same result, regardless of specific software and hardware used.1
1 Overview
This chapter provides an overview of the main features and design principles of TVM. More detail on each topic is provided in subsequent chapters.1.0 Notation for bitstrings
The following notation is used for bit strings (or bitstrings)—i.e., finite strings consisting of binary digits (bits), and —throughout this document.1.0.1. Hexadecimal notation for bitstrings
When the length of a bitstring is a multiple of four, we subdivide it into groups of four bits and represent each group by one of sixteen hexadecimal digits —, — in the usual manner: , , , . The resulting hexadecimal string is our equivalent representation for the original binary string.1.0.2. Bitstrings of lengths not divisible by four
If the length of a binary string is not divisible by four, we augment it by one and several (maybe zero) s at the end, so that its length becomes divisible by four, and then transform it into a string of hexadecimal digits as described above. To indicate that such a transformation has taken place, a special “completion tag” is added to the end of the hexadecimal string. The reverse transformation (applied if the completion tag is present) consists in first replacing each hexadecimal digit by four corresponding bits, and then removing all trailing zeroes (if any) and the last immediately preceding them (if the resulting bitstring is non-empty at this point). Notice that there are several admissible hexadecimal representations for the same bitstring. Among them, the shortest one is “canonical”. It can be deterministically obtained by the above procedure. For example, corresponds to binary string , while and both correspond to . An empty bitstring may be represented by either ’ ’, '', '', '', or ''.1.0.3. Emphasizing that a string is a hexadecimal representation of a bitstring
Sometimes we need to emphasize that a string of hexadecimal digits (with or without a at the end) is the hexadecimal representation of a bitstring. In such cases, we either prepend to the resulting string (e.g., ), or prepend and append (e.g., , which is ). This should not be confused with hexadecimal numbers, usually prepended by (e.g., or , which is the integer 729).1.0.4. Serializing a bitstring into a sequence of octets
When a bitstring needs to be represented as a sequence of 8-bit bytes (octets), which take values in integers , this is achieved essentially in the same fashion as above: we split the bitstring into groups of eight bits and interpret each group as the binary representation of an integer . If the length of the bitstring is not a multiple of eight, the bitstring is augmented by a binary and up to seven binary s before being split into groups. The fact that such a completion has been applied is usually reflected by a “completion tag” bit. For instance, corresponds to the sequence of two octets (hexadecimal), or (decimal), along with a completion tag bit equal to (meaning that the completion has been applied), which must be stored separately. In some cases, it is more convenient to assume the completion is enabled by default rather than store an additional completion tag bit separately. Under such conventions, -bit strings are represented by octets, with the last octet always equal to .1.1 TVM is a stack machine
First of all, TVM is a stack machine. This means that, instead of keeping values in some “variables” or “general-purpose registers”, they are kept in a (LIFO) stack, at least from the “low-level” (TVM) perspective.3 Most operations and user-defined functions take their arguments from the top of the stack, and replace them with their result. For example, the integer addition primitive (built-in operation) does not take any arguments describing which registers or immediate values should be added together and where the result should be stored. Instead, the two top values are taken from the stack, they are added together, and their sum is pushed into the stack in their place.1.1.1. TVM values
The entities that can be stored in the TVM stack will be called TVM values, or simply values for brevity. They belong to one of several predefined value types. Each value belongs to exactly one value type. The values are always kept on the stack along with tags uniquely determining their types, and all built-in TVM operations (or primitives) only accept values of predefined types. For example, the integer addition primitive accepts only two integer values, and returns one integer value as a result. One cannot supply with two strings instead of two integers expecting it to concatenate these strings or to implicitly transform the strings into their decimal integer values; any attempt to do so will result in a run-time type-checking exception.1.1.2. Static typing, dynamic typing, and run-time type checking
In some respects TVM performs a kind of dynamic typing using run-time type checking. However, this does not make the TVM code a “dynamically typed language” like PHP or Javascript, because all primitives accept values and return results of predefined (value) types, each value belongs to strictly one type, and values are never implicitly converted from one type to another. If, on the other hand, one compares the TVM code to the conventional microprocessor machine code, one sees that the TVM mechanism of value tagging prevents, for example, using the address of a string as a number—or, potentially even more disastrously, using a number as the address of a string—thus eliminating the possibility of all sorts of bugs and security vulnerabilities related to invalid memory accesses, usually leading to memory corruption and segmentation faults. This property is highly desirable for a VM used to execute smart contracts in a blockchain. In this respect, TVM’s insistence on tagging all values with their appropriate types, instead of reinterpreting the bit sequence in a register depending on the needs of the operation it is used in, is just an additional run-time type-safety mechanism. An alternative would be to somehow analyze the smart-contract code for type correctness and type safety before allowing its execution in the VM, or even before allowing it to be uploaded into the blockchain as the code of a smart contract. Such a static analysis of code for a Turing-complete machine appears to be a time-consuming and non-trivial problem (likely to be equivalent to the stopping problem for Turing machines), something we would rather avoid in a blockchain smart-contract context. One should bear in mind that one always can implement compilers from statically typed high-level smart-contract languages into the TVM code (and we do expect that most smart contracts for TON will be written in such languages), just as one can compile statically typed languages into conventional machine code (e.g., x86 architecture). If the compiler works correctly, the resulting machine code will never generate any run-time type-checking exceptions. All type tags attached to values processed by TVM will always have expected values and may be safely ignored during the analysis of the resulting TVM code, apart from the fact that the run-time generation and verification of these type tags by TVM will slightly slow down the execution of the TVM code.1.1.3. Preliminary list of value types
A preliminary list of value types supported by TVM is as follows:- Integer — Signed 257-bit integers, representing integer numbers in the range , as well as a special “not-a-number” value .
- Cell — A TVM cell consists of at most 1023 bits of data, and of at most four references to other cells. All persistent data (including TVM code) in the TON Blockchain is represented as a collection of TVM cells (cf. [1], 2.5.14).
- Tuple — An ordered collection of up to 255 components, having arbitrary value types, possibly distinct. May be used to represent non-persistent values of arbitrary algebraic data types.
- Null — A type with exactly one value , used for representing empty lists, empty branches of binary trees, absence of return value in some situations, and so on.
- Slice — A TVM cell slice, or slice for short, is a contiguous “sub-cell” of an existing cell, containing some of its bits of data and some of its references. Essentially, a slice is a read-only view for a subcell of a cell. Slices are used for unpacking data previously stored (or serialized) in a cell or a tree of cells.
- Builder — A TVM cell builder, or builder for short, is an “incomplete” cell that supports fast operations of appending bitstrings and cell references at its end. Builders are used for packing (or serializing) data from the top of the stack into new cells (e.g., before transferring them to persistent storage).
- Continuation — Represents an “execution token” for TVM, which may be invoked (executed) later. As such, it generalizes function addresses (i.e., function pointers and references), subroutine return addresses, instruction pointer addresses, exception handler addresses, closures, partial applications, anonymous functions, and so on.
1.2 Categories of TVM instructions
TVM instructions, also called primitives and sometimes (built-in) operations, are the smallest operations atomically performed by TVM that can be present in the TVM code. They fall into several categories, depending on the types of values (cf. 1.1.3) they work on. The most important of these categories are:- Stack (manipulation) primitives — Rearrange data in the TVM stack, so that the other primitives and user-defined functions can later be called with correct arguments. Unlike most other primitives, they are polymorphic, i.e., work with values of arbitrary types.
- Tuple (manipulation) primitives — Construct, modify, and decompose Tuples. Similarly to the stack primitives, they are polymorphic.
- Constant or literal primitives — Push into the stack some “constant” or “literal” values embedded into the TVM code itself, thus providing arguments to the other primitives. They are somewhat similar to stack primitives, but are less generic because they work with values of specific types.
- Arithmetic primitives — Perform the usual integer arithmetic operations on values of type Integer.
- Cell (manipulation) primitives — Create new cells and store data in them (cell creation primitives) or read data from previously created cells (cell parsing primitives). Because all memory and persistent storage of TVM consists of cells, these cell manipulation primitives actually correspond to “memory access instructions” of other architectures. Cell creation primitives usually work with values of type Builder, while cell parsing primitives work with Slices.
- Continuation and control flow primitives — Create and modify Continuations, as well as execute existing Continuations in different ways, including conditional and repeated execution.
- Custom or application-specific primitives — Efficiently perform specific high-level actions required by the application (in our case, the TON Blockchain), such as computing hash functions, performing elliptic curve cryptography, sending new blockchain messages, creating new smart contracts, and so on. These primitives correspond to standard library functions rather than microprocessor instructions.
1.3 Control registers
While TVM is a stack machine, some rarely changed values needed in almost all functions are better passed in certain special registers, and not near the top of the stack. Otherwise, a prohibitive number of stack reordering operations would be required to manage all these values. To this end, the TVM model includes, apart from the stack, up to 16 special control registers, denoted by to , or to . The original version of TVM makes use of only some of these registers; the rest may be supported later.1.3.1. Values kept in control registers
The values kept in control registers are of the same types as those kept on the stack. However, some control registers accept only values of specific types, and any attempt to load a value of a different type will lead to an exception.1.3.2. List of control registers
The original version of TVM defines and uses the following control registers:- — Contains the next continuation or return continuation (similar to the subroutine return address in conventional designs). This value must be a Continuation.
- — Contains the alternative (return) continuation; this value must be a Continuation. It is used in some (experimental) control flow primitives, allowing TVM to define and call “subroutines with two exit points”.
- — Contains the exception handler. This value is a Continuation, invoked whenever an exception is triggered.
- — Contains the current dictionary, essentially a hashmap containing the code of all functions used in the program. For reasons explained later in 4.6, this value is also a Continuation, not a Cell as one might expect.
- — Contains the root of persistent data, or simply the data. This value is a Cell. When the code of a smart contract is invoked, points to the root cell of its persistent data kept in the blockchain state. If the smart contract needs to modify this data, it changes before returning.
- — Contains the output actions. It is also a Cell initialized by a reference to an empty cell, but its final value is considered one of the smart contract outputs. For instance, the primitive, specific for the TON Blockchain, simply inserts the message into a list stored in the output actions.
- — Contains the root of temporary data. It is a Tuple, initialized by a reference to an empty Tuple before invoking the smart contract and discarded after its termination.4
1.4 Total state of TVM (SCCCG)
The total state of TVM consists of the following components:- Stack (cf. 1.1) — Contains zero or more values (cf. 1.1.1), each belonging to one of value types listed in 1.1.3.
- Control registers – — Contain some specific values as described in 1.3.2. (Only seven control registers are used in the current version.)
- Current continuation — Contains the current continuation (i.e., the code that would be normally executed after the current primitive is completed). This component is similar to the instruction pointer register () in other architectures.
- Current codepage — A special signed 16-bit integer value that selects the way the next TVM opcode will be decoded. For example, future versions of TVM might use different codepages to add new opcodes while preserving backward compatibility.
- Gas limits — Contains four signed 64-bit integers: the current gas limit , the maximal gas limit , the remaining gas , and the gas credit . Always , , and ; is usually initialized by zero, is initialized by and gradually decreases as the TVM runs. When becomes negative or if the final value of is less than , an out of gas exception is triggered.
1.5 Integer arithmetic
All arithmetic primitives of TVM operate on several arguments of type Integer, taken from the top of the stack, and return their results, of the same type, into the stack. Recall that Integer represents all integer values in the range , and additionally contains a special value (“not-a-number”). If one of the results does not fit into the supported range of integers—or if one of the arguments is a —then this result or all of the results are replaced by a , and (by default) an integer overflow exception is generated. However, special “quiet” versions of arithmetic operations will simply produce s and keep going. If these s end up being used in a “non-quiet” arithmetic operation, or in a non-arithmetic operation, an integer overflow exception will occur.1.5.1. Absence of automatic conversion of integers
Notice that TVM Integers are “mathematical” integers, and not 257-bit strings interpreted differently depending on the primitive used, as is common for other machine code designs. For example, TVM has only one multiplication primitive , rather than two ( for unsigned multiplication and for signed multiplication) as occurs, for example, in the popular x86 architecture.1.5.2. Automatic overflow checks
Notice that all TVM arithmetic primitives perform overflow checks of the results. If a result does not fit into the Integer type, it is replaced by a , and (usually) an exception occurs. In particular, the result is not automatically reduced modulo or , as is common for most hardware machine code architectures.1.5.3. Custom overflow checks
In addition to automatic overflow checks, TVM includes custom overflow checks, performed by primitives and , where . These primitives check whether the value on (the top of) the stack is an integer in the range or , respectively, and replace the value with a and (optionally) generate an integer overflow exception if this is not the case. This greatly simplifies the implementation of arbitrary -bit integer types, signed or unsigned: the programmer or the compiler must insert appropriate or primitives either after each arithmetic operation (which is more reasonable, but requires more checks) or before storing computed values and returning them from functions. This is important for smart contracts, where unexpected integer overflows happen to be among the most common source of bugs.1.5.4. Reduction modulo
TVM also has a primitive , which reduces the integer at the top of the stack modulo , with the result ranging from to .1.5.5. Integer is 257-bit, not 256-bit
One can understand now why TVM’s Integer is (signed) 257-bit, not 256-bit. The reason is that it is the smallest integer type containing both signed 256-bit integers and unsigned 256-bit integers, which does not require automatic reinterpreting of the same 256-bit string depending on the operation used (cf. 1.5.1).1.5.6. Division and rounding
The most important division primitives are , , and . All of them take two numbers from the stack, and ( is taken from the top of the stack, and is originally under it), compute the quotient and remainder of the division of by (i.e., two integers such that and ), and return either , , or both of them. If is zero, then all of the expected results are replaced by s, and (usually) an integer overflow exception is generated. The implementation of division in TVM somewhat differs from most other implementations with regards to rounding. By default, these primitives round to , meaning that , and has the same sign as . (Most conventional implementations of division use “rounding to zero” instead, meaning that has the same sign as .) Apart from this “floor rounding”, two other rounding modes are available, called “ceiling rounding” (with , and and having opposite signs) and “nearest rounding” (with and ). These rounding modes are selected by using other division primitives, with letters and appended to their mnemonics. For example, computes both the quotient and the remainder using rounding to the nearest integer.1.5.7. Combined multiply-divide, multiply-shift, and shift-divide operations
To simplify implementation of fixed-point arithmetic, TVM supports combined multiply-divide, multiply-shift, and shift-divide operations with double-length (i.e., 514-bit) intermediate product. For example, takes three integer arguments from the stack, , , and , first computes using a 514-bit intermediate result, and then divides by using rounding to the nearest integer. If is zero or if the quotient does not fit into Integer, either two s are returned, or an integer overflow exception is generated, depending on whether a quiet version of the operation has been used. Otherwise, both the quotient and the remainder are pushed into the stack.2 The stack
This chapter contains a general discussion and comparison of register and stack machines, expanded further in Appendix C, and describes the two main classes of stack manipulation primitives employed by TVM: the basic and the compound stack manipulation primitives. An informal explanation of their sufficiency for all stack reordering required for correctly invoking other primitives and user-defined functions is also provided. Finally, the problem of efficiently implementing TVM stack manipulation primitives is discussed in 2.3.2.1 Stack calling conventions
A stack machine, such as TVM, uses the stack—and especially the values near the top of the stack—to pass arguments to called functions and primitives (such as built-in arithmetic operations) and receive their results. This section discusses the TVM stack calling conventions, introduces some notation, and compares TVM stack calling conventions with those of certain register machines.2.1.1. Notation for “stack registers”
Recall that a stack machine, as opposed to a more conventional register machine, lacks general-purpose registers. However, one can treat the values near the top of the stack as a kind of “stack registers”. We denote by or the value at the top of the stack, by or the value immediately under it, and so on. The total number of values in the stack is called its depth. If the depth of the stack is , then , , , are well-defined, while and all subsequent with are not. Any attempt to use with should produce a stack underflow exception. A compiler, or a human programmer in “TVM code”, would use these “stack registers” to hold all declared variables and intermediate values, similarly to the way general-purpose registers are used on a register machine.2.1.2. Pushing and popping values
When a value is pushed into a stack of depth , it becomes the new ; at the same time, the old becomes the new , the old —the new , and so on. The depth of the resulting stack is . Similarly, when a value is popped from a stack of depth , it is the old value of (i.e., the old value at the top of the stack). After this, it is removed from the stack, and the old becomes the new (the new value at the top of the stack), the old becomes the new , and so on. The depth of the resulting stack is . If originally , then the stack is empty, and a value cannot be popped from it. If a primitive attempts to pop a value from an empty stack, a stack underflow exception occurs.2.1.3. Notation for hypothetical general-purpose registers
In order to compare stack machines with sufficiently general register machines, we will denote the general-purpose registers of a register machine by , , and so on, or by , , , , where is the total number of registers. When we need a specific value of , we will use , corresponding to the very popular x86-64 architecture.2.1.4. The top-of-stack register vs. the accumulator register
Some register machine architectures require one of the arguments for most arithmetic and logical operations to reside in a special register called the accumulator. In our comparison, we will assume that the accumulator is the general-purpose register ; otherwise we could simply renumber the registers. In this respect, the accumulator is somewhat similar to the top-of-stack “register” of a stack machine, because virtually all operations of a stack machine both use as one of their arguments and return their result as .2.1.5. Register calling conventions
When compiled for a register machine, high-level language functions usually receive their arguments in certain registers in a predefined order. If there are too many arguments, these functions take the remainder from the stack (yes, a register machine usually has a stack, too!). Some register calling conventions pass no arguments in registers at all, however, and only use the stack (for example, the original calling conventions used in implementations of Pascal and C, although modern implementations of C use some registers as well). For simplicity, we will assume that up to function arguments are passed in registers, and that these registers are , , , , in that order (if some other registers are used, we can simply renumber them).62.1.6. Order of function arguments
If a function or primitive requires arguments , , , they are pushed by the caller into the stack in the same order, starting from . Therefore, when the function or primitive is invoked, its first argument is in , its second argument is in , and so on. The last argument is in (i.e., at the top of the stack). It is the called function or primitive’s responsibility to remove its arguments from the stack. In this respect the TVM stack calling conventions—obeyed, at least, by TVM primitives—match those of Pascal and Forth, and are the opposite of those of C (in which the arguments are pushed into the stack in the reverse order, and are removed by the caller after it regains control, not the callee). Of course, an implementation of a high-level language for TVM might choose some other calling conventions for its functions, different from the default ones. This might be useful for certain functions—for instance, if the total number of arguments depends on the value of the first argument, as happens for “variadic functions” such as and . In such cases, the first one or several arguments are better passed near the top of the stack, not somewhere at some unknown location deep in the stack.2.1.7. Arguments to arithmetic primitives on register machines
On a stack machine, built-in arithmetic primitives (such as or ) follow the same calling conventions as user-defined functions. In this respect, user-defined functions (for example, a function computing the square root of a number) might be considered as “extensions” or “custom upgrades” of the stack machine. This is one of the clearest advantages of stack machines (and of stack programming languages such as Forth) compared to register machines. In contrast, arithmetic instructions (built-in operations) on register machines usually get their parameters from general-purpose registers encoded in the full opcode. A binary operation, such as , thus requires two arguments, and , with and specified by the instruction. A register for storing the result also must be specified. Arithmetic operations can take several possible forms, depending on whether , , and are allowed to take arbitrary values:- Three-address form — Allows the programmer to arbitrarily choose not only the two source registers and , but also a separate destination register . This form is common for most RISC processors, and for the XMM and AVX SIMD instruction sets in the x86-64 architecture.
- Two-address form — Uses one of the two operand registers (usually ) to store the result of an operation, so that is never indicated explicitly. Only and are encoded inside the instruction. This is the most common form of arithmetic operations on register machines, and is quite popular on microprocessors (including the x86 family).
- One-address form — Always takes one of the arguments from the accumulator , and stores the result in as well; then , and only needs to be specified by the instruction. This form is used by some simpler microprocessors (such as Intel 8080).
2.1.8. Return values of functions
In stack machines such as TVM, when a function or primitive needs to return a result value, it simply pushes it into the stack (from which all arguments to the function have already been removed). Therefore, the caller will be able to access the result value through the top-of-stack “register” . This is in complete accordance with Forth calling conventions, but differs slightly from Pascal and C calling conventions, where the accumulator register is normally used for the return value.2.1.9. Returning several values
Some functions might want to return several values , , , with not necessarily equal to one. In these cases, the return values are pushed into the stack in their natural order, starting from . For example, the “divide with remainder” primitive needs to return two values, the quotient and the remainder . Therefore, pushes and into the stack, in that order, so that the quotient is available thereafter at and the remainder at . The net effect of is to divide the original value of by the original value of , and return the quotient in and the remainder in . In this particular case the depth of the stack and the values of all other “stack registers” remain unchanged, because takes two arguments and returns two results. In general, the values of other “stack registers” that lie in the stack below the arguments passed and the values returned are shifted according to the change of the depth of the stack. In principle, some primitives and user-defined functions might return a variable number of result values. In this respect, the remarks above about variadic functions (cf. 2.1.6) apply: the total number of result values and their types should be determined by the values near the top of the stack. (For example, one might push the return values , , , and then push their total number as an integer. The caller would then determine the total number of returned values by inspecting .) In this respect TVM, again, faithfully observes Forth calling conventions.2.1.10. Stack notation
When a stack of depth contains values , , , in that order, with the deepest element and the top of the stack, the contents of the stack are often represented by a list , in that order. When a primitive transforms the original stack state into a new state , this is often written as — ; this is the so-called stack notation. For example, the action of the division primitive can be described by — , where is any list of values. This is usually abbreviated as — , tacitly assuming that all other values deeper in the stack remain intact. Alternatively, one can describe as a primitive that runs on a stack of depth , divides by , and returns the floor-rounded quotient as of the new stack of depth . The new value of equals the old value of for . These descriptions are equivalent, but saying that transforms into , or into , is more concise. The stack notation is extensively used throughout Appendix A, where all currently defined TVM primitives are listed.2.1.11. Explicitly defining the number of arguments to a function
Stack machines usually pass the current stack in its entirety to the invoked primitive or function. That primitive or function accesses only the several values near the top of the stack that represent its arguments, and pushes the return values in their place, by convention leaving all deeper values intact. Then the resulting stack, again in its entirety, is returned to the caller. Most TVM primitives behave in this way, and we expect most user-defined functions to be implemented under such conventions. However, TVM provides mechanisms to specify how many arguments must be passed to a called function (cf. 4.1.10). When these mechanisms are employed, the specified number of values are moved from the caller’s stack into the (usually initially empty) stack of the called function, while deeper values remain in the caller’s stack and are inaccessible to the callee. The caller can also specify how many return values it expects from the called function. Such argument-checking mechanisms might be useful, for example, for a library function that calls user-provided functions passed as arguments to it.2.2 Stack manipulation primitives
A stack machine, such as TVM, employs a lot of stack manipulation primitives to rearrange arguments to other primitives and user-defined functions, so that they become located near the top of the stack in correct order. This section discusses which stack manipulation primitives are necessary and sufficient for achieving this goal, and which of them are used by TVM. Some examples of code using these primitives can be found in Appendix C.2.2.1. Basic stack manipulation primitives
The most important stack manipulation primitives used by TVM are the following:- Top-of-stack exchange operation: or — Exchanges values of and . When , operation is traditionally denoted by . When , this is a (an operation that does nothing, at least if the stack is non-empty).
- Arbitrary exchange operation: — Exchanges values of and . Notice that this operation is not strictly necessary, because it can be simulated by three top-of-stack exchanges: ; ; . However, it is useful to have arbitrary exchanges as primitives, because they are required quite often.
- Push operation: — Pushes a copy of the (old) value of into the stack. Traditionally, is also denoted by (it duplicates the value at the top of the stack), and by .
- Pop operation: — Removes the top-of-stack value and puts it into the (new) , or the old . Traditionally, is also denoted by (it simply drops the top-of-stack value), and by .
2.2.2. Basic stack manipulation primitives suffice
A compiler or a human TVM-code programmer might use the basic stack primitives as follows. Suppose that the function or primitive to be invoked is to be passed, say, three arguments , , and , currently located in stack registers , , and . In this circumstance, the compiler (or programmer) might issue operation (if a copy of is needed after the call to this primitive) or (if it will not be needed afterwards) to put the first argument into the top of the stack. Then, the compiler (or programmer) could use either or , where or , to put into the new top of the stack.8 Proceeding in this manner, we see that we can put the original values of , , and —or their copies, if needed—into locations , , and , using a sequence of push and exchange operations (cf. 2.2.4 and 2.2.5 for a more detailed explanation). In order to generate this sequence, the compiler will need to know only the three values , and , describing the old locations of variables or temporary values in question, and some flags describing whether each value will be needed thereafter or is needed only for this primitive or function call. The locations of other variables and temporary values will be affected in the process, but a compiler (or a human programmer) can easily track their new locations. Similarly, if the results returned from a function need to be discarded or moved to other stack registers, a suitable sequence of exchange and pop operations will do the job. In the typical case of one return value in , this is achieved either by an or a (in most cases, a ) operation.9 Rearranging the result value or values before returning from a function is essentially the same problem as arranging arguments for a function call, and is achieved similarly.2.2.3. Compound stack manipulation primitives
In order to improve the density of the TVM code and simplify development of compilers, compound stack manipulation primitives may be defined, each combining up to four exchange and push or exchange and pop basic primitives. Such compound stack operations might include, for example:- — Equivalent to ; .
- — Equivalent to ; .
- — Equivalent to ; .
- — Equivalent to ; ; . When and , it is also equivalent to ; ; .
- — Equivalent to ; ; .
- — Equivalent to ; ; .
2.2.4. Mnemonics of compound stack operations
The mnemonics of compound stack operations, some examples of which have been provided in 2.2.3, are created as follows. The formal arguments , , to such an operation represent the values in the original stack that will end up in , , after the execution of this compound operation, at least if all , , are distinct and at least . The mnemonic itself of the operation is a sequence of two-letter strings and , with meaning that the corresponding argument is to be PUshed (i.e., a copy is to be created), and meaning that the value is to be eXChanged (i.e., no other copy of the original value is created). Sequences of several or strings may be abbreviated to one or followed by the number of copies. (For instance, we write instead of .) As an exception, if a mnemonic would consist of only or only strings, so that the compound operation is equivalent to a sequence of PUSHes or eXCHanGes, the notation or is used instead of or .2.2.5. Semantics of compound stack operations
Each compound -ary operation ,, is translated into an equivalent sequence of basic stack operations by induction in as follows:- As a base of induction, if , the only nullary compound stack operation corresponds to an empty sequence of basic stack operations.
- Equivalently, we might begin the induction from . Then corresponds to the sequence consisting of one basic operation , and corresponds to the one-element sequence consisting of .
-
For (or for , if we use as induction base), there are two subcases:
- ,,, with , where is a compound operation of arity (i.e., the mnemonic of consists of strings and ). Let be the total quantity of shes in , and be that of ehanges, so that . Then the original operation is translated into , followed by the translation of ,,, defined by the induction hypothesis.
- ,,, with , where is a compound operation of arity . Then the original operation is translated into ; , followed by the translation of ,,, defined by the induction hypothesis.10
2.2.6. Stack manipulation instructions are polymorphic
Notice that the stack manipulation instructions are almost the only “polymorphic” primitives in TVM—i.e., they work with values of arbitrary types (including the value types that will appear only in future revisions of TVM). For example, always interchanges the two top values of the stack, even if one of them is an integer and the other is a cell. Almost all other instructions, especially the data processing instructions (including arithmetic instructions), require each of their arguments to be of some fixed type (possibly different for different arguments).2.3 Efficiency of stack manipulation primitives
Stack manipulation primitives employed by a stack machine, such as TVM, have to be implemented very efficiently, because they constitute more than half of all the instructions used in a typical program. In fact, TVM performs all these instructions in a (small) constant time, regardless of the values involved (even if they represent very large integers or very large trees of cells).2.3.1. Implementation of stack manipulation primitives: using references for operations instead of objects
The efficiency of TVM’s implementation of stack manipulation primitives results from the fact that a typical TVM implementation keeps in the stack not the value objects themselves, but only the references (pointers) to such objects. Therefore, a instruction only needs to interchange the references at and , not the actual objects they refer to.2.3.2. Efficient implementation of and instructions using copy-on-write
Furthermore, a (or, more generally, ) instruction, which appears to make a copy of a potentially large object, also works in small constant time, because it uses a copy-on-write technique of delayed copying: it copies only the reference instead of the object itself, but increases the “reference counter” inside the object, thus sharing the object between the two references. If an attempt to modify an object with a reference counter greater than one is detected, a separate copy of the object in question is made first (incurring a certain “non-uniqueness penalty” or “copying penalty” for the data manipulation instruction that triggered the creation of a new copy).2.3.3. Garbage collecting and reference counting
When the reference counter of a TVM object becomes zero (for example, because the last reference to such an object has been consumed by a operation or an arithmetic instruction), it is immediately freed. Because cyclic references are impossible in TVM data structures, this method of reference counting provides a fast and convenient way of freeing unused objects, replacing slow and unpredictable garbage collectors.2.3.4. Transparency of the implementation: Stack values are “values”, not “references”
Regardless of the implementation details just discussed, all stack values are really “values”, not “references”, from the perspective of the TVM programmer, similarly to the values of all types in functional programming languages. Any attempt to modify an existing object referred to from any other objects or stack locations will result in a transparent replacement of this object by its perfect copy before the modification is actually performed. In other words, the programmer should always act as if the objects themselves were directly manipulated by stack, arithmetic, and other data transformation primitives, and treat the previous discussion only as an explanation of the high efficiency of the stack manipulation primitives.2.3.5. Absence of circular references
One might attempt to create a circular reference between two cells, and , as follows: first create and write some data into it; then create and write some data into it, along with a reference to previously constructed cell ; finally, add a reference to into . While it may seem that after this sequence of operations we obtain a cell , which refers to , which in turn refers to , this is not the case. In fact, we obtain a new cell , which contains a copy of the data originally stored into cell along with a reference to cell , which contains a reference to (the original) cell . In this way the transparent copy-on-write mechanism and the “everything is a value” paradigm enable us to create new cells using only previously constructed cells, thus forbidding the appearance of circular references. This property also applies to all other data structures: for instance, the absence of circular references enables TVM to use reference counting to immediately free unused memory instead of relying on garbage collectors. Similarly, this property is crucial for storing data in the TON Blockchain.3 Cells, memory, and persistent storage
This chapter briefly describes TVM cells, used to represent all data structures inside the TVM memory and its persistent storage, and the basic operations used to create cells, write (or serialize) data into them, and read (or deserialize) data from them.3.1 Generalities on cells
This section presents a classification and general descriptions of cell types.3.1.1. TVM memory and persistent storage consist of cells
Recall that the TVM memory and persistent storage consist of (TVM) cells. Each cell contains up to 1023 bits of data and up to four references to other cells.11 Circular references are forbidden and cannot be created by means of TVM (cf. 2.3.5). In this way, all cells kept in TVM memory and persistent storage constitute a directed acyclic graph (DAG).3.1.2. Ordinary and exotic cells
Apart from the data and references, a cell has a cell type, encoded by an integer . A cell of type is called ordinary; such cells do not require any special processing. Cells of other types are called exotic, and may be loaded—automatically replaced by other cells when an attempt to deserialize them (i.e., to convert them into a Slice by a instruction) is made. They may also exhibit a non-trivial behavior when their hashes are computed. The most common use for exotic cells is to represent some other cells—for instance, cells present in an external library, or pruned from the original tree of cells when a Merkle proof has been created. The type of an exotic cell is stored as the first eight data bits of its data. If an exotic cell has less than eight data bits, it is invalid.3.1.3. The level of a cell
Every cell has another attribute called its (de Brujn) level, which currently takes integer values in the range . The level of an ordinary cell is always equal to the maximum of the levels of all its children : for an ordinary cell containing references to cells , , . If , . Exotic cells may have different rules for setting their level. A cell’s level affects the number of higher hashes it has. More precisely, a level cell has higher hashes , , in addition to its representation hash . Cells of non-zero level appear inside Merkle proofs and Merkle updates, after some branches of the tree of cells representing a value of an abstract data type are pruned.3.1.4. Standard cell representation
When a cell needs to be transferred by a network protocol or stored in a disk file, it must be serialized. The standard representation of a cell as an octet (byte) sequence is constructed as follows:- Two descriptor bytes and are serialized first. Byte equals , where is the quantity of cell references contained in the cell, is the level of the cell, and is for exotic cells and for ordinary cells. Byte equals , where is the quantity of data bits in .
- Then the data bits are serialized as 8-bit octets (bytes). If is not a multiple of eight, a binary and up to six binary s are appended to the data bits. After that, the data is split into eight-bit groups, and each group is interpreted as an unsigned big-endian integer and stored into an octet.
- Finally, each of the cell references is represented by 32 bytes containing the 256-bit representation hash , explained below in 3.1.5, of the cell referred to.
3.1.5. The representation hash of a cell
The 256-bit representation hash or simply hash of a cell is recursively defined as the SHA-256 of the standard representation of the cell : Notice that cyclic cell references are not allowed and cannot be created by means of the TVM (cf. 2.3.5), so this recursion always ends, and the representation hash of any cell is well-defined.3.1.6. The higher hashes of a cell
Recall that a cell of level has higher hashes , , as well. Exotic cells have their own rules for computing their higher hashes. Higher hashes of an ordinary cell are computed similarly to its representation hash, but using the higher hashes of its children instead of their representation hashes . By convention, we set , and for all .123.1.7. Types of exotic cells
TVM currently supports the following cell types:- Type : Ordinary cell — Contains up to 1023 bits of data and up to four cell references.
- Type 1: Pruned branch cell — May have any level . It contains exactly data bits: first an 8-bit integer equal to 1 (representing the cell’s type), then its higher hashes , , . The level of a pruned branch cell may be called its de Brujn index, because it determines the outer Merkle proof or Merkle update during the construction of which the branch has been pruned. An attempt to load a pruned branch cell usually leads to an exception.
- Type 2: Library reference cell — Always has level 0, and contains data bits, including its 8-bit type integer 2 and the representation hash of the library cell being referred to. When loaded, a library reference cell may be transparently replaced by the cell it refers to, if found in the current library context.
- Type 3: Merkle proof cell — Has exactly one reference and level , which must be one less than the level of its only child : The data bits of a Merkle proof cell contain its 8-bit type integer 3, followed by (assumed to be equal to if ). The higher hashes of are computed similarly to the higher hashes of an ordinary cell, but with used instead of . When loaded, a Merkle proof cell is replaced by .
- Type 4: Merkle update cell — Has two children and . Its level is given by A Merkle update behaves like a Merkle proof for both and , and contains data bits with and . However, an extra requirement is that all pruned branch cells that are descendants of and are bound by must also be descendants of .13 When a Merkle update cell is loaded, it is replaced by .
3.1.8. All values of algebraic data types are trees of cells
Arbitrary values of arbitrary algebraic data types (e.g., all types used in functional programming languages) can be serialized into trees of cells (of level 0), and such representations are used for representing such values within TVM. The copy-on-write mechanism (cf. 2.3.2) allows TVM to identify cells containing the same data and references, and to keep only one copy of such cells. This actually transforms a tree of cells into a directed acyclic graph (with the additional property that all its vertices be accessible from a marked vertex called the “root”). However, this is a storage optimization rather than an essential property of TVM. From the perspective of a TVM code programmer, one should think of TVM data structures as trees of cells.3.1.9. TVM code is a tree of cells
The TVM code itself is also represented by a tree of cells. Indeed, TVM code is simply a value of some complex algebraic data type, and as such, it can be serialized into a tree of cells. The exact way in which the TVM code (e.g., TVM assembly code) is transformed into a tree of cells is explained later (cf. 4.1.4 and 5.2), in sections discussing control flow instructions, continuations, and TVM instruction encoding.3.1.10. “Everything is a bag of cells” paradigm
As described in [1, 2.5.14], all the data used by the TON Blockchain, including the blocks themselves and the blockchain state, can be represented—and are represented—as collections, or “bags”, of cells. We see that TVM’s structure of data (cf. 3.1.8) and code (cf. 3.1.9) nicely fits into this “everything is a bag of cells” paradigm. In this way, TVM can naturally be used to execute smart contracts in the TON Blockchain, and the TON Blockchain can be used to store the code and persistent data of these smart contracts between invocations of TVM. (Of course, both TVM and the TON Blockchain have been designed so that this would become possible.)3.2 Data manipulation instructions and cells
The next large group of TVM instructions consists of data manipulation instructions, also known as cell manipulation instructions or simply cell instructions. They correspond to memory access instructions of other architectures.3.2.1. Classes of cell manipulation instructions
The TVM cell instructions are naturally subdivided into two principal classes:- Cell creation instructions or serialization instructions, used to construct new cells from values previously kept in the stack and previously constructed cells.
- Cell parsing instructions or deserialization instructions, used to extract data previously stored into cells by cell creation instructions.
3.2.2. Builder and Slice values
Cell creation instructions usually work with Builder values, which can be kept only in the stack (cf. 1.1.3). Such values represent partially constructed cells, for which fast operations for appending bitstrings, integers, other cells, and references to other cells can be defined. Similarly, cell parsing instructions make heavy use of Slice values, which represent either the remainder of a partially parsed cell, or a value (subcell) residing inside such a cell and extracted from it by a parsing instruction.3.2.3. Builder and Slice values exist only as stack values
Notice that Builder and Slice objects appear only as values in a TVM stack. They cannot be stored in “memory” (i.e., trees of cells) or “persistent storage” (which is also a bag of cells). In this sense, there are far more Cell objects than Builder or Slice objects in a TVM environment, but, somewhat paradoxically, a TVM program sees Builder and Slice objects in its stack more often than Cells. In fact, a TVM program does not have much use for Cell values, because they are immutable and opaque; all cell manipulation primitives require that a Cell value be transformed into either a Builder or a Slice first, before it can be modified or inspected.3.2.4. TVM has no separate Bitstring value type
Notice that TVM offers no separate bitstring value type. Instead, bitstrings are represented by Slices that happen to have no references at all, but can still contain up to 1023 data bits.3.2.5. Cells and cell primitives are bit-oriented, not byte-oriented
An important point is that TVM regards data kept in cells as sequences (strings, streams) of (up to 1023) bits, not of bytes. In other words, TVM is a bit-oriented machine, not a byte-oriented machine. If necessary, an application is free to use, say, 21-bit integer fields inside records serialized into TVM cells, thus using fewer persistent storage bytes to represent the same data.3.2.6. Taxonomy of cell creation (serialization) primitives
Cell creation primitives usually accept a Builder argument and an argument representing the value to be serialized. Additional arguments controlling some aspects of the serialization process (e.g., how many bits should be used for serialization) can be also provided, either in the stack or as an immediate value inside the instruction. The result of a cell creation primitive is usually another Builder, representing the concatenation of the original builder and the serialization of the value provided. Therefore, one can suggest a classification of cell serialization primitives according to the answers to the following questions:- Which is the type of values being serialized?
- How many bits are used for serialization? If this is a variable number, does it come from the stack, or from the instruction itself?
- What happens if the value does not fit into the prescribed number of bits? Is an exception generated, or is a success flag equal to zero silently returned in the top of stack?
- What happens if there is insufficient space left in the Builder? Is an exception generated, or is a zero success flag returned along with the unmodified original Builder?
- The type of values being serialized and the serialization format (e.g., for signed integers, for unsigned integers).
- The source of the field width in bits to be used (e.g., for integer serialization instructions means that the bit width is supplied in the stack; otherwise it has to be embedded into the instruction as an immediate value).
- The action to be performed if the operation cannot be completed (by default, an exception is generated; “quiet” versions of serialization instructions are marked by a letter in their mnemonics).
3.2.7. Integer serialization primitives
Integer serialization primitives can be classified according to the above taxonomy as well. For example:- There are signed and unsigned (big-endian) integer serialization primitives.
- The size of the bit field to be used ( for signed integers, for unsigned integers) can either come from the top of stack or be embedded into the instruction itself.
- If the integer to be serialized is not in the range (for signed integer serialization) or (for unsigned integer serialization), a range check exception is usually generated, and if bits cannot be stored into the provided Builder, a cell overflow exception is usually generated.
- Quiet versions of serialization instructions do not throw exceptions; instead, they push on top of the resulting Builder upon success, or return the original Builder with on top of it to indicate failure.
3.2.8. Integers in cells are big-endian by default
Notice that the default order of bits in Integers serialized into Cells is big-endian, not little-endian.14 In this respect TVM is a big-endian machine. However, this affects only the serialization of integers inside cells. The internal representation of the Integer value type is implementation-dependent and irrelevant for the operation of TVM. Besides, there are some special primitives such as for (de)serializing little-endian integers, which must be stored into an integral number of bytes (otherwise “little-endianness” does not make sense, unless one is also willing to revert the order of bits inside octets). Such primitives are useful for interfacing with the little-endian world—for instance, for parsing custom-format messages arriving to a TON Blockchain smart contract from the outside world.3.2.9. Other serialization primitives
Other cell creation primitives serialize bitstrings (i.e., cell slices without references), either taken from the stack or supplied as literal arguments; cell slices (which are concatenated to the cell builder in an obvious way); other Builders (which are also concatenated); and cell references ().3.2.10. Other cell creation primitives
In addition to the cell serialization primitives for certain built-in value types described above, there are simple primitives that create a new empty Builder and push it into the stack (), or transform a Builder into a Cell (), thus finishing the cell creation process. An can be combined with a into a single instruction , which finishes the creation of a cell and immediately stores a reference to it in an “outer” Builder. There are also primitives that obtain the quantity of data bits or references already stored in a Builder, and check how many data bits or references can be stored.3.2.11. Taxonomy of cell deserialisation primitives
Cell parsing, or deserialization, primitives can be classified as described in 3.2.6, with the following modifications:- They work with Slices (representing the remainder of the cell being parsed) instead of Builders.
- They return deserialized values instead of accepting them as arguments.
- They may come in two flavors, depending on whether they remove the deserialized portion from the Slice supplied (“fetch operations”) or leave it unmodified (“prefetch operations”).
- Their mnemonics usually begin with (or for prefetch operations) instead of .
3.2.12. Other cell slice primitives
In addition to the cell deserialisation primitives outlined above, TVM provides some obvious primitives for initializing and completing the cell deserialization process. For instance, one can convert a Cell into a Slice (), so that its deserialisation might begin; or check whether a Slice is empty, and generate an exception if it is not (); or deserialize a cell reference and immediately convert it into a Slice (, equivalent to two instructions and ).3.2.13. Modifying a serialized value in a cell
The reader might wonder how the values serialized inside a cell may be modified. Suppose a cell contains three serialized 29-bit integers, , representing the coordinates of a point in space, and we want to replace with , leaving the other coordinates intact. How would we achieve this? TVM does not offer any ways to modify existing values (cf. 2.3.4 and 2.3.5), so our example can only be accomplished with a series of operations as follows:- Deserialize the original cell into three Integers , , in the stack (e.g., by ; ; ; ; ).
- Increase by one (e.g., by ; ; ).
- Finally, serialize the resulting Integers into a new cell (e.g., by ; ; ; ; ; ).
3.2.14. Modifying the persistent storage of a smart contract
If the TVM code wants to modify its persistent storage, represented by the tree of cells rooted at , it simply needs to rewrite control register by the root of the tree of cells containing the new value of its persistent storage. (If only part of the persistent storage needs to be modified, cf. 3.2.13.)3.3 Hashmaps, or dictionaries
Hashmaps, or dictionaries, are a specific data structure represented by a tree of cells. Essentially, a hashmap represents a map from keys, which are bitstrings of either fixed or variable length, into values of an arbitrary type , in such a way that fast lookups and modifications be possible. While any such structure might be inspected or modified with the aid of generic cell serialization and deserialization primitives, TVM introduces special primitives to facilitate working with these hashmaps.3.3.1. Basic hashmap types
The two most basic hashmap types predefined in TVM are HashmapE or HashmapE, which represents a partially defined map from -bit strings (called keys) for some fixed into values of some type , and Hashmap, which is similar to HashmapE but is not allowed to be empty (i.e., it must contain at least one key-value pair). Other hashmap types are also available—for example, one with keys of arbitrary length up to some predefined bound (up to 1023 bits).3.3.2. Hashmaps as Patricia trees
The abstract representation of a hashmap in TVM is a Patricia tree, or a compact binary trie. It is a binary tree with edges labelled by bitstrings, such that the concatenation of all edge labels on a path from the root to a leaf equals a key of the hashmap. The corresponding value is kept in this leaf (for hashmaps with keys of fixed length), or optionally in the intermediate vertices as well (for hashmaps with keys of variable length). Furthermore, any intermediate vertex must have two children, and the label of the left child must begin with a binary zero, while the label of the right child must begin with a binary one. This enables us not to store the first bit of the edge labels explicitly. It is easy to see that any collection of key-value pairs (with distinct keys) is represented by a unique Patricia tree.3.3.3. Serialization of hashmaps
The serialization of a hashmap into a tree of cells (or, more generally, into a Slice) is defined by the following TL-B scheme:153.3.4. Brief explanation of TL-B schemes
A TL-B scheme, like the one above, includes the following components. The right-hand side of each “equation” is a type, either simple (such as or ) or parametrized (such as ). The parameters of a type must be either natural numbers (i.e., non-negative integers, which are required to fit into 32 bits in practice), such as in , or other types, such as in . The left-hand side of each equation describes a way to define, or even to serialize, a value of the type indicated in the right-hand side. Such a description begins with the name of a constructor, such as or , immediately followed by an optional constructor tag, such as #_ or $10, which describes the bitstring used to encode (serialize) the constructor in question. Such tags may be given in either binary (after a dollar sign) or hexadecimal notation (after a hash sign), using the conventions described in 1.0. If a tag is not explicitly provided, TL-B computes a default 32-bit constructor tag by hashing the text of the “equation” defining this constructor in a certain fashion. Therefore, empty tags must be explicitly provided by #_ or $_. All constructor names must be distinct, and constructor tags for the same type must constitute a prefix code (otherwise the deserialization would not be unique). The constructor and its optional tag are followed by field definitions. Each field definition is of the form , where is an identifier with the name of the field16 (replaced by an underscore for anonymous fields), and is the field’s type. The type provided here is a type expression, which may include simple types or parametrized types with suitable parameters. Variables—i.e., the (identifiers of the) previously defined fields of types (natural numbers) or (type of types)—may be used as parameters for the parametrized types. The serialization process recursively serializes each field according to its type, and the serialization of a value ultimately consists of the concatenation of bitstrings representing the constructor (i.e., the constructor tag) and the field values. Some fields may be implicit. Their definitions are surrounded by curly braces, which indicate that the field is not actually present in the serialization, but that its value must be deduced from other data (usually the parameters of the type being serialized). Some occurrences of “variables” (i.e., already-defined fields) are prefixed by a tilde. This indicates that the variable’s occurrence is used in the opposite way of the default behavior: in the left-hand side of the equation, it means that the variable will be deduced (computed) based on this occurrence, instead of substituting its previously computed value; in the right-hand side, conversely, it means that the variable will not be deduced from the type being serialized, but rather that it will be computed during the deserialization process. In other words, a tilde transforms an “input argument” into an “output argument”, and vice versa.17 Finally, some equalities may be included in curly brackets as well. These are certain “equations”, which must be satisfied by the “variables” included in them. If one of the variables is prefixed by a tilde, its value will be uniquely determined by the values of all other variables participating in the equation (which must be known at this point) when the definition is processed from the left to the right. A caret (^) preceding a type means that instead of serializing a value of type as a bitstring inside the current cell, we place this value into a separate cell, and add a reference to it into the current cell. Therefore^ means “the type of references to cells containing values of type ”.
Parametrized type with (this notation means ” of type ”, i.e., a natural number) denotes the subtype of the natural numbers type , consisting of integers ; it is serialized into bits as an unsigned big-endian integer. Type by itself is serialized as an unsigned 32-bit integer. Parametrized type with is equivalent to (i.e., it is an unsigned -bit integer).
3.3.5. Application to the serialization of hashmaps
Let us explain the net result of applying the general rules described in 3.3.4 to the TL-B scheme presented in 3.3.3. Suppose we wish to serialize a value of type HashmapE for some integer and some type (i.e., a dictionary with -bit keys and values of type , admitting an abstract representation as a Patricia tree (cf. 3.3.2)). First of all, if our dictionary is empty, it is serialized into a single binary , which is the tag of nullary constructor . Otherwise, its serialization consists of a binary (the tag of ), along with a reference to a cell containing the serialization of a value of type Hashmap (i.e., a necessarily non-empty dictionary). The only way to serialize a value of type Hashmap is given by the constructor, which instructs us to serialize first the label of the edge leading to the root of the subtree under consideration (i.e., the common prefix of all keys in our (sub)dictionary). This label is of type , which means that it is a bitstring of length at most , serialized in such a way that the true length of the label, , becomes known from the serialization of the label. (This special serialization method is described separately in 3.3.6.) The label must be followed by the serialization of a of type , where . It corresponds to a vertex of the Patricia tree, representing a non-empty subdictionary of the original dictionary with -bit keys, obtained by removing from all the keys of the original subdictionary their common prefix of length . If , a value of type is given by the constructor, which describes a leaf of the Patricia tree—or, equivalently, a subdictionary with -bit keys. A leaf simply consists of the corresponding of type and is serialized accordingly. On the other hand, if , a value of type corresponds to a fork (i.e., an intermediate node) in the Patricia tree, and is given by the constructor. Its serialization consists of and , two references to cells containing values of type , which correspond to the left and the right child of the intermediate node in question—or, equivalently, to the two subdictionaries of the original dictionary consisting of key-value pairs with keys beginning with a binary or a binary , respectively. Because the first bit of all keys in each of these subdictionaries is known and fixed, it is removed, and the resulting (necessarily non-empty) subdictionaries are recursively serialized as values of type .3.3.6. Serialization of labels
There are several ways to serialize a label of length at most , if its exact length is (recall that the exact length must be deducible from the serialization of the label itself, while the upper bound is known before the label is serialized or deserialized). These ways are described by the three constructors , , and of type :- — Describes a way to serialize “short” labels, of small length . Such a serialization consists of a binary (the constructor tag of ), followed by binary s and one binary (the unary representation of the length ), followed by bits comprising the label itself.
- — Describes a way to serialize “long” labels, of arbitrary length . Such a serialization consists of a binary (the constructor tag of ), followed by the big-endian binary representation of the length in bits, followed by bits comprising the label itself.
- — Describes a way to serialize “long” labels, consisting of repetitions of the same bit . Such a serialization consists of (the constructor tag of ), followed by the bit , followed by the length stored in bits as before.
3.3.7. An example of dictionary serialization
Consider a dictionary with three 16-bit keys 13, 17, and 239 (considered as big-endian integers) and corresponding 16-bit values 169, 289, and 57121. In binary form:3.3.8. Ways to describe the serialization of type
Notice that the built-in TVM primitives for dictionary manipulation need to know something about the serialization of type ; otherwise, they would not be able to work correctly with , because values of type are immediately contained in the Patricia tree leaf cells. There are several options available to describe the serialization of type :- The simplest case is when for some other type . In this case the serialization of itself always consists of one reference to a cell, which in fact must contain a value of type , something that is not relevant for dictionary manipulation primitives.
- Another simple case is when the serialization of any value of type always consists of data bits and references. Integers and can then be passed to a dictionary manipulation primitive as a simple description of . (Notice that the previous case corresponds to , .)
- A more sophisticated case can be described by four integers , , with and used when the first bit of the serialization equals . When and , this case reduces to the previous one.
- Finally, the most general description of the serialization of a type is given by a splitting function for , which accepts one Slice parameter , and returns two Slices, and , where is the only prefix of that is the serialization of a value of type , and is the remainder of . If no such prefix exists, the splitting function is expected to throw an exception. Notice that a compiler for a high-level language, which supports some or all algebraic TL-B types, is likely to automatically generate splitting functions for all types defined in the program.
3.3.9. A simplifying assumption on the serialization of
One may notice that values of type always occupy the remaining part of an / cell inside the serialization of a HashmapE . Therefore, if we do not insist on strict validation of all dictionaries accessed, we may assume that everything left unparsed in an / cell after deserializing its is a value of type . This greatly simplifies the creation of dictionary manipulation primitives, because in most cases they turn out not to need any information about at all.3.3.10. Basic dictionary operations
Let us present a classification of basic operations with dictionaries (i.e., values of type HashmapE ):- — Given :HashmapE and a key , returns the corresponding value kept in .
- — Given :HashmapE, a key , and a value , sets to in a copy of , and returns the resulting dictionary (cf. 2.3.4).
- — Similar to , but adds the key-value pair to only if key is absent in .
- — Similar to , but changes to only if key is already present in .
- , , — Similar to , , and , respectively, but returns the old value of as well.
- — Deletes key from dictionary , and returns the resulting dictionary .
- , — Gets the minimal or maximal key from dictionary , along with the associated value .
- , — Similar to and , but also removes the key in question from dictionary , and returns the modified dictionary . May be used to iterate over all elements of , effectively using (a copy of) itself as an iterator.
- — Computes the minimal key (or in a variant) and returns it along with the corresponding value . May be used to iterate over all elements of .
- — Computes the maximal key (or in a variant) and returns it along with the corresponding value .
- — Creates an empty dictionary :HashmapE.
- — Checks whether a dictionary is empty.
- — Given , creates a dictionary from a list of key-value pairs passed in stack.
- — Given :HashmapE and some -bit string for , returns subdictionary of , consisting of keys beginning with . The result may be of either type HashmapE or type HashmapE.
- — Given :HashmapE, , , and :HashmapE, replaces with the subdictionary of consisting of keys beginning with , and returns the resulting dictionary :HashmapE. Some variants of may also return the old value of the subdictionary in question.
- — Equivalent to with being an empty dictionary.
- — Given :HashmapE, returns and :HashmapE, the two subdictionaries of consisting of all keys beginning with and , respectively.
- — Given and :HashmapE, computes :HashmapE, such that and .
- — Executes a function with two arguments and , with running over all key-value pairs of a dictionary in lexicographical order.18
- — Similar to , but processes all key-value pairs in reverse order.
- — Given :HashmapE, a value , and two functions and , performs a “tree reduction” of by first applying to all the leaves, and then using to compute the value corresponding to a fork starting from the values assigned to its children.19
3.3.11. Taxonomy of dictionary primitives
The dictionary primitives, described in detail in Appendix A.10, can be classified according to the following categories:- Which dictionary operation (cf. 3.3.10) do they perform?
- Are they specialized for the case If so, do they represent values of type by Cells or by Slices? (Generic versions always represent values of type as Slices.)
- Are the dictionaries themselves passed and returned as Cells or as Slices? (Most primitives represent dictionaries as Slices.)
- Is the key length fixed inside the primitive, or is it passed in the stack?
- Are the keys represented by Slices, or by signed or unsigned Integers?
3.4 Hashmaps with variable-length keys
TVM provides some support for dictionaries, or hashmaps, with variable-length keys, in addition to its support for dictionaries with fixed-length keys (as described in 3.3 above).3.4.1. Serialization of dictionaries with variable-length keys
The serialization of a VarHashmap into a tree of cells (or, more generally, into a Slice) is defined by a TL-B scheme, similar to that described in 3.3.3:3.4.2. Serialization of prefix codes
One special case of a dictionary with variable-length keys is that of a prefix code, where the keys cannot be prefixes of each other. Values in such dictionaries may occur only in the leaves of a Patricia tree. The serialization of a prefix code is defined by the following TL-B scheme:4 Control flow, continuations, and exceptions
This chapter describes continuations, which may represent execution tokens and exception handlers in TVM. Continuations are deeply involved with the control flow of a TVM program; in particular, subroutine calls and conditional and iterated execution are implemented in TVM using special primitives that accept one or more continuations as their arguments. We conclude this chapter with a discussion of the problem of recursion and of families of mutually recursive functions, exacerbated by the fact that cyclic references are not allowed in TVM data structures (including TVM code).4.1 Continuations and subroutines
Recall (cf. 1.1.3) that Continuation values represent “execution tokens” that can be executed later—for example, by = (“execute” or “call indirect”) or (“jump indirect”) primitives. As such, the continuations are heavily used by control flow primitives, enabling subroutine calls, conditional expressions, loops, and so on.4.1.1. Ordinary continuations
The most common kind of continuations are the ordinary continuations, containing the following data:- A Slice (cf. 1.1.3 and 3.2.2), containing (the remainder of) the TVM code to be executed.
- A (possibly empty) Stack , containing the original contents of the stack for the code to be executed.
- A (possibly empty) list of pairs (also called “savelist”), containing the values of control registers to be restored before the execution of the code.
- A 16-bit integer value , selecting the TVM codepage used to interpret the TVM code from .
- An optional non-negative integer , indicating the number of arguments expected by the continuation.
4.1.2. Simple ordinary continuations
In most cases, the ordinary continuations are the simplest ones, having empty and . They consist essentially of a reference to (the remainder of) the code to be executed, and of the codepage to be used while decoding the instructions from this code.4.1.3. Current continuation cc
The “current continuation” is an important part of the total state of TVM, representing the code being executed right now (cf. 1.1). In particular, what we call “the current stack” (or simply “the stack”) when discussing all other primitives is in fact the stack of the current continuation. All other components of the total state of TVM may be also thought of as parts of the current continuation ; however, they may be extracted from the current continuation and kept separately as part of the total state for performance reasons. This is why we describe the stack, the control registers, and the codepage as separate parts of the TVM state in 1.4.4.1.4. Normal work of TVM, or the main loop
TVM usually performs the following operations: If the current continuation is an ordinary one, it decodes the first instruction from the Slice , similarly to the way other cells are deserialized by TVM primitives (cf. 3.2 and 3.2.11): it decodes the opcode first, and then the parameters of the instruction (e.g., 4-bit fields indicating “stack registers” involved for stack manipulation primitives, or constant values for “push constant” or “literal” primitives). The remainder of the Slice is then put into the of the new , and the decoded operation is executed on the current stack. This entire process is repeated until there are no operations left in . If the is empty (i.e., contains no bits of data and no references), or if a (rarely needed) explicit subroutine return () instruction is encountered, the current continuation is discarded, and the “return continuation” from control register is loaded into instead (this process is discussed in more detail starting in 4.1.6).20 Then the execution continues by parsing operations from the new current continuation.4.1.5. Extraordinary continuations
In addition to the ordinary continuations considered so far (cf. 4.1.1), TVM includes some extraordinary continuations, representing certain less common states. Examples of extraordinary continuations include:- The continuation with its parameter set to zero, which represents the end of the work of TVM. This continuation is the original value of when TVM begins executing the code of a smart contract.
- The continuation , which contains references to two other continuations (ordinary or not) representing the body of the loop being executed and the code to be executed after the loop.
4.1.6. Switching to another continuation: and
The process of switching to another continuation may be performed by such instructions as (which takes from the stack) or (which uses as ). This process is slightly more complex than simply setting the value of to : before doing this, either all values or the top values in the current stack are moved to the stack of the continuation , and only then is the remainder of the current stack discarded. If all values need to be moved (the most common case), and if the continuation has an empty stack (also the most common case; notice that extraordinary continuations are assumed to have an empty stack), then the new stack of equals the stack of the current continuation, so we can simply transfer the current stack in its entirety to . (If we keep the current stack as a separate part of the total state of TVM, we have to do nothing at all.)4.1.7. Determining the number of arguments passed to the next continuation
By default, equals the depth of the current stack. However, if has an explicit value of (number of arguments to be provided), then is computed as , equal to . minus the current depth of ‘s stack. Furthermore, there are special forms of and that provide an explicit value , the number of parameters from the current stack to be passed to continuation . If is provided, it must be less than or equal to the depth of the current stack, or else a stack underflow exception occurs. If both and are provided, we must have , in which case is used. If is provided and is not, then is used. One could also imagine that the default value of equals the depth of the original stack, and that values are always removed from the top of the original stack even if only of them are actually moved to the stack of the next continuation . Even though the remainder of the current stack is discarded afterwards, this description will become useful later.4.1.8. Restoring control registers from the new continuation
After the new stack is computed, the values of control registers present in . are restored accordingly, and the current codepage is also set to .. Only then does TVM set equal to the new and begin its execution.224.1.9. Subroutine calls: or primitives
The execution of continuations as subroutines is slightly more complicated than switching to continuations. Consider the or primitive, which takes a continuation from the (current) stack and executes it as a subroutine. Apart from doing the stack manipulations described in 4.1.6 and 4.1.7 and setting the new control registers and codepage as described in 4.1.8, these primitives perform several additional steps:- After the top values are removed from the current stack (cf. 4.1.7), the (usually empty) remainder is not discarded, but instead is stored in the (old) current continuation .
- The old value of the special register is saved into the (previously empty) savelist .
- The continuation thus modified is not discarded, but instead is set as the new , which performs the role of “next continuation” or “return continuation” for the subroutine being called.
- After that, the switching to continues as before. In particular, some control registers are restored from ., potentially overwriting the value of set in the previous step. (Therefore, a good optimization would be to check that is present in . from the very beginning, and skip the three previous steps as useless in this case.)
4.1.10. Determining the number of arguments passed to and/or return values accepted from a subroutine
Similarly to and , also has special (rarely used) forms, which allow us to explicitly specify the number of arguments passed from the current stack to the called subroutine (by default, equals the depth of the current stack, i.e., it is passed in its entirety). Furthermore, a second number can be specified, used to set of the modified continuation before storing it into the new ; the new equals the depth of the old stack minus plus . This means that the caller is willing to pass exactly arguments to the called subroutine, and is willing to accept exactly results in their stead. Such forms of and are mostly intended for library functions that accept functional arguments and want to invoke them safely. Another application is related to the “virtualization support” of TVM, which enables TVM code to run other TVM code inside a “virtual TVM machine”. Such virtualization techniques might be useful for implementing sophisticated payment channels in the TON Blockchain (cf. [1, 5]).4.1.11. : call with current continuation
Notice that TVM supports a form of the “call with current continuation” primitive. Namely, primitive is similar to or in that it takes a continuation from the stack and switches to it; however, does not discard the previous current continuation (as does) and does not write to (as does), but rather pushes into the (new) stack as an extra argument to . The primitive does a similar thing, but pushes only the (remainder of the) code of the previous current continuation as a Slice.4.2 Control flow primitives: conditional and iterated execution
4.2.1. Conditional execution: , ,
An important modification of (or ) consists in its conditional forms. For example, accepts an integer and a continuation , and executes (in the same way as would do it) only if is non-zero; otherwise both values are simply discarded from the stack. Similarly, accepts and , but executes only if . Finally, accepts , , and , removes these values from the stack, and executes if or if .4.2.2. Iterated execution and loops
More sophisticated modifications of include:- — Takes an integer and a continuation , and executes times.23
- — Takes and , executes , and then takes the top value from the stack. If is non-zero, it executes and then begins a new loop by executing again; if is zero, it stops.
- — Takes , executes it, and then takes the top integer from the stack. If is zero, a new iteration begins; if is non-zero, the previously executed code is resumed.
4.2.3. Constant, or literal, continuations
We see that we can create arbitrarily complex conditional expressions and loops in the TVM code, provided we have a means to push constant continuations into the stack. In fact, TVM includes special versions of “literal” or “constant” primitives that cut the next bytes or bits from the remainder of the current code into a cell slice, and then push it into the stack not as a Slice (as a does) but as a simple ordinary Continuation (which has only and ). The simplest of these primitives is , which has an immediate argument describing the number of subsequent bytes (in a byte-oriented version of TVM) or bits to be converted into a simple continuation. Another primitive is , which removes the first cell reference from the current continuation , converts the cell referred to into a cell slice, and finally converts the cell slice into a simple continuation.4.2.4. Constant continuations combined with conditional or iterated execution primitives
Because constant continuations are very often used as arguments to conditional or iterated execution primitives, combined versions of these primitives (e.g., or ) may be defined in a future revision of TVM, which combine a or with another primitive. If one inspects the resulting code, looks very much like the more customary “conditional-branch-forward” instruction.4.3 Operations with continuations
4.3.1 Continuations are opaque
Notice that all continuations are opaque, at least in the current version of TVM, meaning that there is no way to modify a continuation or inspect its internal data. Almost the only use of a continuation is to supply it to a control flow primitive. While there are some arguments in favor of including support for non-opaque continuations in TVM (along with opaque continuations, which are required for virtualization), the current revision offers no such support.4.3.2. Allowed operations with continuations
However, some operations with opaque continuations are still possible, mostly because they are equivalent to operations of the kind “create a new continuation, which will do something special, and then invoke the original continuation”. Allowed operations with continuations include:- Push one or several values into the stack of a continuation (thus creating a partial application of a function, or a closure).
- Set the saved value of a control register inside the savelist . of a continuation . If there is already a value for the control register in question, this operation silently does nothing.
4.3.3. Example: operations with control registers
TVM has some primitives to set and inspect the values of control registers. The most important of them are (pushes the current value of into the stack) and (sets the value of from the stack, if the supplied value is of the correct type). However, there is also a modified version of the latter instruction, called , which saves the old value of (for ) into the continuation at as described in 4.3.2 before setting the new value.4.3.4. Example: setting the number of arguments to a function in its code
The primitive demonstrates another application of continuations in an operation: it leaves only the top values of the current stack, and moves the remainder to the stack of the continuation in . This primitive enables a called function to “return” unneeded arguments to its caller’s stack, which is useful in some situations (e.g., those related to exception handling).4.3.5. Boolean circuits
A continuation may be thought of as a piece of code with two optional exit points kept in the savelist of : the principal exit point given by .:=.(), and the auxiliary exit point given by .:=.(). If executed, a continuation performs whatever action it was created for, and then (usually) transfers control to the principal exit point, or, on some occasions, to the auxiliary exit point. We sometimes say that a continuation with both exit points . and . defined is a two-exit continuation, or a boolean circuit, especially if the choice of the exit point depends on some internally-checked condition.4.3.6. Composition of continuations
One can compose two continuations and simply by setting . or . to . This creates a new continuation denoted by or , which differs from in its savelist. (Recall that if the savelist of already has an entry corresponding to the control register in question, such an operation silently does nothing as explained in 4.3.2). By composing continuations, one can build chains or other graphs, possibly with loops, representing the control flow. In fact, the resulting graph resembles a flow chart, with the boolean circuits corresponding to the “condition nodes” (containing code that will transfer control either to or to depending on some condition), and the one-exit continuations corresponding to the “action nodes”.4.3.7. Basic continuation composition primitives
Two basic primitives for composing continuations are (also known as and ) and (also known as and ), which take and from the stack, set . or . to , and return the resulting continuation or . All other continuation composition operations can be expressed in terms of these two primitives.4.3.8. Advanced continuation composition primitives
However, TVM can compose continuations not only taken from stack, but also taken from or , or from the current continuation ; likewise, the result may be pushed into the stack, stored into either or , or used as the new current continuation (i.e., the control may be transferred to it). Furthermore, TVM can define conditional composition primitives, performing some of the above actions only if an integer value taken from the stack is non-zero. For instance, can be described as , with continuation taken from the original stack. Similarly, is , and (also known as in a boolean circuit context) is . Other interesting primitives include () and (). Finally, some “experimental” primitives also involve and . For example:- or does .
- Conditional versions of and may also be useful: takes an integer from the stack, and performs if , otherwise.
- does ; if the two continuations in and represent the two branches we should select depending on some boolean expression, negates this expression on the outer level.
- does .. to a continuation taken from the stack.
- Variants of include () and ().
- takes a continuation from the stack and does . If represents a boolean circuit, the net effect is to evaluate it and push either or into the stack before continuing.
4.4 Continuations as objects
4.4.1. Representing objects using continuations
Object-oriented programming in Smalltalk (or Objective C) style may be implemented with the aid of continuations. For this, an object is represented by a special continuation . If it has any data fields, they can be kept in the stack of , making a partial application (i.e., a continuation with a non-empty stack). When somebody wants to invoke a method of with arguments , , , , she pushes the arguments into the stack, then pushes a magic number corresponding to the method , and then executes passing arguments (cf. 4.1.10). Then uses the top-of-stack integer to select the branch with the required method, and executes it. If needs to modify its state, it simply computes a new continuation of the same sort (perhaps with the same code as , but with a different initial stack). The new continuation is returned to the caller along with whatever other return values need to be returned.4.4.2. Serializable objects
Another way of representing Smalltalk-style objects as continuations, or even as trees of cells, consists in using the primitive (a variant of , cf. 4.1.11), which takes the first cell reference from the code of the current continuation, transforms the cell referred to into a simple ordinary continuation, and transfers control to it, first pushing the remainder of the current continuation as a Slice into the stack. In this way, an object might be represented by a cell that contains at the beginning of its data, and the actual code of the object in the first reference (one might say that the first reference of cell is the class of object ). Remaining data and references of this cell will be used for storing the fields of the object. Such objects have the advantage of being trees of cells, and not just continuations, meaning that they can be stored into the persistent storage of a TON smart contract.4.4.3. Unique continuations and capabilities
It might make sense (in a future revision of TVM) to mark some continuations as unique, meaning that they cannot be copied, even in a delayed manner, by increasing their reference counter to a value greater than one. If an opaque continuation is unique, it essentially becomes a capability, which can either be used by its owner exactly once or be transferred to somebody else. For example, imagine a continuation that represents the output stream to a printer (this is an example of a continuation used as an object, cf. 4.4.1). When invoked with one integer argument , this continuation outputs the character with code to the printer, and returns a new continuation of the same kind reflecting the new state of the stream. Obviously, copying such a continuation and using the two copies in parallel would lead to some unintended side effects; marking it as unique would prohibit such adverse usage.4.5 Exception handling
TVM’s exception handling is quite simple and consists in a transfer of control to the continuation kept in control register .4.5.1. Two arguments of the exception handler: exception parameter and exception number
Every exception is characterized by two arguments: the exception number (an Integer) and the exception parameter (any value, most often a zero Integer). Exception numbers 0–31 are reserved for TVM, while all other exception numbers are available for user-defined exceptions.4.5.2. Primitives for throwing an exception
There are several special primitives used for throwing an exception. The most general of them, , takes two arguments, and , from the stack, and throws the exception with number and value . There are variants of this primitive that assume to be a zero integer, store as a literal value, and/or are conditional on an integer value taken from the stack. User-defined exceptions may use arbitrary values as (e.g., trees of cells) if needed.4.5.3. Exceptions generated by TVM
Of course, some exceptions are generated by normal primitives. For example, an arithmetic overflow exception is generated whenever the result of an arithmetic operation does not fit into a signed 257-bit integer. In such cases, the arguments of the exception, and , are determined by TVM itself.4.5.4. Exception handling
The exception handling itself consists in a control transfer to the exception handler—i.e., the continuation specified in control register , with and supplied as the two arguments to this continuation, as if a to had been requested with arguments (cf. 4.1.7 and 4.1.6). As a consequence, and end up in the top of the stack of the exception handler. The remainder of the old stack is discarded. Notice that if the continuation in has a value for in its savelist, it will be used to set up the new value of before executing the exception handler. In particular, if the exception handler invokes , it will re-throw the original exception with the restored value of . This trick enables the exception handler to handle only some exceptions, and pass the rest to an outer exception handler.4.5.5. Default exception handler
When an instance of TVM is created, contains a reference to the “default exception handler continuation”, which is an extraordinary continuation (cf. 4.1.5). Its execution leads to the termination of the execution of TVM, with the arguments and of the exception returned to the outside caller. In the context of the TON Blockchain, will be stored as a part of the transaction’s result.4.5.6. primitive
A primitive can be used to implement C++- like exception handling. This primitive accepts two continuations, and . It stores the old value of into the savelist of , sets to , and executes just as would, but additionally saving the old value of into the savelist of the new as well. Usually a version of the primitive with an explicit number of arguments passed to the continuation is used. The net result is roughly equivalent to C’s { } { } operator.4.5.7. List of predefined exceptions
Predefined exceptions of TVM correspond to exception numbers in the range 0–31. They include:- Normal termination () — Should never be generated, but it is useful for some tricks.
- Alternative termination () — Again, should never be generated.
- Stack underflow () — Not enough arguments in the stack for a primitive.
- Stack overflow () — More values have been stored on a stack than allowed by this version of TVM.
- Integer overflow () — Integer does not fit into , or a division by zero has occurred.
- Range check error () — Integer out of expected range.
- Invalid opcode () — Instruction or its immediate arguments cannot be decoded.
- Type check error () — An argument to a primitive is of incorrect value type.
- Cell overflow () — Error in one of the serialization primitives.
- Cell underflow () — Deserialization error.
- Dictionary error () — Error while deserializing a dictionary object.
- Unknown error () — Unknown error, may be thrown by user programs.
- Fatal error () — Thrown by TVM in situations deemed impossible.
- Out of gas () — Thrown by TVM when the remaining gas () becomes negative. This exception usually cannot be caught and leads to an immediate termination of TVM.
4.5.8. Order of stack underflow, type check, and range check exceptions
All TVM primitives first check whether the stack contains the required number of arguments, generating a stack underflow exception if this is not the case. Only then are the type tags of the arguments and their ranges (e.g., if a primitive expects an argument not only to be an Integer, but also to be in the range from 0 to 256) checked, starting from the value in the top of the stack (the last argument) and proceeding deeper into the stack. If an argument’s type is incorrect, a type-checking exception is generated; if the type is correct, but the value does not fall into the expected range, a range check exception is generated. Some primitives accept a variable number of arguments, depending on the values of some small fixed subset of arguments located near the top of the stack. In this case, the above procedure is first run for all arguments from this small subset. Then it is repeated for the remaining arguments, once their number and types have been determined from the arguments already processed.4.6 Functions, recursion, and dictionaries
4.6.1. The problem of recursion
The conditional and iterated execution primitives described in 4.2—along with the unconditional branch, call, and return primitives described in 4.1—enable one to implement more or less arbitrary code with nested loops and conditional expressions, with one notable exception: one can only create new constant continuations from parts of the current continuation. (In particular, one cannot invoke a subroutine from itself in this way.) Therefore, the code being executed—i.e., the current continuation—gradually becomes smaller and smaller.244.6.2. Y-combinator solution: pass a continuation as an argument to itself
One way of dealing with the problem of recursion is by passing a copy of the continuation representing the body of a recursive function as an extra argument to itself. Consider, for example, the following code for a factorial function:4.6.3. A variant of Y-combinator solution
Another way of recursively computing the factorial, more closely following the classical recursive definition is as follows:4.6.4. Comparison: non-recursive definition of the factorial function
Incidentally, a non-recursive definition of the factorial with the aid of a loop is also possible, and it is much shorter than both recursive definitions:4.6.5. Several mutually recursive functions
If one has a collection , , of mutually recursive functions, one can use the same trick by passing the whole collection of continuations in the stack as an extra arguments to each of these functions. However, as grows, this becomes more and more cumbersome, since one has to reorder these extra arguments in the stack to work with the “true” arguments, and then push their copies into the top of the stack before any recursive call.4.6.6. Combining several functions into one tuple
One might also combine a collection of continuations representing functions , , into a “tuple” , and pass this tuple as one stack element . For instance, when , each function can be represented by a cell (along with the tree of cells rooted in this cell), and the tuple may be represented by a cell , which has references to its component cells . However, this would lead to the necessity of “unpacking” the needed component from this tuple before each recursive call.4.6.7. Combining several functions into a selector function
Another approach is to combine several functions , , into one “selector function” , which takes an extra argument , , from the top of the stack, and invokes the appropriate function . Stack machines such as TVM are well-suited to this approach, because they do not require the functions to have the same number and types of arguments. Using this approach, one would need to pass only one extra argument, , to each of these functions, and push into the stack an extra argument before each recursive call to to select the correct function to be called.4.6.8. Using a dedicated register to keep the selector function
However, even if we use one of the two previous approaches to combine all functions into one extra argument, passing this argument to all mutually recursive functions is still quite cumbersome and requires a lot of additional stack manipulation operations. Because this argument changes very rarely, one might use a dedicated register to keep it and transparently pass it to all functions called. This is the approach used by TVM by default.4.6.9. Special register for the selector function
In fact, TVM uses a dedicated register to keep the continuation representing the current or global “selector function”, which can be used to invoke any of a family of mutually recursive functions. Special primitives or (cf. A.8.7) are equivalent to ; ; , and similarly or are equivalent to ; ; . In this way a TVM program, which ultimately is a large collection of mutually recursive functions, may initialize with the correct selector function representing the family of all the functions in the program, and then use to invoke any of these functions by its index (sometimes also called the selector of a function).4.6.10. Initialization of
A TVM program might initialize by means of a instruction. However, because this usually is the very first action undertaken by a program (e.g., a smart contract), TVM makes some provisions for the automatic initialization of . Namely, is initialized by the code (the initial value of ) of the program itself, and an extra zero (or, in some cases, some other predefined number ) is pushed into the stack before the program’s execution. This is approximately equivalent to invoking (or ) at the very beginning of a program—i.e., the function with index zero is effectively themain() function for the program.
4.6.11. Creating selector functions and statements
TVM makes special provisions for simple and concise implementation of selector functions (which usually constitute the top level of a TVM program) or, more generally, arbitraryswitch or case statements (which are also useful in TVM programs). The most important primitives included for this purpose are , , , and (cf. A.8.2). They effectively enable one to combine subroutines, kept either in separate cells or as subslices of certain cells, into a binary decision tree with decisions made according to the indicated bits of the integer passed in the top of the stack.
Another instruction, useful for the implementation of sum-product types, is (cf. A.7.2). This instruction preloads the first several bits of a Slice into an Integer, which can later be inspected by and other similar instructions.
4.6.12. Alternative: using a hashmap to select the correct function
Yet another alternative is to use a Hashmap (cf. 3.3) to hold the “collection” or “dictionary” of the code of all functions in a program, and use the hashmap lookup primitives (cf. A.10) to select the code of the required function, which can then be ed into a continuation (cf. A.8.5) and executed. Special combined “lookup, bless, and execute” primitives, such as and , are also available (cf. A.10.11). This approach may be more efficient for larger programs andswitch statements.
5. Codepages and instruction encoding
This chapter describes the codepage mechanism, which allows TVM to be flexible and extendable while preserving backward compatibility with respect to previously generated code. We also discuss some general considerations about instruction encodings (applicable to arbitrary machine code, not just TVM), as well as the implications of these considerations for TVM and the choices made while designing TVM’s (experimental) codepage zero. The instruction encodings themselves are presented later in Appendix A.5.1 Codepages and interoperability of different TVM versions
The codepages are an essential mechanism of backward compatibility and of future extensions to TVM. They enable transparent execution of code written for different revisions of TVM, with transparent interaction between instances of such code. The mechanism of the codepages, however, is general and powerful enough to enable some other originally unintended applications.5.1.1. Codepages in continuations
Every ordinary continuation contains a 16-bit codepage field (cf. 4.1.1), which determines the codepage that will be used to execute its code. If a continuation is created by a (cf. 4.2.3) or similar primitive, it usually inherits the current codepage (i.e., the codepage of ).255.1.2. Current codepage
The current codepage (cf. 1.4) is the codepage of the current continuation . It determines the way the next instruction will be decoded from , the remainder of the current continuation’s code. Once the instruction has been decoded and executed, it determines the next value of the current codepage. In most cases, the current codepage is left unchanged. On the other hand, all primitives that switch the current continuation load the new value of from the new current continuation. In this way, all code in continuations is always interpreted exactly as it was intended to be.5.1.3. Different versions of TVM may use different codepages
Different versions of TVM may use different codepages for their code. For example, the original version of TVM might use codepage zero. A newer version might use codepage one, which contains all the previously defined opcodes, along with some newly defined ones, using some of the previously unused opcode space. A subsequent version might use yet another codepage, and so on. However, a newer version of TVM will execute old code for codepage zero exactly as before. If the old code contained an opcode used for some new operations that were undefined in the original version of TVM, it will still generate an invalid opcode exception, because the new operations are absent in codepage zero.5.1.4. Changing the behavior of old operations
New codepages can also change the effects of some operations present in the old codepages while preserving their opcodes and mnemonics. For example, imagine a future 513-bit upgrade of TVM (replacing the current 257-bit design). It might use a 513-bit Integer type within the same arithmetic primitives as before. However, while the opcodes and instructions in the new codepage would look exactly like the old ones, they would work differently, accepting 513-bit integer arguments and results. On the other hand, during the execution of the same code in codepage zero, the new machine would generate exceptions whenever the integers used in arithmetic and other primitives do not fit into 257 bits.26 In this way, the upgrade would not change the behavior of the old code.5.1.5. Improving instruction encoding
Another application for codepages is to change instruction encodings, reflecting improved knowledge of the actual frequencies of such instructions in the code base. In this case, the new codepage will have exactly the same instructions as the old one, but with different encodings, potentially of differing lengths. For example, one might create an experimental version of the first version of TVM, using a (prefix) bitcode instead of the original bytecode, aiming to achieve higher code density.5.1.6. Making instruction encoding context-dependent
Another way of using codepages to improve code density is to use several codepages with different subsets of the whole instruction set defined in each of them, or with the whole instruction set defined, but with different length encodings for the same instructions in different codepages. Imagine, for instance, a “stack manipulation” codepage, where stack manipulation primitives have short encodings at the expense of all other operations, and a “data processing” codepage, where all other operations are shorter at the expense of stack manipulation operations. If stack manipulation operations tend to come one after another, we can automatically switch to “stack manipulation” codepage after executing any such instruction. When a data processing instruction occurs, we switch back to “data processing” codepage. If conditional probabilities of the class of the next instruction depending on the class of the previous instruction are considerably different from corresponding unconditional probabilities, this technique—automatically switching into stack manipulation mode to rearrange the stack with shorter instructions, then switching back—might considerably improve the code density.5.1.7. Using codepages for status and control flags
Another potential application of multiple codepages inside the same revision of TVM consists in switching between several codepages depending on the result of the execution of some instructions. For example, imagine a version of TVM that uses two new codepages, 2 and 3. Most operations do not change the current codepage. However, the integer comparison operations will switch to codepage 2 if the condition is false, and to codepage 3 if it is true. Furthermore, a new operation , similar to , will indeed be equivalent to in codepage 3, but will instead be a in codepage 2. Such a trick effectively uses bit 0 of the current codepage as a status flag. Alternatively, one might create a couple of codepages—say, 4 and 5—which differ only in their cell deserialisation primitives. For instance, in codepage 4 they might work as before, while in codepage 5 they might deserialize data not from the beginning of a Slice, but from its end. Two new instructions—say, and —might be used for switching to codepage 4 or codepage 5. Clearly, we have now described a status flag, affecting the execution of some instructions in a certain new manner.5.1.8. Setting the codepage in the code itself
For convenience, we reserve some opcode in all codepages—say, —for the instruction , with from 0 to 255 (cf. A.13). Then by inserting such an instruction into the very beginning of (the main function of) a program (e.g., a TON Blockchain smart contract) or a library function, we can ensure that the code will always be executed in the intended codepage.5.2 Instruction encoding
This section discusses the general principles of instruction encoding valid for all codepages and all versions of TVM. Later, 5.3 discusses the choices made for the experimental “codepage zero”.5.2.1. Instructions are encoded by a binary prefix code
All complete instructions (i.e., instructions along with all their parameters, such as the names of stack registers or other embedded constants) of a TVM codepage are encoded by a binary prefix code. This means that a (finite) binary string (i.e., a bitstring) corresponds to each complete instruction, in such a way that binary strings corresponding to different complete instructions do not coincide, and no binary string among the chosen subset is a prefix of another binary string from this subset.5.2.2. Determining the first instruction from a code stream
As a consequence of this encoding method, any binary string admits at most one prefix, which is an encoding of some complete instruction. In particular, the code of the current continuation (which is a Slice, and thus a bitstring along with some cell references) admits at most one such prefix, which corresponds to the (uniquely determined) instruction that TVM will execute first. After execution, this prefix is removed from the code of the current continuation, and the next instruction can be decoded.5.2.3. Invalid opcode
If no prefix of encodes a valid instruction in the current codepage, an invalid opcode exception is generated (cf. 4.5.7). However, the case of an empty is treated separately as explained in 4.1.4 (the exact behavior may depend on the current codepage).5.2.4. Special case: end-of-code padding
As an exception to the above rule, some codepages may accept some values of that are too short to be valid instruction encodings as additional variants of , thus effectively using the same procedure for them as for an empty . Such bitstrings may be used for padding the code near its end. For example, if binary string (i.e., , cf. 1.0.3) is used in a codepage to encode , its proper prefixes cannot encode any instructions. So this codepage may accept , , , , as variants of if this is all that is left in , instead of generating an invalid opcode exception. Such a padding may be useful, for example, if the primitive (cf. 4.2.3) creates only continuations with code consisting of an integral number of bytes, but not all instructions are encoded by an integral number of bytes.5.2.5. TVM code is a bitcode, not a bytecode
Recall that TVM is a bit-oriented machine in the sense that its Cells (and Slices) are naturally considered as sequences of bits, not just of octets (bytes), cf. 3.2.5. Because the TVM code is also kept in cells (cf. 3.1.9 and 4.1.4), there is no reason to use only bitstrings of length divisible by eight as encodings of complete instructions. In other words, generally speaking, the TVM code is a bitcode, not a bytecode. That said, some codepages (such as our experimental codepage zero) may opt to use a bytecode (i.e., to use only encodings consisting of an integral number of bytes)—either for simplicity, or for the ease of debugging and of studying memory (i.e., cell) dumps.275.2.6. Opcode space used by a complete instruction
Recall from coding theory that the lengths of bitstrings used in a binary prefix code satisfy Kraft–McMillan inequality . This is applicable in particular to the (complete) instruction encoding used by a TVM codepage. We say that a particular complete instruction (or, more precisely, the encoding of a complete instruction) utilizes the portion of the opcode space, if it is encoded by an -bit string. One can see that all complete instructions together utilize at most (i.e., “at most the whole opcode space”).5.2.7. Opcode space used by an instruction, or a class of instructions
The above terminology is extended to instructions (considered with all admissible values of their parameters), or even classes of instructions (e.g., all arithmetic instructions). We say that an (incomplete) instruction, or a class of instructions, occupies portion of the opcode space, if is the sum of the portions of the opcode space occupied by all complete instructions belonging to that class.5.2.8. Opcode space for bytecodes
A useful approximation of the above definitions is as follows: Consider all 256 possible values for the first byte of an instruction encoding. Suppose that of these values correspond to the specific instruction or class of instructions we are considering. Then this instruction or class of instructions occupies approximately the portion of the opcode space. This approximation shows why all instructions cannot occupy together more than the portion of the opcode space, at least without compromising the uniqueness of instruction decoding.5.2.9. Almost optimal encodings
Coding theory tells us that in an optimally dense encoding, the portion of the opcode space used by a complete instruction (, if the complete instruction is encoded in bits) should be approximately equal to the probability or frequency of its occurrence in real programs.28 The same should hold for (incomplete) instructions, or primitives (i.e., generic instructions without specified values of parameters), and for classes of instructions.5.2.10. Example: stack manipulation primitives
For instance, if stack manipulation instructions constitute approximately half of all instructions in a typical TVM program, one should allocate approximately half of the opcode space for encoding stack manipulation instructions. One might reserve the first bytes (“opcodes”) - for such instructions. If a quarter of these instructions are , it would make sense to reserve - for s. Similarly, if half of all s involve the top of stack , it would make sense to use - to encode .5.2.11. Simple encodings of instructions
In most cases, simple encodings of complete instructions are used. Simple encodings begin with a fixed bitstring called the opcode of the instruction, followed by, say, 4-bit fields containing the indices of stack registers specified in the instruction, followed by all other constant (literal, immediate) parameters included in the complete instruction. While simple encodings may not be exactly optimal, they admit short descriptions, and their decoding and encoding can be easily implemented. If a (generic) instruction uses a simple encoding with an -bit opcode, then the instruction will utilize portion of the opcode space. This observation might be useful for considerations described in 5.2.9 and 5.2.10.5.2.12. Optimizing code density further: Huffman codes
One might construct optimally dense binary code for the set of all complete instructions, provided their probabilities or frequences in real code are known. This is the well-known Huffman code (for the given probability distribution). However, such code would be highly unsystematic and hard to decode.5.2.13. Practical instruction encodings
In practice, instruction encodings used in TVM and other virtual machines offer a compromise between code density and ease of encoding and decoding. Such a compromise may be achieved by selecting simple encodings (cf. 5.2.11) for all instructions (maybe with separate simple encodings for some often used variants, such as among all ), and allocating opcode space for such simple encodings using the heuristics outlined in 5.2.9 and 5.2.10; this is the approach currently used in TVM.5.3 Instruction encoding in codepage zero
This section provides details about the experimental instruction encoding for codepage zero, as described elsewhere in this document (cf. Appendix A) and used in the preliminary test version of TVM.5.3.1. Upgradability
First of all, even if this preliminary version somehow gets into the production version of the TON Blockchain, the codepage mechanism (cf. 5.1) enables us to introduce better versions later without compromising backward compatibility.29 So in the meantime, we are free to experiment.5.3.2. Choice of instructions
We opted to include many “experimental” and not strictly necessary instructions in codepage zero just to see how they might be used in real code. For example, we have both the basic (cf. 2.2.1) and the compound (cf. 2.2.3) stack manipulation primitives, as well as some “unsystematic” ones such as (mostly borrowed from Forth). If such primitives are rarely used, their inclusion just wastes some part of the opcode space and makes the encodings of other instructions slightly less effective, something we can afford at this stage of TVM’s development.5.3.3. Using experimental instructions
Some of these experimental instructions have been assigned quite long opcodes, just to fit more of them into the opcode space. One should not be afraid to use them just because they are long; if these instructions turn out to be useful, they will receive shorter opcodes in future revisions. Codepage zero is not meant to be fine-tuned in this respect.5.3.4. Choice of bytecode
We opted to use a bytecode (i.e., to use encodings of complete instructions of lengths divisible by eight). While this may not produce optimal code density, because such a length restriction makes it more difficult to match portions of opcode space used for the encoding of instructions with estimated frequencies of these instructions in TVM code (cf. 5.2.11 and 5.2.9), such an approach has its advantages: it admits a simpler instruction decoder and simplifies debugging (cf. 5.2.5). After all, we do not have enough data on the relative frequencies of different instructions right now, so our code density optimizations are likely to be very approximate at this stage. The ease of debugging and experimenting and the simplicity of implementation are more important at this point.5.3.5. Simple encodings for all instructions
For similar reasons, we opted to use simple encodings for all instructions (cf. 5.2.11 and 5.2.13), with separate simple encodings for some very frequently used subcases as outlined in 5.2.13. That said, we tried to distribute opcode space using the heuristics described in 5.2.9 and 5.2.10.5.3.6. Lack of context-dependent encodings
This version of TVM also does not use context-dependent encodings (cf. 5.1.6). They may be added at a later stage, if deemed useful.5.3.7. The list of all instructions
The list of all instructions available in codepage zero, along with their encodings and (in some cases) short descriptions, may be found in Appendix A.A Instructions and opcodes
This appendix lists all instructions available in the (experimental) codepage zero of TVM, as explained in 5.3. We list the instructions in lexicographical opcode order. However, the opcode space is distributed in such way as to make all instructions in each category (e.g., arithmetic primitives) have neighboring opcodes. So we first list a number of stack manipulation primitives, then constant primitives, arithmetic primitives, comparison primitives, cell primitives, continuation primitives, dictionary primitives, and finally application-specific primitives. We use hexadecimal notation (cf. 1.0) for bitstrings. Stack registers usually have , and is encoded in a 4-bit field (or, on a few rare occasions, in an 8-bit field). Other immediate parameters are usually 4-bit, 8-bit, or variable length. The stack notation described in 2.1.10 is extensively used throughout this appendix.A.1 Gas prices
The gas price for most primitives equals the basic gas price, computed as , where is the instruction length in bits and is the number of cell references included in the instruction. When the gas price of an instruction differs from this basic price, it is indicated in parentheses after its mnemonics, either as (), meaning that the total gas price equals , or as (+), meaning . Apart from integer constants, the following expressions may appear:- — The total price of “reading” cells (i.e., transforming cell references into cell slices). Currently equal to 100 or 25 gas units per cell depending on whether it is the first time a cell with this hash is being “read” during the current run of the VM or not.
- — The total price of loading cells. Depends on the loading action required.
- — The total price of creating new Builders. Currently equal to 0 gas units per builder.
- — The total price of creating new Cells from Builders. Currently equal to 500 gas units per cell.
A.2 Stack manipulation primitives
This section includes both the basic (cf. 2.2.1) and the compound (cf. 2.2.3) stack manipulation primitives, as well as some “unsystematic” ones. Some compound stack manipulation primitives, such as or , turn out to have the same length as an equivalent sequence of simpler operations. We have included these primitives regardless, so that they can easily be allocated shorter opcodes in a future revision of TVM—or removed for good. Some stack manipulation instructions have two mnemonics: one Forth-style (e.g., ), the other conforming to the usual rules for identifiers (e.g., ). Whenever a stack manipulation primitive (e.g., ) accepts an integer parameter from the stack, it must be within the range ; otherwise a range check exception happens before any further checks.A.2 Stack manipulation primitives
A.2.1. Basic stack manipulation primitives
- — , does nothing.
- — , also known as .
- — or , interchanges the top of the stack with , .
- — , , interchanges with .
- — , with .
- — , .
- — , , pushes a copy of the old into the stack.
- — , also known as .
- — , also known as .
- — , , pops the old top-of-stack value into the old .
- — , also known as , discards the top-of-stack value.
- — , also known as .
A.2.2. Compound stack manipulation primitives
Parameters , , and of the following primitives all are 4-bit integers in the range .- — , equivalent to ; ; , with .
- — , equivalent to ; .
- — , equivalent to ; .
- — , equivalent to ; ; .
- — , equivalent to ; .
- — (long form).
- — , equivalent to ; .
- — , equivalent to ; .
- — , equivalent to ; .
- — , equivalent to ; ; .
- — , equivalent to ; .
- — , equivalent to ; ; .
- — , equivalent to ; .
- — unused.
A.2.3. Exotic stack manipulation primitives
- — ,, permutes two blocks and , for . Equivalent to ,; ,0; ,0.
- — or ( — ), rotates the three topmost pairs of stack entries.
- — , rotates the top stack entries. Equivalent to ,.
- — or , rotates the top stack entries in the other direction. Equivalent to ,1.
- — for .
- — for .
- — ( — ), equivalent to or to .
- — or ( — ), equivalent to or to .
- — or ( — ), equivalent to or to .
- — or ( — ), equivalent to ; .
- — or ( — ), equivalent to .
- — or ( — ), equivalent to .
- — ,, reverses the order of for ; equivalent to a sequence of s.
- — , equivalent to performed times.
- — ,, equivalent to performed times, , .
- — or , pops integer from the stack, then performs .
- — , pops integer from the stack, then performs ,.
- — or , pops integer from the stack, then performs ,1.
- — , pops integers , from the stack, then performs ,.
- — , pops integers , from the stack, then performs ,.
- — , pops integer from the stack, then performs .
- — ( - ), equivalent to ; or to .
- — , pops integer from the stack, then performs .
- — , pushes the current depth of the stack.
- — , pops integer from the stack, then checks whether there are at least elements, generating a stack underflow exception otherwise.
- — , pops integer from the stack, then removes all but the top elements.
- — , pops integer from the stack, then leaves only the bottom elements. Approximately equivalent to ; ; ; .
- — — reserved for stack operations.
- — ,, drops stack elements under the top elements, where and . Equivalent to ,0; ; ,0.
A.3 Tuple, List, and Null primitives
Tuples are ordered collections consisting of at most 255 TVM stack values of arbitrary types (not necessarily the same). Tuple primitives create, modify, and unpack Tuples; they manipulate values of arbitrary types in the process, similarly to the stack primitives. We do not recommend using Tuples of more than 15 elements. When a Tuple contains elements , …, (in that order), we write ; number is the length of Tuple . It is also denoted by . Tuples of length two are called pairs, and Tuples of length three are triples. Lisp-style lists are represented with the aid of pairs, i.e., tuples consisting of exactly two elements. An empty list is represented by a Null value, and a non-empty list is represented by pair , where is the first element of the list, and is its tail.A.3.1. Null primitives
The following primitives work with (the only) value of type Null, useful for representing empty lists, empty branches of binary trees, and absence of values in Maybe types. An empty Tuple created by could have been used for the same purpose; however, Null is more efficient and costs less gas.- — or ( — ), pushes the only value of type Null.
- — ( — ), checks whether is a Null, and returns or accordingly.
A.3.2. Tuple primitives
- — ( — ), creates a new Tuple containing values , , , where .
- — ( — ), pushes the only Tuple of length zero.
- — ( — ), creates a singleton , i.e., a Tuple of length one.
- — or ( — ), creates pair .
- — ( — ), creates triple .
- — ( — ), returns the -th element of a Tuple , where . In other words, returns if . If , throws a range check exception.
- — or ( — ), returns the first element of a Tuple.
- — or ( — ), returns the second element of a Tuple.
- — ( — ), returns the third element of a Tuple.
- — ( — ), unpacks a Tuple of length equal to . If is not a Tuple, of if , a type check exception is thrown.
- — ( — ), unpacks a singleton .
- — or ( — ), unpacks a pair .
- — ( — ), unpacks a triple .
- — ( — ), unpacks first elements of a Tuple . If , throws a type check exception.
- — ( — ), checks whether is a Tuple.
- — ( — ), unpacks a Tuple and returns its length , but only if . Otherwise throws a type check exception.
- — ( — ), computes Tuple that differs from only at position , which is set to . In other words, , for , and , for given . If , throws a range check exception.
- — ( — ), sets the first component of Tuple to and returns the resulting Tuple .
- — ( — ), sets the second component of Tuple to and returns the resulting Tuple .
- — ( — ), sets the third component of Tuple to and returns the resulting Tuple .
- — ( — ), returns the -th element of a Tuple , where . In other words, returns if . If , or if is Null, returns a Null instead of .
- — ( — ), sets the -th component of Tuple to , where , and returns the resulting Tuple . If , first extends the original Tuple to length by setting all new components to Null. If the original value of is Null, treats it as an empty Tuple. If is not Null or Tuple, throws an exception. If is Null and either or is Null, then always returns (and does not consume tuple creation gas).
- — ( — ), creates a new Tuple of length similarly to , but with taken from the stack.
- — ( — ), similar to , but with taken from the stack.
- — ( — ), similar to , but with taken from the stack.
- — ( — ), similar to , but with taken from the stack.
- — ( — ), similar to , but with taken from the stack.
- — ( — ), similar to , but with taken from the stack.
- — ( — ), similar to , but with taken from the stack.
- — ( — ), similar to , but with taken from the stack.
- — ( — ), returns the length of a Tuple.
- — ( — or ), similar to , but returns if is not a Tuple.
- — ( — ), returns or depending on whether is a Tuple.
- — ( — ), returns the last element of a non-empty Tuple .
- — or ( — ), appends a value to a Tuple , but only if the resulting Tuple is of length at most 255. Otherwise throws a type check exception.
- — ( — ), detaches the last element from a non-empty Tuple , and returns both the resulting Tuple and the original last element .
- — ( — or ), pushes a Null under the topmost Integer , but only if .
- — ( — or ), pushes a Null under the topmost Integer , but only if . May be used for stack alignment after quiet primitives such as .
- — ( — or ), pushes a Null under the second stack entry from the top, but only if the topmost Integer is non-zero.
- — ( — or ), pushes a Null under the second stack entry from the top, but only if the topmost Integer is zero. May be used for stack alignment after quiet primitives such as .
- — ( — or ), pushes two Nulls under the topmost Integer , but only if . Equivalent to ; .
- — ( — or ), pushes two Nulls under the topmost Integer , but only if . Equivalent to ; .
- — ( — or ), pushes two Nulls under the second stack entry from the top, but only if the topmost Integer is non-zero. Equivalent to ; .
- — ( — or ), pushes two Nulls under the second stack entry from the top, but only if the topmost Integer is zero. Equivalent to ; .
- — , ( — ), recovers for . Equivalent to ; .
- — ( — ), recovers .
- — ( — ), recovers .
- — ,, ( — ), recovers for . Equivalent to ,; .
- — ( — ), recovers .
- — ( — ), recovers .
A.4 Constant, or literal primitives
The following primitives push into the stack one literal (or unnamed constant) of some type and range, stored as a part (an immediate argument) of the instruction. Therefore, if the immediate argument is absent or too short, an “invalid or too short opcode” exception (code 6) is thrown.A.4.1. Integer and boolean constants
- — with , pushes integer into the stack; here equals four lower-order bits of (i.e., ).
- — , , or , pushes a zero.
- — or .
- — or .
- — or 10.
- — or .
- — with .
- — with a signed 16-bit big-endian integer.
- — .
- — , where 5-bit determines the length of signed big-endian integer . The total length of this instruction is bytes or bits.
- — .
- — , (quietly) pushes for .
- — , pushes a .
- — , pushes for .
- — , pushes for .
- , — reserved for integer constants.
A.4.2. Constant slices, continuations, cells, and references
Most of the instructions listed below push literal slices, continuations, cells, and cell references, stored as immediate arguments to the instruction. Therefore, if the immediate argument is absent or too short, an “invalid or too short opcode” exception (code ) is thrown.- — , pushes the first reference of into the stack as a Cell (and removes this reference from the current continuation).
- — , similar to , but converts the cell into a Slice.
- — , similar to , but makes a simple ordinary Continuation out of the cell.
- — , pushes the (prefix) subslice of consisting of its first bits and no references (i.e., essentially a bitstring), where . A completion tag is assumed, meaning that all trailing zeroes and the last binary one (if present) are removed from this bitstring. If the original bitstring consists only of zeroes, an empty slice will be pushed.
- — , pushes an empty slice (bitstring ’ ’).
- — , pushes bitstring .
- — , pushes bitstring .
- — , pushes the (prefix) subslice of consisting of its first references and up to first bits of data, with . A completion tag is also assumed.
- is equivalent to .
- — , pushes the subslice of consisting of references and up to bits of data, with . A completion tag is assumed.
- — unused (reserved).
- — , where is the simple ordinary continuation made from the first references and the first bytes of .
- — , pushes an -byte continuation for .
A.5 Arithmetic primitives
A.5.1. Addition, subtraction, multiplication
- — ( — ), adds together two integers.
- — ( — ).
- — ( — ), equivalent to ; .
- — ( — ), equivalent to or to ; . Notice that it triggers an integer overflow exception if .
- — ( — ), equivalent to .
- — ( — ), equivalent to .
- — ( — ), .
- — ( — ), .
- — ( — ).
A.5.2. Division
The general encoding of a , , or operation is , with an optional pre-multiplication and an optional replacement of the division or multiplication by a shift. Variable one- or two-bit fields , , , , and are as follows:- — Indicates whether there is pre-multiplication ( operation and its variants), possibly replaced by a left shift.
- — Indicates whether either the multiplication or the division have been replaced by shifts: —no replacement, —division replaced by a right shift, —multiplication replaced by a left shift (possible only for ).
- — Indicates whether there is a constant one-byte argument for the shift operator (if ). For , . If , then , and the shift is performed by bits. If and , then the shift amount is provided to the instruction as a top-of-stack Integer in range .
- — Indicates which results of division are required: —only the quotient, —only the remainder, —both.
- — Rounding mode: —floor, —nearest integer, —ceiling (cf. 1.5.6).
- — ( — ).
- — ( — ).
- — ( — ).
- — ( — ), where , .
- — ( — ), where , .
- — ( — ), where , .
- — ( — ), where , .
- — same as : ( — ) for .
- — same as : ( — ).
- — : ( — ).
- — ( — ), where .
- — ( — ), where , . This operation always succeeds for and returns the correct value of , even if the intermediate result or the quotient do not fit into 257 bits.
- — ( — ), where , (same as in Forth).
- — ( — ) for .
- — ( — ) for .
- — ( — ).
- — ( — ).
- — ( — ) for .
- — ( — ) for .
- — ( — ).
- — ( — ).
A.5.3. Shifts, logical operations
- — ( — ), .
- — , equivalent to or to Forth’s .
- — ( — ), .
- — ( — ), .
- — ( — ), .
- — ( — ), , equivalent to ; ; .
- — reserved.
- — ( — ), bitwise “and” of two signed integers and , sign-extended to infinity.
- — ( — ), bitwise “or” of two integers.
- — ( — ), bitwise “xor” of two integers.
- — ( — ), bitwise “not” of an integer.
- — ( — ), checks whether is a -bit signed integer for (i.e., whether ). If not, either triggers an integer overflow exception, or replaces with a (quiet version).
- — or ( — ), checks whether is a “boolean value” (i.e., either 0 or -1).
- — ( — ), checks whether is a -bit unsigned integer for (i.e., whether ).
- — or , checks whether is a binary digit (i.e., zero or one).
- — ( — ), checks whether is a -bit signed integer for .
- — ( — ), checks whether is a -bit unsigned integer for .
- — ( — ), computes smallest such that fits into a -bit signed integer ().
- — ( — ), computes smallest such that fits into a -bit unsigned integer (), or throws a range check exception.
- — ( — or ), computes the minimum of two integers and .
- — ( — or ), computes the maximum of two integers and .
- — or ( — or ), sorts two integers. Quiet version of this operation returns two s if any of the arguments are s.
- — ( — ), computes the absolute value of an integer .
A.5.4. Quiet arithmetic primitives
We opted to make all arithmetic operations “non-quiet” (signaling) by default, and create their quiet counterparts by means of a prefix. Such an encoding is definitely sub-optimal. It is not yet clear whether it should be done in this way, or in the opposite way by making all arithmetic operations quiet by default, or whether quiet and non-quiet operations should be given opcodes of equal length; this can only be settled by practice.- — prefix, transforming any arithmetic operation into its “quiet” variant, indicated by prefixing a
Qto its mnemonic. Such operations return s instead of throwing integer overflow exceptions if the results do not fit in Integers, or if one of their arguments is a . Notice that this does not extend to shift amounts and other parameters that must be within a small range (e.g., 0—1023). Also notice that this does not disable type-checking exceptions if a value of a type other than Integer is supplied. - — ( — ), always works if and are Integers, but returns a if the addition cannot be performed.
- — ( — ), returns the product of and if . Otherwise returns a , even if and is a .
- — ( — ), returns a if , or if and , or if either of or is a .
- — ( — ), where , . If , or if at least one of , , or is a , both and are set to . Otherwise the correct value of is always returned, but is replaced with if or .
- — ( — ), bitwise “and” (similar to ), but returns a if either or is a instead of throwing an integer overflow exception. However, if one of the arguments is zero, and the other is a , the result is zero.
- — ( — ), bitwise “or”. If or , the result is always , even if the other argument is a .
- — ( — ), checks whether is an unsigned byte (i.e., whether ), and replaces with a if this is not the case; leaves intact otherwise (i.e., if is an unsigned byte or a ).
A.6 Comparison primitives
A.6.1. Integer comparison
All integer comparison primitives return integer (“true”) or (“false”) to indicate the result of the comparison. We do not define their “boolean circuit” counterparts, which would transfer control to or depending on the result of the comparison. If needed, such instructions can be simulated with the aid of . Quiet versions of integer comparison primitives are also available, encoded with the aid of the prefix (). If any of the integers being compared are s, the result of a quiet comparison will also be a (“undefined”), instead of a (“yes”) or (“no”), thus effectively supporting ternary logic.- — ( — ), computes the sign of an integer : if , if , if .
- — ( — ), returns if , otherwise.
- — ( — ), returns if , otherwise.
- — ( — ).
- — ( — ).
- — ( — ), equivalent to ; .
- — ( — ), equivalent to ; .
- — ( — ), computes the sign of : if , if , if . No integer overflow can occur here unless or is a .
- — ( — ) for .
- — , checks whether an integer is zero. Corresponds to Forth’s .
- — ( — ) for .
- — , checks whether an integer is negative. Corresponds to Forth’s .
- — , checks whether an integer is non-positive.
- — ( — ) for .
- — , checks whether an integer is positive. Corresponds to Forth’s .
- — , checks whether an integer is non-negative.
- — ( — ) for .
- — ( — ), checks whether is a .
- — ( — ), throws an arithmetic overflow exception if is a .
- — reserved for integer comparison.
A.6.2. Other comparison
Most of these “other comparison” primitives actually compare the data portions of Slices as bitstrings.- — ( — ), checks whether a Slice is empty (i.e., contains no bits of data and no cell references).
- — ( — ), checks whether Slice has no bits of data.
- — ( — ), checks whether Slice has no references.
- — ( — ), checks whether the first bit of Slice is a one.
- — ( — ), compares the data of lexicographically with the data of , returning , 0, or 1 depending on the result.
- — ( — ), checks whether the data parts of and coincide, equivalent to ; .
- — ( — ), checks whether is a prefix of .
- — ( — ), checks whether is a prefix of , equivalent to ; .
- — ( — ), checks whether is a proper prefix of (i.e., a prefix distinct from ).
- — ( — ), checks whether is a proper prefix of .
- — ( — ), checks whether is a suffix of .
- — ( — ), checks whether is a suffix of .
- — ( — ), checks whether is a proper suffix of .
- — ( — ), checks whether is a proper suffix of .
- — ( — ), returns the number of leading zeroes in .
- — ( — ), returns the number of leading ones in .
- — ( — ), returns the number of trailing zeroes in .
- — ( — ), returns the number of trailing ones in .
A.7 Cell primitives
The cell primitives are mostly either cell serialization primitives, which work with Builders, or cell deserialization primitives, which work with Slices.A.7.1. Cell serialization primitives
All these primitives first check whether there is enough space in the Builder, and only then check the range of the value being serialized.- — ( — ), creates a new empty Builder.
- — ( — ), converts a Builder into an ordinary Cell.
- — ( — ), stores a signed -bit integer into Builder for , throws a range check exception if does not fit into bits.
- — ( — ), stores an unsigned -bit integer into Builder . In all other respects it is similar to .
- — ( — ), stores a reference to Cell into Builder .
- — or ( — ), equivalent to ; ; .
- — ( — ), stores Slice into Builder .
- — ( — ), stores a signed -bit integer into for .
- — ( — ), stores an unsigned -bit integer into for .
- — ( — ), similar to , but with arguments in a different order.
- — ( — ), similar to , but with arguments in a different order.
- — ( — or ), a quiet version of . If there is no space in , sets and . If does not fit into bits, sets and . If the operation succeeds, is the new Builder and . However, , with a range check exception if this is not so.
- — ( — ).
- — ( — or ).
- — ( — or ).
- — a longer version of .
- — a longer version of .
- — ( — ), equivalent to ; .
- — ( — ), equivalent to ; .
- — ( — or ).
- — ( — or ).
- — ( — or ).
- — ( — or ).
- — a longer version of ( — ).
- — ( — ), equivalent to ; .
- — a longer version of ( — ).
- — ( — ), appends all data from Builder to Builder .
- — ( — ).
- — ( — ), a longer encoding of .
- — ( — ).
- — ( — ), concatenates two Builders, equivalent to ; .
- — ( — or ).
- — ( — or ).
- — ( — or ).
- — ( — or ).
- — ( — or ).
- — ( — or ).
- — ( — or ).
- — ( — or ).
- — , equivalent to ; .
- — , equivalent to ; .
- — ( — ), if , creates a special or exotic cell (cf. 3.1.2) from Builder . The type of the exotic cell must be stored in the first 8 bits of . If , it is equivalent to . Otherwise some validity checks on the data and references of are performed before creating the exotic cell.
- — ( — ), stores a little-endian signed 32-bit integer.
- — ( — ), stores a little-endian unsigned 32-bit integer.
- — ( — ), stores a little-endian signed 64-bit integer.
- — ( — ), stores a little-endian unsigned 64-bit integer.
- — ( — ), returns the depth of Builder . If no cell references are stored in , then ; otherwise is one plus the maximum of depths of cells referred to from .
- — ( — ), returns the number of data bits already stored in Builder .
- — ( — ), returns the number of cell references already stored in .
- — ( — ), returns the numbers of both data bits and cell references in .
- — ( — ), returns the number of data bits that can still be stored in .
- — ( — ).
- — ( — ).
- — ( —), checks whether bits can be stored into , where .
- — ( — ), checks whether bits can be stored into , . If there is no space for more bits in , or if is not within the range , throws an exception.
- — ( — ), checks whether references can be stored into , .
- — ( — ), checks whether bits and references can be stored into , , .
- — ( — ), checks whether bits can be stored into , where .
- — ( — ), checks whether bits can be stored into , .
- — ( — ), checks whether references can be stored into , .
- — ( — ), checks whether bits and references can be stored into , , .
- — ( — ), stores binary zeroes into Builder .
- — ( — ), stores binary ones into Builder .
- — ( — ), stores binary es () into Builder .
- — ( — ), stores a constant subslice consisting of references and up to data bits, with . Completion bit is assumed.
- — or ( — ), stores one binary zero.
- — or ( — ), stores one binary one.
- — equivalent to .
- — almost equivalent to ; .
- — equivalent to .
- — .
A.7.2. Cell deserialization primitives
- — ( — ), converts a Cell into a Slice. Notice that must be either an ordinary cell, or an exotic cell (cf. 3.1.2) which is automatically loaded to yield an ordinary cell , converted into a Slice afterwards.
- — ( — ), removes a Slice from the stack, and throws an exception if it is not empty.
- — ( — ), loads (i.e., parses) a signed -bit integer from Slice , and returns the remainder of as .
- — ( — ), loads an unsigned -bit integer from Slice .
- — ( — ), loads a cell reference from .
- — ( — ), equivalent to ; ; .
- — ( — ), cuts the next bits of into a separate Slice .
- — ( — ), loads a signed -bit () integer from Slice , and returns the remainder of as .
- — ( — ), loads an unsigned -bit integer from (the first bits of) , with .
- — ( — ), preloads a signed -bit integer from Slice , for .
- — ( — ), preloads an unsigned -bit integer from , for .
- — ( — or ), quiet version of : loads a signed -bit integer from similarly to , but returns a success flag, equal to on success or to on failure (if does not have bits), instead of throwing a cell underflow exception.
- — ( — or ), quiet version of .
- — ( — or ), quiet version of .
- — ( — or ), quiet version of .
- — ( — ), a longer encoding for .
- — ( — ), a longer encoding for .
- — ( — ), preloads a signed -bit integer from Slice .
- — ( — ), preloads an unsigned -bit integer from .
- — ( — or ), a quiet version of .
- — ( — or ), a quiet version of .
- — ( — or ), a quiet version of .
- — ( — or ), a quiet version of .
- — ( — ), preloads the first bits of Slice into an unsigned integer , for . If is shorter than necessary, missing bits are assumed to be zero. This operation is intended to be used along with and similar instructions.
- — ( — ), loads the first bits from Slice into a separate Slice , returning the remainder of as .
- — ( — ), returns the first bits of as .
- — ( — or ), a quiet version of .
- — ( — or ), a quiet version of .
- — ( — ), a longer encoding for .
- — ( — ), returns the first bits of as .
- — ( — or ), a quiet version of .
- — ( — or ), a quiet version of .
- — ( — ), returns the first bits of . It is equivalent to .
- — ( — ), returns all but the first bits of . It is equivalent to ; .
- — ( — ), returns the last bits of .
- — ( — ), returns all but the last bits of .
- — ( — ), returns bits of starting from offset , thus extracting a bit substring out of the data of .
- — ( — ), checks whether begins with (the data bits of) , and removes from on success. On failure throws a cell deserialization exception. Primitive can be considered a quiet version of .
- — ( — or ), a quiet version of .
- — ( — ), checks whether begins with constant bitstring of length (with continuation bit assumed), where , and removes from on success.
- — ( — ), checks whether begins with a binary zero.
- — ( — ), checks whether begins with a binary one.
- — ( — or ), a quiet version of .
- — ( — ), returns the first bits and first references of .
- — ( — ).
- — ( — ), returns the last data bits and last references of .
- — ( — ).
- — ( — ), returns bits and references from Slice , after skipping the first bits and first references.
- — ( — ), splits the first data bits and first references from into , returning the remainder of as .
- — ( — or ), a quiet version of .
- — ( — ), transforms an ordinary or exotic cell into a Slice, as if it were an ordinary cell. A flag is returned indicating whether is exotic. If that be the case, its type can later be deserialized from the first eight bits of .
- — ( — ), loads an exotic cell and returns an ordinary cell . If is already ordinary, does nothing. If cannot be loaded, throws an exception.
- — ( — or ), loads an exotic cell as , but returns 0 on failure.
- — ( — ), checks whether there are at least data bits in Slice . If this is not the case, throws a cell deserialisation (i.e., cell underflow) exception.
- — ( — ), checks whether there are at least references in Slice .
- — ( — ), checks whether there are at least data bits and references in Slice .
- — ( — ), checks whether there are at least data bits in Slice .
- — ( — ), checks whether there are at least references in Slice .
- — ( — ), checks whether there are at least data bits and references in Slice .
- — ( — ), returns the -th cell reference of Slice for .
- — ( — ), returns the number of data bits in Slice .
- — ( — ), returns the number of references in Slice .
- — ( — ), returns both the number of data bits and the number of references in .
- — ( — ), returns the -th cell reference of Slice , where .
- — ( — ), preloads the first cell reference of a Slice.
- — ( — ), loads a little-endian signed 32-bit integer.
- — ( — ), loads a little-endian unsigned 32-bit integer.
- — ( — ), loads a little-endian signed 64-bit integer.
- — ( — ), loads a little-endian unsigned 64-bit integer.
- — ( — ), preloads a little-endian signed 32-bit integer.
- — ( — ), preloads a little-endian unsigned 32-bit integer.
- — ( — ), preloads a little-endian signed 64-bit integer.
- — ( — ), preloads a little-endian unsigned 64-bit integer.
- — ( — or ), quietly loads a little-endian signed 32-bit integer.
- — ( — or ), quietly loads a little-endian unsigned 32-bit integer.
- — ( — or ), quietly loads a little-endian signed 64-bit integer.
- — ( — or ), quietly loads a little-endian unsigned 64-bit integer.
- — ( — or ), quietly preloads a little-endian signed 32-bit integer.
- — ( — or ), quietly preloads a little-endian unsigned 32-bit integer.
- — ( — or ), quietly preloads a little-endian signed 64-bit integer.
- — ( — or ), quietly preloads a little-endian unsigned 64-bit integer.
- — ( — ), returns the count of leading zero bits in , and removes these bits from .
- — ( — ), returns the count of leading one bits in , and removes these bits from .
- — ( — ), returns the count of leading bits equal to in , and removes these bits from .
- — ( — ), returns the depth of Slice . If has no references, then ; otherwise is one plus the maximum of depths of cells referred to from .
- — ( — ), returns the depth of Cell . If has no references, then ; otherwise is one plus the maximum of depths of cells referred to from . If is a Null instead of a Cell, returns zero.
A.8 Continuation and control flow primitives
A.8.1. Unconditional control flow primitives
- — or ( - ), calls or executes continuation (i.e., ).
- — ( - ), jumps, or transfers control, to continuation (i.e., , or rather ). The remainder of the previous current continuation is discarded.
- — , ( - ), calls continuation with parameters and expecting return values, , .
- — , ( - ), calls continuation with parameters, expecting an arbitrary number of return values.
- — ( - ), jumps to continuation , passing only the top values from the current stack to it (the remainder of the current stack is discarded).
- — , returns to , with return values taken from the current stack.
- — or , returns to the continuation at (i.e., performs ). The remainder of the current continuation is discarded. Approximately equivalent to ; .
- — or , returns to the continuation at (i.e., ). Approximately equivalent to ; .
- — or ( - ), performs if integer , or if .
- — ( - ), call with current continuation, transfers control to , pushing the old value of into ‘s stack (instead of discarding it or writing it into new ).
- — ( - ), similar to , but the remainder of the current continuation (the old value of ) is converted into a Slice before pushing it into the stack of .
- — , ( - ), similar to , but pushes the old value of (along with the top values from the original stack) into the stack of newly-invoked continuation , setting to .
- — ( - ), similar to , but takes from the stack. The next three operations also take and from the stack, both in the range .
- — ( - ), similar to .
- — ( - ), similar to .
- — ( - ), similar to .
- — , equivalent to ; .
- — , equivalent to ; .
- — , equivalent to ; .
- — , equivalent to ; . In this way, the remainder of the current continuation is converted into a Slice and returned to the caller.
A.8.2. Conditional control flow primitives
- — ( - ), performs a , but only if integer is non-zero. If is a , throws an integer overflow exception.
- — ( - ), performs a , but only if integer is zero.
- — ( - ), performs for (i.e., executes ), but only if integer is non-zero. Otherwise simply discards both values.
- — ( - ), executes continuation , but only if integer is zero. Otherwise simply discards both values.
- — ( - ), jumps to (similarly to ), but only if is non-zero.
- — ( - ), jumps to (similarly to ), but only if is zero.
- — ( - ), if integer is non-zero, executes , otherwise executes . Equivalent to ; .
- — ( - ), equivalent to ; , with the optimization that the cell reference is not actually loaded into a Slice and then converted into an ordinary Continuation if . Similar remarks apply to the next three primitives.
- — ( - ), equivalent to ; .
- — ( - ), equivalent to ; .
- — ( - ), equivalent to ; .
- — ( - or ), if integer is non-zero, returns , otherwise returns . Notice that no type checks are performed on and ; as such, it is more like a conditional stack operation. Roughly equivalent to ; ; ; ; .
- — ( - or ), same as , but first checks whether and have the same type.
- — ( -), performs if integer .
- — ( -), performs if integer .
- — ( -), equivalent to ; ; , with the optimization that the cell reference is not actually loaded into a Slice and then converted into an ordinary Continuation if . Similar remarks apply to the next two primitives: Cells are converted into Continuations only when necessary.
- — ( -), equivalent to ; .
- — ( -), equivalent to ; ; .
- — — reserved for loops with break operators, cf. A.8.4 below.
- — ( - ), checks whether bit is set in integer , and if so, performs to continuation . Value is left in the stack.
- — ( - ), jumps to if bit is not set in integer .
- — ( - ), performs a if bit is set in integer .
- — ( - ), performs a if bit is not set in integer .
A.8.3. Control flow primitives: loops
Most of the loop primitives listed below are implemented with the aid of extraordinary continuations, such as (cf. 4.1.5), with the loop body and the original current continuation stored as the arguments to this extraordinary continuation. Typically a suitable extraordinary continuation is constructed, and then saved into the loop body continuation savelist as ; after that, the modified loop body continuation is loaded into and executed in the usual fashion. All of these loop primitives have versions, adapted for breaking out of a loop; they additionally set to the original current continuation (or original for versions), and save the old into the savelist of the original current continuation (or of the original for versions).- — ( - ), executes continuation times, if integer is non-negative. If or , generates a range check exception. Notice that a inside the code of works as a , not as a . One should use either alternative (experimental) loops or alternative (along with a before the loop) to out of a loop.
- — ( - ), similar to , but it is applied to the current continuation .
- — ( - ), executes continuation , then pops an integer from the resulting stack. If is zero, performs another iteration of this loop. The actual implementation of this primitive involves an extraordinary continuation (cf. 4.1.5) with its arguments set to the body of the loop (continuation ) and the original current continuation . This extraordinary continuation is then saved into the savelist of as . and the modified is then executed. The other loop primitives are implemented similarly with the aid of suitable extraordinary continuations.
- — ( - ), similar to , but executes the current continuation in a loop. When the loop exit condition is satisfied, performs a .
- — ( - ), executes and pops an integer from the resulting stack. If is zero, exists the loop and transfers control to the original . If is non-zero, executes , and then begins a new iteration.
- — ( - ), similar to , but uses the current continuation as the loop body.
- — ( - ), similar to , but executes infinitely many times. A only begins a new iteration of the infinite loop, which can be exited only by an exception, or a (or an explicit ).
- — ( - ), similar to , but performed with respect to the current continuation .
- — ( - ), similar to , but also sets to the original after saving the old value of into the savelist of the original . In this way could be used to break out of the loop body.
- — ( - ), similar to , but also sets to the original after saving the old value of into the savelist of the original . Equivalent to ; .
- — ( - ), similar to , but also modifies in the same way as .
- — ( - ), equivalent to ; .
- — ( - ), similar to , but also modifies in the same way as .
- — ( - ), equivalent to ; .
- — ( - ), similar to , but also modifies in the same way as .
- — ( - ), equivalent to ; .
A.8.4. Manipulating the stack of continuations
- — , ( - ), similar to , but sets . to the final size of the stack of plus . In other words, transforms into a closure or a partially applied function, with arguments missing.
- — or , ( - ), sets . to plus the current depth of ‘s stack, where . If . is already set to a non-negative value, does nothing.
- — or , ( - ), pushes values into the stack of (a copy of) the continuation , starting with . If the final depth of ‘s stack turns out to be greater than ., a stack overflow exception is generated.
- — ( - ), leaves only the top values in the current stack (somewhat similarly to ), with all the unused bottom values not discarded, but saved into continuation in the same way as does.
- — ( - ), similar to , but with Integer taken from the stack.
- — ( - ), similar to , but with and taken from the stack.
- — ( - ), where . If , this operation does nothing (). Otherwise its action is similar to , but with taken from the stack.
A.8.5. Creating simple continuations and closures
- — ( - ), transforms a Slice into a simple ordinary continuation , with . and an empty stack and savelist.
- — ( - ), equivalent to ; ; ; .
- — ( - ), where , , equivalent to ; . The value of is represented inside the instruction by the 4-bit integer .
- — or , ( - ), also transforms a Slice into a Continuation , but sets . to .
A.8.6. Operations with continuation savelists and control registers
- — or ( - ), pushes the current value of control register . If the control register is not supported in the current codepage, or if it does not have a value, an exception is triggered.
- — or , pushes the “global data root” cell reference, thus enabling access to persistent smart-contract data.
- — or ( - ), pops a value from the stack and stores it into control register , if supported in the current codepage. Notice that if a control register accepts only values of a specific type, a type-checking exception may occur.
- — or , sets the “global data root” cell reference, thus allowing modification of persistent smart-contract data.
- — or ( - ), stores into the savelist of continuation as , and returns the resulting continuation . Almost all operations with continuations may be expressed in terms of , , and .
- — ( - ), equivalent to ; ; .
- — ( - ), equivalent to ; ; .
- — or ( -), similar to , but also saves the old value of into continuation . Equivalent (up to exceptions) to ; .
- — or ( - ), saves the current value of into the savelist of continuation . If an entry for is already present in the savelist of , nothing is done. Equivalent to ; .
- — or ( - ), similar to , but saves the current value of into the savelist of , not .
- — or ( - ), equivalent to ; ; .
- — ( - ), similar to , but with , , taken from the stack. Notice that this primitive is one of the few “exotic” primitives, which are not polymorphic like stack manipulation primitives, and at the same time do not have well-defined types of parameters and return values, because the type of depends on .
- — ( - ), similar to , but with from the stack.
- — ( - ), similar to , but with from the stack.
- — or ( - ), computes the composition , which has the meaning of “perform , and, if successful, perform ” (if is a boolean circuit) or simply “perform , then ”. Equivalent to ; .
- — or ( - ), computes the alternative composition , which has the meaning of “perform , and, if not successful, perform ” (if is a boolean circuit). Equivalent to ; .
- — ( - ), computes , which has the meaning of “compute boolean circuit , then compute , regardless of the result of ”.
- — ( - ), sets . In other words, will be executed before exiting current subroutine.
- — ( - ), sets . In other words, will be executed before exiting current subroutine by its alternative return path.
- — ( - ), sets . In this way, a subsequent will first execute , then transfer control to the original . This can be used, for instance, to exit from nested loops.
- — ( - ), computes
- — ( - ), computes
- — ( - ), interchanges and .
- — ( - ), performs . If represents a boolean circuit, the net effect is to evaluate it and push either or into the stack before continuing.
- — ( - ), sets . Equivalent to ; .
- — ( - ), sets , but first saves the old value of into the savelist of . Equivalent to ; .
- — ( — ), described in A.8.4.
A.8.7. Dictionary subroutine calls and jumps
- — or ( - ), calls the continuation in , pushing integer into its stack as an argument. Approximately equivalent to ; ; .
- — for ( - ), an encoding of for larger values of .
- — or ( - ), jumps to the continuation in , pushing integer as its argument. Approximately equivalent to ; ; .
- — or ( - ), equivalent to ; , for . In this way, is approximately equivalent to ; , and is approximately equivalent to ; . One might use, for instance, or instead of here.
A.9 Exception generating and handling primitives
A.9.1. Throwing exceptions
- — ( - ), throws exception with parameter zero. In other words, it transfers control to the continuation in , pushing and into its stack, and discarding the old stack altogether.
- — ( - ), throws exception with parameter zero only if integer .
- — ( - ), throws exception with parameter zero only if integer .
- — for , an encoding of for larger values of .
- — ( - ), throws exception with parameter , by copying and into the stack of and transferring control to .
- — ( - ) for .
- — ( - ), throws exception with parameter only if integer .
- — ( - ) for .
- — ( - ), throws exception with parameter only if integer .
- — ( - ), throws exception with parameter zero. Approximately equivalent to ; ; .
- — ( - ), throws exception with parameter , transferring control to the continuation in . Approximately equivalent to ; .
- — ( - ), throws exception with parameter zero only if .
- — ( - ), throws exception with parameter only if .
- — ( - ), throws exception with parameter zero only if .
- — ( - ), throws exception with parameter only if .
A.9.2. Catching and handling exceptions
- — ( - ), sets to , first saving the old value of both into the savelist of and into the savelist of the current continuation, which is stored into . and .. Then runs similarly to . If does not throw any exceptions, the original value of is automatically restored on return from . If an exception occurs, the execution is transferred to , but the original value of is restored in the process, so that can re-throw the exception by if it cannot handle it by itself.
- — , ( - ), similar to , but with , internally used instead of . In this way, all but the top stack elements will be saved into current continuation’s stack, and then restored upon return from either or , with the top values of the resulting stack of or copied as return values.
A.10 Dictionary manipulation primitives
TVM’s dictionary support is discussed at length in 3.3. The basic operations with dictionaries are listed in 3.3.10, while the taxonomy of dictionary manipulation primitives is provided in 3.3.11. Here we use the concepts and notation introduced in those sections. Dictionaries admit two different representations as TVM stack values:- A Slice with a serialization of a TL-B value of type . In other words, consists either of one bit equal to zero (if the dictionary is empty), or of one bit equal to one and a reference to a Cell containing the root of the binary tree, i.e., a serialized value of type .
- A “maybe Cell” , i.e., a value that is either a Cell (containing a serialized value of type as before) or a Null (corresponding to an empty dictionary). When a “maybe Cell” is used to represent a dictionary, we usually denote it by in the stack notation.
A.10.1. Dictionary creation
- — ( - ), returns a new empty dictionary. It is an alternative mnemonics for , cf. A.3.1.
- — ( - ), checks whether dictionary is empty, and returns or accordingly. It is an alternative mnemonics for , cf. A.3.1.
A.10.2. Dictionary serialization and deserialization
- — ( - ), stores a Slice-represented dictionary into Builder . It is actually a synonym for .
- — or ( - ), stores dictionary into Builder , returning the resulting Builder . In other words, if is a cell, performs and ; if is Null, performs and ; otherwise throws a type checking exception.
- — or ( - ), equivalent to ; .
- — ( - ), loads (parses) a (Slice-represented) dictionary from Slice , and returns the remainder of as . This is a “split function” for all dictionary types.
- — ( - ), preloads a (Slice-represented) dictionary from Slice . Approximately equivalent to ; .
- — or ( - ), loads (parses) a dictionary from Slice , and returns the remainder of as . May be applied to dictionaries or to values of arbitrary ) types.
- — or ( - ), preloads a dictionary from Slice . Approximately equivalent to ; .
- — ( - or ), a quiet version of .
- — ( - or ), a quiet version of .
A.10.3. dictionary operations
- — ( - or ), looks up key (represented by a Slice, the first data bits of which are used as a key) in dictionary of type with -bit keys. On success, returns the value found as a Slice .
- — ( - or ), similar to , but with a ; applied to on success. This operation is useful for dictionaries of type ).
- — ( - or ), similar to , but with a signed (big-endian) -bit Integer as a key. If does not fit into bits, returns . If is a , throws an integer overflow exception.
- — ( - or ), combines with : it uses signed -bit Integer as a key and returns a Cell instead of a Slice on success.
- — ( - or ), similar to , but with unsigned (big-endian) -bit Integer used as a key.
- — ( - or ), similar to , but with an unsigned -bit Integer key .
A.10.4. // dictionary operations
The mnemonics of the following dictionary primitives are constructed in a systematic fashion according to the regular expression depending on the type of the key used (a Slice or a signed or unsigned Integer), the dictionary operation to be performed, and the way the values are accepted and returned (as Cells or as Slices). Therefore, we provide a detailed description only for some primitives, assuming that this information is sufficient for the reader to understand the precise action of the remaining primitives.- — ( - ), sets the value associated with -bit key (represented by a Slice as in ) in dictionary (also represented by a Slice) to value (again a Slice), and returns the resulting dictionary as .
- — ( - ), similar to , but with the value set to a reference to Cell .
- — ( - ), similar to , but with the key represented by a (big-endian) signed -bit integer . If does not fit into bits, a range check exception is generated.
- — ( - ), similar to , but with the key a signed -bit integer as in .
- — ( - ), similar to , but with an unsigned -bit integer.
- — ( - ), similar to , but with unsigned.
- — ( - or ), combines with : it sets the value corresponding to key to , but also returns the old value associated with the key in question, if present.
- — ( - or ), combines with similarly to .
- — ( - or ), similar to , but with the key represented by a big-endian signed -bit Integer .
- — ( - or ), a version of with signed Integer as a key.
- — ( - or ), similar to , but with an unsigned -bit integer.
- — ( - or ).
- — ( - or ), a operation, which is similar to , but sets the value of key in dictionary to only if the key was already present in .
- — ( - or ), a counterpart of .
- — ( - or ), a version of with signed -bit Integer used as a key.
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ), a counterpart of : on success, also returns the old value associated with the key in question.
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ), an Add counterpart of : sets the value associated with key in dictionary to , but only if it is not already present in .
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ), an Add counterpart of : sets the value associated with key in dictionary to , but only if key is not already present in . Otherwise, just returns the old value without changing the dictionary.
- — ( - or ), an Add counterpart of .
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
A.10.5. Builder-accepting variants of dictionary operations
The following primitives accept the new value as a Builder instead of a Slice , which often is more convenient if the value needs to be serialized from several components computed in the stack. (This is reflected by appending a to the mnemonics of the corresponding primitives that work with Slices.) The net effect is roughly equivalent to converting into a Slice by ; and executing the corresponding primitive listed in A.10.4.- — ( - ).
- — ( - ).
- — ( - ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
A.10.6. dictionary operations
- — ( - or ), deletes -bit key, represented by a Slice , from dictionary . If the key is present, returns the modified dictionary and the success flag . Otherwise, returns the original dictionary and .
- — ( - ), a version of with the key represented by a signed -bit Integer . If does not fit into bits, simply returns (“key not found, dictionary unmodified”).
- — ( - ), similar to , but with an unsigned -bit integer.
- — ( - or ), deletes -bit key, represented by a Slice , from dictionary . If the key is present, returns the modified dictionary , the original value associated with the key (represented by a Slice), and the success flag . Otherwise, returns the original dictionary and .
- — ( - or ), similar to , but with ; applied to on success, so that the value returned is a Cell.
- — ( - or ), a variant of primitive with signed -bit integer as a key.
- — ( - or ), a variant of primitive returning a Cell instead of a Slice.
- — ( - or ), a variant of primitive with unsigned -bit integer as a key.
- — ( - or ), a variant of primitive returning a Cell instead of a Slice.
A.10.7. “Maybe reference” dictionary operations
The following operations assume that a dictionary is used to store values of type (“Maybe Cell”), which can be used in particular to store dictionaries as values in other dictionaries. The representation is as follows: if is a Cell, it is stored as a value with no data bits and exactly one reference to this Cell. If is Null, then the corresponding key must be absent from the dictionary altogether.- — ( - ), a variant of that returns Null instead of the value if the key is absent from dictionary .
- — ( - ), similar to , but with the key given by signed -bit Integer . If the key is out of range, also returns Null.
- — ( - ), similar to , but with the key given by unsigned -bit Integer .
- — ( - ), a variant of both and that sets the value corresponding to key in dictionary to (if is Null, then the key is deleted instead), and returns the old value (if the key was absent before, returns Null instead).
- — ( - ), similar to primitive , but using signed -bit Integer as a key. If does not fit into bits, throws a range checking exception.
- — ( - ), similar to primitive , but using unsigned -bit Integer as a key.
A.10.8. Prefix code dictionary operations
These are some basic operations for constructing prefix code dictionaries (cf 3.4.2). The primary application for prefix code dictionaries is deserializing TL-B serialized data structures, or, more generally, parsing prefix codes. Therefore, most prefix code dictionaries will be constant and created at compile time, not by the following primitives. Some operations for prefix code dictionaries may be found in A.10.11. Other prefix code dictionary operations include:- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
A.10.9. Variants of and operations
- — ( - or ), computes the minimal key in dictionary that is lexicographically greater than , and returns (represented by a Slice) along with associated value (also represented by a Slice).
- — ( - or ), similar to , but computes the minimal key that is lexicographically greater than or equal to .
- — ( - or ), similar to , but computes the maximal key lexicographically smaller than .
- — ( - or ), similar to , but computes the maximal key lexicographically smaller than or equal to .
- — ( - or ), similar to , but interprets all keys in dictionary as big-endian signed -bit integers, and computes the minimal key that is larger than Integer (which does not necessarily fit into bits).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ), similar to , but interprets all keys in dictionary as big-endian unsigned -bit integers, and computes the minimal key that is larger than Integer (which does not necessarily fit into bits, and is not necessarily non-negative).
- — ( - or ).
- — ( - or ).
- — ( - or ).
A.10.10. , , , operations
- — ( - or ), computes the minimal key (represented by a Slice with data bits) in dictionary , and returns along with the associated value .
- — ( - or ), similar to , but returns the only reference in the value as a Cell .
- — ( - or ), somewhat similar to , but computes the minimal key under the assumption that all keys are big-endian signed -bit integers. Notice that the key and value returned may differ from those computed by and .
- — ( - or ).
- — ( - or ), similar to , but returns the key as an unsigned -bit Integer .
- — ( - or ).
- — ( - or ), computes the maximal key (represented by a Slice with data bits) in dictionary , and returns along with the associated value .
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ), computes the minimal key (represented by a Slice with data bits) in dictionary , removes from the dictionary, and returns along with the associated value and the modified dictionary .
- — ( - or ), similar to , but returns the only reference in the value as a Cell .
- — ( - or ), somewhat similar to , but computes the minimal key under the assumption that all keys are big-endian signed -bit integers. Notice that the key and value returned may differ from those computed by and .
- — ( - or ).
- — ( - or ), similar to , but returns the key as an unsigned -bit Integer .
- — ( - or ).
- — ( - or ), computes the maximal key (represented by a Slice with data bits) in dictionary , removes from the dictionary, and returns along with the associated value and the modified dictionary .
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
- — ( - or ).
A.10.11. Special dictionary and prefix code dictionary operations, and constant dictionaries
- — ( - ), similar to (cf. A.10.12), but with ed into a continuation with a subsequent to it on success. On failure, does nothing. This is useful for implementing / constructions.
- — ( - ), similar to , but performs instead of .
- — ( - ), similar to , but with instead of .
- — ( - ), similar to , but with instead of .
- — ( - ), pushes a non-empty constant dictionary (as a ) along with its key length , stored as a part of the instruction. The dictionary itself is created from the first of remaining references of the current continuation. In this way, the complete instruction can be obtained by first serializing , then the non-empty dictionary itself (one bit and a cell reference), and then the unsigned 10-bit integer (as if by a instruction). An empty dictionary can be pushed by a primitive (cf. A.10.1) instead.
- — ( - or ), looks up the unique prefix of Slice present in the prefix code dictionary represented by and . If found, the prefix of is returned as , and the corresponding value (also a Slice) as . The remainder of is returned as a Slice . If no prefix of is a key in prefix code dictionary , returns the unchanged and a zero flag to indicate failure.
- — ( - ), similar to , but throws a cell deserialization failure exception on failure.
- — ( - or ), similar to , but on success es the value into a Continuation and transfers control to it as if by a . On failure, returns unchanged and continues execution.
- — ( - ), similar to , but utes the continuation found instead of jumping to it. On failure, throws a cell deserialization exception.
- — or ( - or ), combines for with .
- — ( - or nothing), a variant of that returns index on failure.
- — ( - or nothing), a variant of that returns index on failure.
- — ( - or nothing), a variant of that returns index on failure.
- — ( - or nothing), a variant of that returns index on failure.
A.10.12. dictionary operations
- — ( - ), constructs a subdictionary consisting of all keys beginning with prefix (represented by a Slice, the first data bits of which are used as a key) of length in dictionary of type with -bit keys. On success, returns the new subdictionary of the same type as a Slice .
- — ( - ), variant of with the prefix represented by a signed big-endian -bit Integer , where necessarily .
- — ( - ), variant of with the prefix represented by an unsigned big-endian -bit Integer , where necessarily .
- — ( - ), similar to , but removes the common prefix from all keys of the new dictionary , which becomes of type .
- — ( - ), variant of with the prefix represented by a signed big-endian -bit Integer , where necessarily .
- — ( - ), variant of with the prefix represented by an unsigned big-endian -bit Integer , where necessarily .
- – — used by primitives in A.10.11.
A.11 Application-specific primitives
Opcode range … is reserved for the application-specific primitives. When TVM is used to execute TON Blockchain smart contracts, these application-specific primitives are in fact TON Blockchain-specific.A.11.1. External actions and access to blockchain configuration data
Some of the primitives listed below pretend to produce some externally visible actions, such as sending a message to another smart contract. In fact, the execution of a smart contract in TVM never has any effect apart from a modification of the TVM state. All external actions are collected into a linked list stored in special register (“output actions”). Additionally, some primitives use the data kept in the first component of the Tuple stored in (“root of temporary data” cf 1.3.2). Smart contracts are free to modify any other data kept in the cell , provided the first reference remains intact (otherwise some application-specific primitives would be likely to throw exceptions when invoked). Most of the primitives listed below use 16-bit opcodes.A.11.2. Gas-related primitives
Of the following primitives, only the first two are “pure” in the sense that they do not use or .- — , sets current gas limit to its maximal allowed value , and resets the gas credit to zero (cf. 1.4), decreasing the value of by in the process. In other words, the current smart contract agrees to buy some gas to finish the current transaction. This action is required to process external messages, which bring no value (hence no gas) with themselves.
- — ( - ), sets current gas limit to the minimum of and , and resets the gas credit to zero. If the gas consumed so far (including the present instruction) exceeds the resulting value of , an (unhandled) out of gas exception is thrown before setting new gas limits. Notice that with an argument is equivalent to .
- — ( - ), computes the amount of gas that can be bought for nanograms, and sets accordingly in the same way as .
- — ( - ), computes the amount of gas that can be bought for nanograms. If is negative, returns 0. If exceeds , it is replaced with this value.
- — ( - ), computes the price of gas in nanograms.
- – — Reserved for gas-related primitives.
- — ( - ), commits the current state of registers (“persistent data”) and (“actions”) so that the current execution is considered “successful” with the saved values even if an exception is thrown later.
A.11.3. Pseudo-random number generator primitives
The pseudo-random number generator uses the random seed (parameter #6, cf. A.11.4), an unsigned 256-bit Integer, and (sometimes) other data kept in . The initial value of the random seed before a smart contract is executed in TON Blockchain is a hash of the smart contract address and the global block random seed. If there are several runs of the same smart contract inside a block, then all of these runs will have the same random seed. This can be fixed, for example, by running ; before using the pseudo-random number generator for the first time.- — ( - ), generates a new pseudo-random unsigned 256-bit Integer . The algorithm is as follows: if is the old value of the random seed, considered as a 32-byte array (by constructing the big-endian representation of an unsigned 256-bit integer), then its is computed; the first 32 bytes of this hash are stored as the new value of the random seed, and the remaining 32 bytes are returned as the next random value .
- — ( - ), generates a new pseudo-random integer in the range (or , if ). More precisely, an unsigned random value is generated as in ; then is computed. Equivalent to ; .
- — ( - ), sets the random seed to unsigned 256-bit Integer .
- — ( - ), mixes unsigned 256-bit Integer into the random seed by setting the random seed to SHA-256 of the concatenation of two 32-byte strings: the first with the big-endian representation of the old seed , and the second with the big-endian representation of .
- – — Reserved for pseudo-random number generator primitives.
A.11.4. Configuration primitives
The following primitives read configuration data provided in the Tuple stored in the first component of the Tuple at . Whenever TVM is invoked for executing TON Blockchain smart contracts, this Tuple is initialized by a SmartContractInfo structure; configuration primitives assume that it has remained intact.- — ( - ), returns the -th parameter from the Tuple provided at for . Equivalent to ; ; . If one of these internal operations fails, throws an appropriate type checking or range checking exception.
- — ( - ), returns the current Unix time as an Integer. If it is impossible to recover the requested value starting from , throws a type checking or range checking exception as appropriate. Equivalent to .
- — ( - ), returns the starting logical time of the current block. Equivalent to .
- — ( - ), returns the logical time of the current transaction. Equivalent to .
- — ( - ), returns the current random seed as an unsigned 256-bit Integer. Equivalent to .
- — ( - ), returns the remaining balance of the smart contract as a Tuple consisting of an Integer (the remaining Gram balance in nanograms) and a Maybe Cell (a dictionary with 32-bit keys representing the balance of “extra currencies”). Equivalent to . Note that primitives such as do not update this field.
- — ( - ), returns the internal address of the current smart contract as a Slice with a . If necessary, it can be parsed further using primitives such as or . Equivalent to .
- — ( - ), returns the Maybe Cell with the current global configuration dictionary. Equivalent to .
- — ( - ), returns the global configuration dictionary along with its key length (32). Equivalent to ; .
- — ( - or ), returns the value of the global configuration parameter with integer index as a Cell , and a flag to indicate success. Equivalent to ; .
- — ( - ), returns the value of the global configuration parameter with integer index as a Maybe Cell . Equivalent to ; .
- - — Reserved for configuration primitives.
A.11.5. Global variable primitives
The “global variables” may be helpful in implementing some high-level smart-contract languages. They are in fact stored as components of the Tuple at : the -th global variable simply is the -th component of this Tuple, for . By convention, the -th component is used for the “configuration parameters” of A.11.4, so it is not available as a global variable.- — ( - ), returns the -th global variable for . Equivalent to ; ; (cf. A.3.2).
- — ( - ), returns the -th global variable for . Equivalent to ; .
- — ( - ), assigns to the -th global variable for . Equivalent to ; ; ; .
- — ( - ), assigns to the -th global variable for . Equivalent to ; ; ; .
A.11.6. Hashing and cryptography primitives
- — ( - ), computes the representation hash (cf. 3.1.5) of a Cell and returns it as a 256-bit unsigned integer . Useful for signing and checking signatures of arbitrary entities represented by a tree of cells.
- — ( - ), computes the hash of a Slice and returns it as a 256-bit unsigned integer . The result is the same as if an ordinary cell containing only data and references from had been created and its hash computed by .
- — ( - ), computes SHA-256 of the data bits of Slice . If the bit length of is not divisible by eight, throws a cell underflow exception. The hash value is returned as a 256-bit unsigned integer .
- — ( - ), checks the Ed25519-signature of a hash (a 256-bit unsigned integer, usually computed as the hash of some data) using public key (also represented by a 256-bit unsigned integer). The signature must be a Slice containing at least 512 data bits; only the first 512 bits are used. The result is if the signature is valid, otherwise. Notice that is equivalent to ; ; ; ; ; ; , i.e., to with the first argument set to 256-bit Slice containing . Therefore, if is computed as the hash of some data, these data are hashed twice, the second hashing occurring inside .
- — ( - ), checks whether is a valid Ed25519-signature of the data portion of Slice using public key , similarly to . If the bit length of Slice is not divisible by eight, throws a cell underflow exception. The verification of Ed25519 signatures is the standard one, with SHA-256 used to reduce to the 256-bit number that is actually signed.
- - — Reserved for hashing and cryptography primitives.
A.11.7. Miscellaneous primitives
- — ( - or ), recursively computes the count of distinct cells , data bits , and cell references in the dag rooted at Cell , effectively returning the total storage used by this dag taking into account the identification of equal cells. The values of , , and are computed by a depth-first traversal of this dag, with a hash table of visited cell hashes used to prevent visits of already-visited cells. The total count of visited cells cannot exceed non-negative Integer ; otherwise the computation is aborted before visiting the -st cell and a zero is returned to indicate failure. If is Null, returns .
- — ( - ), a non-quiet version of that throws a cell overflow exception (8) on failure.
- — ( - or ), similar to , but accepting a Slice instead of a Cell. The returned value of does not take into account the cell that contains the slice itself; however, the data bits and the cell references of are accounted for in and .
- — ( - ), a non-quiet version of that throws a cell overflow exception (8) on failure.
- - — Reserved for miscellaneous TON-specific primitives that do not fall into any other specific category.
A.11.8. Currency manipulation primitives
- — or ( - ), loads (deserializes) a or amount from CellSlice , and returns the amount as Integer along with the remainder of . The expected serialization of consists of a 4-bit unsigned big-endian integer , followed by an -bit unsigned big-endian representation of . The net effect is approximately equivalent to ; ; ; .
- — ( - ), similar to , but loads a signed Integer . Approximately equivalent to ; ; ; .
- — or ( - ), stores (serializes) an Integer in the range into Builder , and returns the resulting Builder . The serialization of consists of a 4-bit unsigned big-endian integer , which is the smallest integer , such that , followed by an -bit unsigned big-endian representation of . If does not belong to the supported range, a range check exception is thrown.
- — ( - ), similar to , but serializes a signed Integer in the range .
- — ( - ), loads (deserializes) a from CellSlice , and returns the deserialized value as an Integer . The expected serialization of consists of a 5-bit unsigned big-endian integer , followed by an -bit unsigned big-endian representation of . The net effect is approximately equivalent to ; ; ; .
- — ( - ), deserializes a from CellSlice , and returns the deserialized value as an Integer .
- — ( - ), serializes an Integer as a .
- — ( - ), serializes an Integer as a .
- - — Reserved for currency manipulation primitives.
A.11.9. Message and address manipulation primitives
The message and address manipulation primitives listed below serialize and deserialize values according to the following TL-B scheme (cf. 3.3.4):- is represented by , i.e., a Tuple containing exactly one Integer equal to zero.
- is represented by , where Slice contains the field . In other words, is a pair (a Tuple consisting of two entries), containing an Integer equal to one and Slice .
- is represented by , where is either a Null (if is absent) or a Slice containing (if is present). Next, Integer is the , and Slice contains the .
- is represented by , where , , and have the same meaning as for .
- — ( - ), loads from CellSlice the only prefix that is a valid , and returns both this prefix and the remainder of as CellSlices.
- — ( - or ), a quiet version of : on success, pushes an extra ; on failure, pushes the original and a zero.
- — ( - ), decomposes CellSlice containing a valid into a Tuple with separate fields of this . If is not a valid , a cell deserialization exception is thrown.
- — ( - or ), a quiet version of : returns a zero on error instead of throwing an exception.
- — ( - ), parses CellSlice containing a valid (usually a ), applies rewriting from the (if present) to the same-length prefix of the address, and returns both the workchain and the 256-bit address as Integers. If the address is not 256-bit, or if is not a valid serialization of , throws a cell deserialization exception.
- — ( - or ), a quiet version of primitive .
- — ( - ), a variant of that returns the (rewritten) address as a Slice s, even if it is not exactly 256 bit long (represented by a ).
- — ( - or ), a quiet version of primitive .
- - — Reserved for message and address manipulation primitives.
A.11.10. Outbound message and output action primitives
- — ( - ), sends a raw message contained in Cell , which should contain a correctly serialized object , with the only exception that the source address is allowed to have dummy value (to be automatically replaced with the current smart-contract address), and , , and fields can have arbitrary values (to be rewritten with correct values during the action phase of the current transaction). Integer parameter contains the flags. Currently is used for ordinary messages; is used for messages that are to carry all the remaining balance of the current smart contract (instead of the value originally indicated in the message); is used for messages that carry all the remaining value of the inbound message in addition to the value initially indicated in the new message (if bit 0 is not set, the gas fees are deducted from this amount); means that the sender wants to pay transfer fees separately; means that any errors arising while processing this message during the action phase should be ignored. Finally, means that the current account must be destroyed if its resulting balance is zero. This flag is usually employed together with .
- — ( - ), creates an output action which would reserve exactly nanograms (if ), at most nanograms (if ), or all but nanograms (if or ), from the remaining balance of the account. It is roughly equivalent to creating an outbound message carrying nanograms (or nanograms, where is the remaining balance) to oneself, so that the subsequent output actions would not be able to spend more money than the remainder. Bit in means that the external action does not fail if the specified amount cannot be reserved; instead, all remaining balance is reserved. Bit in means before performing any further actions. Bit in means that is increased by the original balance of the current account (before the compute phase), including all extra currencies, before performing any other checks and actions. Currently must be a non-negative integer, and must be in the range .
- — ( - ), similar to , but also accepts a dictionary (represented by a Cell or Null) with extra currencies. In this way currencies other than Grams can be reserved.
- — ( - ), creates an output action that would change this smart contract code to that given by Cell . Notice that this change will take effect only after the successful termination of the current run of the smart contract.
- — ( - ), creates an output action that would modify the collection of this smart contract libraries by adding or removing library with code given in Cell . If , the library is actually removed if it was previously present in the collection (if not, this action does nothing). If , the library is added as a private library, and if , the library is added as a public library (and becomes available to all smart contracts if the current smart contract resides in the masterchain); if the library was present in the collection before, its public/private status is changed according to . Values of other than are invalid.
- — ( - ), creates an output action similarly to , but instead of the library code accepts its hash as an unsigned 256-bit integer . If and the library with hash is absent from the library collection of this smart contract, this output action will fail.
- - — Reserved for output action primitives.
A.12 Debug primitives
Opcodes beginning with are reserved for the debug primitives. These primitives have known fixed operation length, and behave as (multibyte) NOP operations. In particular, they never change the stack contents, and never throw exceptions, unless there are not enough bits to completely decode the opcode. However, when invoked in a TVM instance with debug mode enabled, these primitives can produce specific output into the text debug log of the TVM instance, never affecting the TVM state (so that from the perspective of TVM the behavior of debug primitives in debug mode is exactly the same). For instance, a debug primitive might dump all or some of the values near the top of the stack, display the current state of TVM and so on.A.12.1. Debug primitives as multibyte NOPs
- — , for , is a two-byte NOP.
- — , for , is an -byte NOP, with the -byte “contents string” skipped as well.
A.12.2. Debug primitives as operations without side-effect
Next we describe the debug primitives that might (and actually are) implemented in a version of TVM. Notice that another TVM implementation is free to use these codes for other debug purposes, or treat them as multibyte NOPs. Whenever these primitives need some arguments from the stack, they inspect these arguments, but leave them intact in the stack. If there are insufficient values in the stack, or they have incorrect types, debug primitives may output error messages into the debug log, or behave as NOPs, but they cannot throw exceptions.- — , dumps the stack (at most the top 255 values) and shows the total stack depth.
- — , , dumps the top values from the stack, starting from the deepest of them. If there are values available, dumps only values.
- — , dumps in hexadecimal form, be it a Slice or an Integer.
- — , similar to , except the hexadecimal representation of is not immediately output, but rather concatenated to an output text buffer.
- — , dumps in binary form, similarly to .
- — , outputs the binary representation of to a text buffer.
- — , dumps the Slice at as an UTF-8 string.
- — , similar to , but outputs the string into a text buffer (without carriage return).
- — , disables all debug output until it is re-enabled by a . More precisely, this primitive increases an internal counter, which disables all debug operations (except and ) when strictly positive.
- — , enables debug output (in a debug version of TVM).
- — , , dumps .
- — , , concatenates the text representation of (without any leading or trailing spaces or carriage returns) to a text buffer which will be output before the output of any other debug operation.
- - — Use these opcodes for custom/experimental debug operations.
- — , dumps formatted according to the -byte string . This string might contain (a prefix of) the name of a TL-B type supported by the debugger. If the string begins with a zero byte, simply outputs it (without the first byte) into the debug log. If the string begins with a byte equal to one, concatenates it to a buffer, which will be output before the output of any other debug operation (effectively outputs a string without a carriage return).
- — , string is bytes long.
- — , flushes all pending debug output from the buffer into the debug log.
- — , string is bytes long.
A.13. Codepage primitives
The following primitives, which begin with byte , typically are used at the very beginning of a smart contract’s code or a library subroutine to select another TVM codepage. Notice that we expect all codepages to contain these primitives with the same codes, otherwise switching back to another codepage might be impossible (cf. 5.1.8).- — , selects TVM codepage . If the codepage is not supported, throws an invalid opcode exception.
- — , selects TVM (test) codepage zero as described in this document.
- — , selects TVM codepage for . Negative codepages are reserved for restricted versions of TVM needed to validate runs of TVM in other codepages as explained in B.2.6. Negative codepage is reserved for experimental codepages, not necessarily compatible between different TVM implementations, and should be disabled in the production versions of TVM.
- — ( - ), selects codepage with passed in the top of the stack.
B Formal properties and specifications of TVM
This appendix discusses certain formal properties of TVM that are necessary for executing smart contracts in the TON Blockchain and validating such executions afterwards.B.1 Serialization of the TVM state
Recall that a virtual machine used for executing smart contracts in a blockchain must be deterministic, otherwise the validation of each execution would require the inclusion of all intermediate steps of the execution into a block, or at least of the choices made when indeterministic operations have been performed. Furthermore, the state of such a virtual machine must be (uniquely) serializable, so that even if the state itself is not usually included in a block, its hash is still well-defined and can be included into a block for verification purposes.B.1.1. TVM stack values
TVM stack values can be serialized as follows:B.1.2. TVM stack
The TVM stack can be serialized as follows:B.1.3. TVM control registers
Control registers in TVM can be serialized as follows:B.1.4. TVM gas limits
Gas limits in TVM can be serialized as follows:B.1.5. TVM library environment
The TVM library environment can be serialized as follows:B.1.6. TVM continuations
Continuations in TVM can be serialized as follows:B.1.7. TVM state
The total state of TVM can be serialized as follows:B.2 Step function of TVM
A formal specification of TVM would be completed by the definition of a step function . This function deterministically transforms a valid VM state into a valid subsequent VM state, and is allowed to throw exceptions or return an invalid subsequent state if the original state was invalid.B.2.1. A high-level definition of the step function
We might present a very long formal definition of the TVM step function in a high-level functional programming language. Such a specification, however, would mostly be useful as a reference for the (human) developers. We have chosen another approach, better adapted to automated formal verification by computers.B.2.2. An operational definition of the step function
Notice that the step function is a well-defined computable function from trees of cells into trees of cells. As such, it can be computed by a universal Turing machine. Then a program computing on such a machine would provide a machine-checkable specification of the step function . This program effectively is an emulator of TVM on this Turing machine.B.2.3. A reference implementation of the TVM emulator inside TVM
We see that the step function of TVM may be defined by a reference implementation of a TVM emulator on another machine. An obvious idea is to use TVM itself, since it is well-adapted to working with trees of cells. However, an emulator of TVM inside itself is not very useful if we have doubts about a particular implementation of TVM and want to check it. For instance, if such an emulator interpreted a instruction simply by invoking this instruction itself, then a bug in the underlying implementation of TVM would remain unnoticed.B.2.4. Reference implementation inside a minimal version of TVM
We see that using TVM itself as a host machine for a reference implementation of TVM emulator would yield little insight. A better idea is to define a stripped-down version of TVM, which supports only the bare minimum of primitives and 64-bit integer arithmetic, and provide a reference implementation of the TVM step function for this stripped-down version of TVM. In that case, one must carefully implement and check only a handful of primitives to obtain a stripped-down version of TVM, and compare the reference implementation running on this stripped-down version to the full custom TVM implementation being verified. In particular, if there are any doubts about the validity of a specific run of a custom TVM implementation, they can now be easily resolved with the aid of the reference implementation.B.2.5. Relevance for the TON Blockchain
The TON Blockchain adopts this approach to validate the runs of TVM (e.g., those used for processing inbound messages by smart contracts) when the validators’ results do not match one another. In this case, a reference implementation of TVM, stored inside the masterchain as a configurable parameter (thus defining the current revision of TVM), is used to obtain the correct result.B.2.6. Codepage
Codepage of TVM is reserved for the stripped-down version of TVM. Its main purpose is to execute the reference implementation of the step function of the full TVM. This codepage contains only special versions of arithmetic primitives working with “tiny integers” (64-bit signed integers); therefore, TVM’s 257-bit Integer arithmetic must be defined in terms of 64-bit arithmetic. Elliptic curve cryptography primitives are also implemented directly in codepage , without using any third-party libraries. Finally, a reference implementation of the hash function is also provided in codepage .B.2.7. Codepage
This bootstrapping process could be iterated even further, by providing an emulator of the stripped-down version of TVM written for an even simpler version of TVM that supports only boolean values (or integers 0 and 1)—a “codepage ”. All 64-bit arithmetic used in codepage would then need to be defined by means of boolean operations, thus providing a reference implementation for the stripped-down version of TVM used in codepage . In this way, if some of the TON Blockchain validators did not agree on the results of their 64-bit arithmetic, they could regress to this reference implementation to find the correct answer.30C Code density of stack and register machines
This appendix extends the general consideration of stack manipulation primitives provided in 2.2, explaining the choice of such primitives for TVM, with a comparison of stack machines and register machines in terms of the quantity of primitives used and the code density. We do this by comparing the machine code that might be generated by an optimizing compiler for the same source files, for different (abstract) stack and register machines. It turns out that the stack machines (at least those equipped with the basic stack manipulation primitives described in 2.2.1) have far superior code density. Furthermore, the stack machines have excellent extendability with respect to additional arithmetic and arbitrary data processing operations, especially if one considers machine code automatically generated by optimizing compilers.C.1 Sample leaf function
We start with a comparison of machine code generated by an (imaginary) optimizing compiler for several abstract register and stack machines, corresponding to the same high-level language source code that contains the definition of a leaf function (i.e., a function that does not call any other functions). For both the register machines and stack machines, we observe the notation and conventions introduced in 2.1.C.1.1. Sample source file for a leaf function
The source file we consider contains one function that takes six integer arguments , , , , , and and returns two integer values, and , which are the solutions of the system of two linear equations: The source code of the function, in a programming language similar to C, might look as follows:C.1.2. Three-address register machine
The machine code (or rather the corresponding assembly code) for a three-address register machine (cf. 2.1.7) might look as follows:C.1.3. Two-address register machine
The machine code for a two-address register machine might look as follows:C.1.4. One-address register machine
The machine code for a one-address register machine might look as follows:C.1.5. Stack machine with basic stack primitives
The machine code for a stack machine equipped with basic stack manipulation primitives described in 2.2.1 might look as follows:C.1.6. Stack machine with compound stack primitives
A stack machine with compound stack primitives (cf. 2.2.3) would not significantly improve code density of the code presented above, at least in terms of bytes used. The only difference is that, if we were not allowed to use commutativity of multiplication, the extra inserted before the third might be combined with two previous operations , into one compound operation ; we provide the resulting code below. To make this less redundant, we show a version of the code that computes subexpression before as specified in the original source file. We see that this replaces six operations (starting from line 15) with five other operations, and disables the optimization:C.1.7. Stack machine with compound stack primitives and manually optimized code
The previous version of code for a stack machine with compound stack primitives can be manually optimized as follows. By interchanging operations with preceding , , and arithmetic operations whenever possible, we obtain code fragment ; ; , which can then be replaced by compound operation . This compound operation would admit a two-byte encoding, thus leading to 27-byte code using only 21 operations:C.2 Comparison of machine code for sample leaf function
Table 1 summarizes the properties of machine code corresponding to the same source file described in C.1.1, generated for a hypothetical three-address register machine (cf. C.1.2), with both “optimistic” and “realistic” instruction encodings; a two-address machine (cf. C.1.3); a one-address machine (cf. C.1.4); and a stack machine, similar to TVM, using either only the basic stack manipulation primitives (cf. C.1.5) or both the basic and the composite stack primitives (cf. C.1.7). The meaning of the columns in Table 1 is as follows:- “Operations” — The quantity of instructions used, split into “data” (i.e., register move and exchange instructions for register machines, and stack manipulation instructions for stack machines) and “arithmetic” (instructions for adding, subtracting, multiplying and dividing integer numbers). The “total” is one more than the sum of these two, because there is also a one-byte instruction at the end of machine code.
- “Code bytes” — The total amount of code bytes used.
- “Opcode space” — The portion of “opcode space” (i.e., of possible choices for the first byte of the encoding of an instruction) used by data and arithmetic instructions in the assumed instruction encoding. For example, the “optimistic” encoding for the three-address machine assumes two-byte encodings for all arithmetic instructions op . Each arithmetic instruction would then consume portion of the opcode space. Notice that for the stack machine we have assumed one-byte encodings for , and in all cases, augmented by for the basic stack instructions case only. As for the compound stack operations, we have assumed two-byte encodings for , , , , , , but not for .
| Machine | Operations | Code bytes | Opcode space | ||||||
|---|---|---|---|---|---|---|---|---|---|
| data | arith | total | data | arith | total | data | arith | total | |
| 3-addr. (opt.) | 0 | 11 | 12 | 0 | 22 | 23 | 0/256 | 64/256 | 65/256 |
| 3-addr. (real.) | 0 | 11 | 12 | 0 | 30 | 31 | 0/256 | 34/256 | 35/256 |
| 2-addr. | 4 | 11 | 16 | 8 | 22 | 31 | 1/256 | 4/256 | 6/256 |
| 1-addr. | 11 | 11 | 23 | 17 | 11 | 29 | 17/256 | 64/256 | 82/256 |
| stack (basic) | 16 | 11 | 28 | 16 | 11 | 28 | 64/256 | 4/256 | 69/256 |
| stack (comp.) | 9 | 11 | 21 | 15 | 11 | 27 | 84/256 | 4/256 | 89/256 |
C.2.1. Register calling conventions: some registers must be preserved by functions
Up to this point, we have considered the machine code of only one function, without taking into account the interplay between this function and other functions in the same program. Usually a program consists of more than one function, and when a function is not a “simple” or “leaf” function, it must call other functions. Therefore, it becomes important whether a called function preserves all or at least some registers. If it preserves all registers except those used to return results, the caller can safely keep its local and temporary variables in certain registers; however, the callee needs to save all the registers it will use for its temporary values somewhere (usually into the stack, which also exists on register machines), and then restore the original values. On the other hand, if the called function is allowed to destroy all registers, it can be written in the manner described in C.1.2, C.1.3, and C.1.4, but the caller will now be responsible for saving all its temporary values into the stack before the call, and restoring these values afterwards. In most cases, calling conventions for register machines require preservation of some but not all registers. We will assume that registers will be preserved by functions (unless they are used for return values), and that these registers are . Case corresponds to the case “the callee is free to destroy all registers” considered so far; it is quite painful for the caller. Case corresponds to the case “the callee must preserve all registers”; it is quite painful for the callee, as we will see in a moment. Usually a value of around is used in practice. The following sections consider cases , , and for our register machines with registers.C.2.2. Case : no registers to preserve
This case has been considered and summarized in C.2 and Table 1 above.C.2.3. Case : all registers must be preserved
This case is the most painful one for the called function. It is especially difficult for leaf functions like the one we have been considering, which do not benefit at all from the fact that other functions preserve some registers when called—they do not call any functions, but instead must preserve all registers themselves. In order to estimate the consequences of assuming , we will assume that all our register machines are equipped with a stack, and with one-byte instructions and , which push or pop a register into/from the stack. For example, the three-address machine code provided in C.1.2 destroys the values in registers , , , and ; this means that the code of this function must be augmented by four instructions ; ; ; at the beginning, and by four instructions ; ; ; right before the instruction, in order to restore the original values of these registers from the stack. These four additional / pairs would increase the operation count and code size in bytes by . A similar analysis can be done for other register machines as well, leading to Table 2.| Machine | Operations | Code bytes | Opcode space | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| data | arith | total | data | arith | total | data | arith | total | ||
| 3-addr. (opt.) | 4 | 8 | 11 | 20 | 8 | 22 | 31 | 32/256 | 64/256 | 97/256 |
| 3-addr. (real.) | 4 | 8 | 11 | 20 | 8 | 30 | 39 | 32/256 | 34/256 | 67/256 |
| 2-addr. | 5 | 14 | 11 | 26 | 18 | 22 | 41 | 33/256 | 4/256 | 38/256 |
| 1-addr. | 6 | 23 | 11 | 35 | 29 | 11 | 41 | 49/256 | 64/256 | 114/256 |
| stack (basic) | 0 | 16 | 11 | 28 | 16 | 11 | 28 | 64/256 | 4/256 | 69/256 |
| stack (comp.) | 0 | 9 | 11 | 21 | 15 | 11 | 27 | 84/256 | 4/256 | 89/256 |
C.2.4. Case , : registers must be preserved
The analysis of this case is similar to the previous one. The results are summarized in Table 3.| Machine | Operations | Code bytes | Opcode space | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| data | arith | total | data | arith | total | data | arith | total | ||
| 3-addr. (opt.) | 0 | 0 | 11 | 12 | 0 | 22 | 23 | 32/256 | 64/256 | 97/256 |
| 3-addr. (real.) | 0 | 0 | 11 | 12 | 0 | 30 | 31 | 32/256 | 34/256 | 67/256 |
| 2-addr. | 0 | 4 | 11 | 16 | 8 | 22 | 31 | 33/256 | 4/256 | 38/256 |
| 1-addr. | 1 | 13 | 11 | 25 | 19 | 11 | 31 | 49/256 | 64/256 | 114/256 |
| stack (basic) | 0 | 16 | 11 | 28 | 16 | 11 | 28 | 64/256 | 4/256 | 69/256 |
| stack (comp.) | 0 | 9 | 11 | 21 | 15 | 11 | 27 | 84/256 | 4/256 | 89/256 |
C.2.5. A fairer comparison using a binary code instead of a byte code
The reader may have noticed that our preceding discussion of -address register machines and stack machines depended very much on our insistence that complete instructions be encoded by an integer number of bytes. If we had been allowed to use a “bit” or “binary code” instead of a byte code for encoding instructions, we could more evenly balance the opcode space used by different machines. For instance, the opcode ofSUB for a three-address machine had to be either 4-bit (good for code density, bad for opcode space) or 12-bit (very bad for code density), because the complete instruction has to occupy a multiple of eight bits (e.g., 16 or 24 bits), and of those bits have to be used for the three register names.
Therefore, let us get rid of this restriction.
Now that we can use any number of bits to encode an instruction, we can choose all opcodes of the same length for all the machines considered. For instance, all arithmetic instructions can have 8-bit opcodes, as the stack machine does, using of the opcode space each; then the three-address register machine will use 20 bits to encode each complete arithmetic instruction. All s, s, es, and s on register machines can be assumed to have 4-bit opcodes, because this is what we do for the most common stack manipulation primitives on a stack machine. The results of these changes are shown in Table 4.
We can see that the performance of the various machines is much more balanced, with the stack machine still the winner in terms of the code density, but with the three-address machine enjoying the second place it really merits. If we were to consider the decoding speed and the possibility of parallel execution of instructions, we would have to choose the three-address machine, because it uses only 12 instructions instead of 21.
| Machine | Operations | Code bytes | Opcode space | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| data | arith | total | data | arith | total | data | arith | total | ||
| 3-addr. | 0 | 0 | 11 | 12 | 0 | 27.5 | 28.5 | 64/256 | 4/256 | 69/256 |
| 2-addr. | 0 | 4 | 11 | 16 | 6 | 22 | 29 | 64/256 | 4/256 | 69/256 |
| 1-addr. | 1 | 13 | 11 | 25 | 16 | 16.5 | 32.5 | 64/256 | 4/256 | 69/256 |
| stack (basic) | 0 | 16 | 11 | 28 | 16 | 11 | 28 | 64/256 | 4/256 | 69/256 |
| stack (comp.) | 0 | 9 | 11 | 21 | 15 | 11 | 27 | 84/256 | 4/256 | 89/256 |
C.3 Sample non-leaf function
This section compares the machine code for different register machines for a sample non-leaf function. Again, we assume that either , , or registers are preserved by called functions, with representing the compromise made by most modern compilers and operating systems.C.3.1. Sample source code for a non-leaf function
A sample source file may be obtained by replacing the built-in integer type with a custom Rational type, represented by a pointer to an object in memory, in our function for solving systems of two linear equations (cf. C.1.1):C.3.2. Three-address and two-address register machines, preserved registers
Because our sample function does not use built-in arithmetic instructions at all, compilers for our hypothetical three-address and two-address register machines will produce the same machine code. Apart from the previously introduced and one-byte instructions, we assume that our two- and three-address machines support the following two-byte instructions: , , and , for . Such instructions occupy only 3/256 of the opcode space, so their addition seems quite natural. We first assume that (i.e., that all subroutines are free to destroy the values of all registers). In this case, our machine code for does not have to preserve any registers, but has to save all registers containing useful values into the stack before calling any subroutines. A size-optimizing compiler might produce the following code:C.3.3. Three-address and two-address register machines, preserved registers
Now we have eight registers, to , that are preserved by subroutine calls. We might keep some intermediate values there instead of pushing them into the stack. However, the penalty for doing so consists in a / pair for every such register that we choose to use, because our function is also required to preserve its original value. It seems that using these registers under such a penalty does not improve the density of the code, so the optimal code for three- and two-address machines for preserved registers is the same as that provided in C.3.2, with a total of 42 instructions and 74 code bytes.C.3.4. Three-address and two-address register machines, preserved registers
This time all registers must be preserved by the subroutines, excluding those used for returning the results. This means that our code must preserve the original values of to , as well as any other registers it uses for temporary values. A straightforward way of writing the code of our subroutine would be to push registers up to, say, into the stack, then perform all the operations required, using — for intermediate values, and finally restore registers from the stack. However, this would not optimize code size. We choose another approach:C.3.5. One-address register machine, preserved registers
For our one-address register machine, we assume that new register-stack instructions work through the accumulator only. Therefore, we have three new instructions, (equivalent to of two-address machines), (equivalent to ), and (equivalent to ). To make the comparison with two-address machines more interesting, we assume one-byte encodings for these new instructions, even though this would consume of the opcode space. By adapting the code provided in C.3.2 to the one-address machine, we obtain the following:C.3.6. One-address register machine, preserved registers
As explained in C.3.3, the preservation of — between subroutine calls does not improve the size of our previously written code, so the one-address machine will use for the same code provided in C.3.5.C.3.7. One-address register machine, preserved registers
We simply adapt the code provided in C.3.4 to the one-address register machine:C.3.8. Stack machine with basic stack primitives
We reuse the code provided in C.1.5, simply replacing arithmetic primitives (VM instructions) with subroutine calls. The only substantive modification is the insertion of the previously optional before the third multiplication, because even an optimizing compiler cannot now know whether is a commutative operation. We have also used the “tail recursion optimization” by replacing the final followed by with .C.3.9. Stack machine with compound stack primitives
We again reuse the code provided in C.1.7, replacing arithmetic primitives with subroutine calls and making the tail recursion optimization:C.4 Comparison of machine code for sample non-leaf function
Table 5 summarizes the properties of machine code corresponding to the same source file provided in C.3.1. We consider only the “realistically” encoded three-address machines. Three-address and two-address machines have the same code density properties, but differ in the utilization of opcode space. The one-address machine, somewhat surprisingly, managed to produced shorter code than the two-address and three-address machines, at the expense of using up more than half of all opcode space. The stack machine is the obvious winner in this code density contest, without compromizing its excellent extendability (measured in opcode space used for arithmetic and other data transformation instructions).| Machine | Operations | Code bytes | Opcode space | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| data | cont. | total | data | cont. | total | data | arith | total | ||
| 3-addr. | 0,8 | 29 | 12 | 41 | 42 | 34 | 76 | 35/256 | 34/256 | 72/256 |
| 16 | 27 | 12 | 39 | 44 | 34 | 78 | ||||
| 2-addr. | 0,8 | 29 | 12 | 41 | 42 | 34 | 76 | 37/256 | 4/256 | 44/256 |
| 16 | 27 | 12 | 39 | 44 | 34 | 78 | ||||
| 1-addr. | 0,8 | 33 | 12 | 45 | 33 | 34 | 67 | 97/256 | 64/256 | 164/256 |
| 16 | 28 | 12 | 40 | 39 | 34 | 73 | ||||
| stack (basic) | 18 | 11 | 29 | 18 | 33 | 51 | 64/256 | 4/256 | 71/256 | |
| stack (comp.) | 9 | 11 | 20 | 15 | 33 | 48 | 84/256 | 4/256 | 91/256 |
C.4.1. Combining with results for leaf functions
It is instructive to compare this table with the results in C.2 for a sample leaf function, summarized in Table 1 (for preserved registers) and the very similar Table 3 (for preserved registers), and, if one is still interested in case (which turned out to be worse than in almost all situations), also to Table 2. We see that the stack machine beats all register machines on non-leaf functions. As for the leaf functions, only the three-address machine with the “optimistic” encoding of arithmetic instructions was able to beat the stack machine, winning by 15%, by compromising its extendability. However, the same three-address machine produces 25% longer code for non-leaf functions. If a typical program consists of a mixture of leaf and non-leaf functions in approximately equal proportion, then the stack machine will still win.C.4.2. A fairer comparison using a binary code instead of a byte code
Similarly to C.2.5, we may offer a fairer comparison of different register machines and the stack machine by using arbitrary binary codes instead of byte codes to encode instructions, and matching the opcode space used for data manipulation and arithmetic instructions by different machines. The results of this modified comparison are summarized in Table 6. We see that the stack machines still win by a large margin, while using less opcode space for stack/data manipulation.| Machine | Operations | Code bytes | Opcode space | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| data | cont. | total | data | cont. | total | data | arith | total | ||
| 3-addr. | 0,8 | 29 | 12 | 41 | 35.5 | 34 | 69.5 | 110/256 | 4/256 | 117/256 |
| 16 | 27 | 12 | 39 | 35.5 | 34 | 69.5 | ||||
| 2-addr. | 0,8 | 29 | 12 | 41 | 35.5 | 34 | 69.5 | 110/256 | 4/256 | 117/256 |
| 16 | 27 | 12 | 39 | 35.5 | 34 | 69.5 | ||||
| 1-addr. | 0,8 | 33 | 12 | 45 | 33 | 34 | 67 | 112/256 | 4/256 | 119/256 |
| 16 | 28 | 12 | 40 | 33.5 | 34 | 67.5 | ||||
| stack (basic) | 18 | 11 | 29 | 18 | 33 | 51 | 64/256 | 4/256 | 71/256 | |
| stack (comp.) | 9 | 11 | 20 | 15 | 33 | 48 | 84/256 | 4/256 | 91/256 |
C.4.3. Comparison with real machines
Note that our hypothetical register machines have been considerably optimized to produce shorter code than actually existing register machines; the latter are subject to other design considerations apart from code density and extendability, such as backward compatibility, faster instruction decoding, parallel execution of neighboring instructions, ease of automatically producing optimized code by compilers, and so on. For example, the very popular two-address register architecture x86-64 produces code that is approximately twice as long as our “ideal” results for the two-address machines. On the other hand, our results for the stack machines are directly applicable to TVM, which has been explicitly designed with the considerations presented in this appendix in mind. Furthermore, the actual TVM code is even shorter (in bytes) than shown in Table 5 because of the presence of the two-byte instruction, allowing TVM to call up to 256 user-defined functions from the dictionary at . This means that one should subtract 10 bytes from the results for stack machines in Table 5 if one wants to specifically consider TVM, rather than an abstract stack machine; this produces a code size of approximately 40 bytes (or shorter), almost half that of an abstract two-address or three-address machine.C.4.4. Automatic generation of optimized code
An interesting point is that the stack machine code in our samples might have been generated automatically by a very simple optimizing compiler, which rearranges values near the top of the stack appropriately before invoking each primitive or calling a function as explained in 2.2.2 and 2.2.5. The only exception is the unimportant “manual” optimization described in C.1.7, which enabled us to shorten the code by one more byte. By contrast, the heavily optimized (with respect to size) code for register machines shown in C.3.2 and C.3.3 is unlikely to be produced automatically by an optimizing compiler. Therefore, if we had compared compiler-generated code instead of manually-generated code, the advantages of stack machines with respect to code density would have been even more striking.References
1 N. Durov, Telegram Open Network, 2017.Footnotes
[1] For example, there are no floating-point arithmetic operations (which could be efficiently implemented using hardware-supported double type on most modern CPUs) present in TVM, because the result of performing such operations is dependent on the specific underlying hardware implementation and rounding mode settings. Instead, TVM supports special integer arithmetic operations, which can be used to simulate fixed-point arithmetic if needed. Back ↑ 2 The production version will likely require some tweaks and modifications prior to launch, which will become apparent only after using the experimental version in the test environment for some time. Back ↑ 3 A high-level smart-contract language might create a visibility of variables for the ease of programming; however, the high-level source code working with variables will be translated into TVM machine code keeping all the values of these variables in the TVM stack. Back ↑ 4 In the TON Blockchain context,c7 is initialized with a singleton Tuple, the only component of which is a Tuple containing blockchain-specific data. The smart contract is free to modify c7 to store its temporary data provided the first component of this Tuple remains intact. Back ↑
5 Strictly speaking, there is also the current library context, which consists of a dictionary with 256-bit keys and cell values, used to load library reference cells of 3.1.7. Back ↑
6 Our inclusion of r0 here creates a minor conflict with our assumption that the accumulator register, if present, is also r0; for simplicity, we will resolve this problem by assuming that the first argument to a function is passed in the accumulator. Back ↑
7 For instance, if one writes a function for extracting square roots, this function will always accept its argument and return its result in the same registers, in contrast with a hypothetical built-in square root instruction, which could allow the programmer to arbitrarily choose the source and destination registers. Therefore, a user-defined function is tremendously less flexible than a built-in instruction on a register machine. Back ↑
8 Of course, if the second option is used, this will destroy the original arrangement of in the top of the stack. In this case, one should either issue a SWAP before XCHG s(j'), or replace the previous operation XCHG s(i) with XCHG s1, s(i), so that is exchanged with s1 from the beginning. Back ↑
9 Notice that the most common XCHG s(i) operation is not really required here if we do not insist on keeping the same temporary value or variable always in the same stack location, but rather keep track of its subsequent locations. We will move it to some other location while preparing the arguments to the next primitive or function call. Back ↑
10 An alternative, arguably better, translation of PU s(i_1),…,s(i_γ) consists of the translation of s(i_2),…,s(i_γ), followed by PUSH s(i_1+α-1); XCHG s(γ-1). Back ↑
11 From the perspective of low-level cell operations, these data bits and cell references are not intermixed. In other words, an (ordinary) cell essentially is a couple consisting of a list of up to 1023 bits and of a list of up to four cell references, without prescribing an order in which the references and the data bits should be deserialized, even though TL-B schemes appear to suggest such an order. Back ↑
12 From a theoretical perspective, we might say that a cell has an infinite sequence of hashes , which eventually stabilizes: . Then the level is simply the largest index , such that . Back ↑
13 A pruned branch cell of level is bound by a Merkle (proof or update) cell if there are exactly Merkle cells on the path from to its descendant , including . Back ↑
14 Negative numbers are represented using two’s complement. For instance, integer is serialized by instruction STI 8 into bitstring xEF. Back ↑
15 A description of an older version of TL may be found at https://core.telegram.org/mtproto/TL. Back ↑
16 The field’s name is useful for representing values of the type being defined in human-readable form, but it does not affect the binary serialization. Back ↑
17 This is the “linear negation” operation of linear logic, hence our notation ~. Back ↑
18 In fact, may receive extra arguments and return modified values, which are passed to the next invocation of . This may be used to implement “map” and “reduce” operations with dictionaries. Back ↑
19 Versions of this operation may be introduced where and receive an additional bitstring argument, equal to the key (for leaves) or to the common prefix of all keys (for forks) in the corresponding subtree. Back ↑
20 If there are no bits of data left in code, but there is still exactly one reference, an implicit JMP to the cell at that reference is performed instead of an implicit RET. Back ↑
21 Technically, TVM may simply invoke a virtual method run() of the continuation currently in cc. Back ↑
22 The already used savelist cc.save of the new cc is emptied before the execution starts. Back ↑
23 The implementation of REPEAT involves an extraordinary continuation that remembers the remaining number of iterations, the body of the loop , and the return continuation . (The latter term represents the remainder of the body of the function that invoked REPEAT, which would be normally stored in c0 of the new cc.) Back ↑
24 An important point here is that the tree of cells representing a TVM program cannot have cyclic references, so using CALLREF along with a reference to a cell higher up the tree would not work. Back ↑
25 This is not exactly true. A more precise statement is that usually the codepage of the newly-created continuation is a known function of the current codepage. Back ↑
26 This is another important mechanism of backward compatibility. All values of newly-added types, as well as values belonging to extended original types that do not belong to the original types (e.g., 513-bit integers that do not fit into 257 bits in the example above), are treated by all instructions (except stack manipulation instructions, which are naturally polymorphic, cf. 2.3.3) in the old codepages as “values of incorrect type”, and generate type-checking exceptions accordingly. Back ↑
27 If the cell dumps are hexadecimal, encodings consisting of an integral number of hexadecimal digits (i.e., having length divisible by four bits) might be equally convenient. Back ↑
28 Notice that it is the probability of occurrence in the code that counts, not the probability of being executed. An instruction occurring in the body of a loop executed a million times is still counted only once. Back ↑
29 Notice that any modifications after launch cannot be done unilaterally; rather they would require the support of at least two-thirds of validators. Back ↑
30 The preliminary version of TVM does not use codepage -2 for this purpose. This may change in the future. Back ↑
31 It is interesting to compare this code with that generated by optimizing C compilers for the x86-64 architecture. First of all, the integer division operation for x86-64 uses the one-address form, with the (double-length) dividend to be supplied in accumulator pair r2:r0. The quotient is also returned in r0. As a consequence, two single-to-double extension operations (CDQ or CQO) and at least one move operation need to be added. Secondly, the encoding used for arithmetic and move operations is less optimistic than in our example above, requiring about three bytes per operation on average. As a result, we obtain a total of 43 bytes for 32-bit integers, and 68 bytes for 64-bit integers. Back ↑
32 Code produced for this function by an optimizing compiler for x86-64 architecture with size-optimization enabled actually occupied 150 bytes, due mostly to the fact that actual instruction encodings are about twice as long as we had optimistically assumed. Back ↑