Blog › ICP guides

Terra developer on retainer: Terra-Lua metaprogramming, compile-time Lua escape, low-level struct programming, and terralib on monthly retainer

September 26, 2026 · ~15 min read

A Terra computational kernel library was being parameterized using Lua metaprogramming. The developer defined a Terra struct to hold kernel configuration and a terra function named compute_kernel that operated on it. To make the kernel’s block size configurable, the developer stored the block size in a Lua table: config = { block_size = 256, stride = 32 }. Inside the terra compute_kernel function body, the developer wrote var i = config.block_size, expecting config.block_size to be available as a value inside the Terra function. In Terra, the compilation model separates Lua runtime from Terra compile time: a terra function is compiled Just-In-Time via LLVM at the point where the Lua program first encounters the terra block; after compilation, the Terra function is a static C-like function pointer with no connection to Lua state; Lua tables, Lua variables, and Lua runtime values are not accessible inside a Terra function body because at the time the Terra function executes, there is no Lua state to query. The Terra compiler, encountering config.block_size inside the Terra function body, rejected it: config is a Lua table and not a Terra value. Compilation errors: 2 functions. The Terra developer on retainer diagnosed the Terra-Lua boundary constraint and restructured the code: before the terra compute_kernel block, the developer added local bs = config.block_size — a Lua-side assignment that captures the value at Lua runtime, which is also Terra compile time; inside the Terra function body, the developer replaced config.block_size with the escape expression [bs], which is the Terra escape syntax for evaluating a Lua expression at compile time and embedding the result as a Terra literal. Compilation errors: 2 → 0.

The work log entry read “fixed kernel configuration boundary errors, 8h.” It names the result and duration. It cannot explain the fundamental Terra execution model — Terra is not a language where Lua and Terra code run interleaved at a single runtime; it is a two-phase model where the Lua program runs first and orchestrates the compilation of Terra functions; when the Lua interpreter encounters a terra function block, it compiles the block to native machine code via LLVM; the compilation happens at Lua runtime (so Lua values that exist at that point can be captured), but the execution of the resulting native function happens independently of the Lua interpreter; the compiled function is a machine code address, callable from Lua as a regular value, but not connected to any Lua state; there is no “calling back into Lua” from inside a running Terra function except through explicitly bound callbacks. It cannot explain the [luaexpr] escape mechanism — inside a terra block, square brackets introduce an escape: [expr] evaluates expr as a Lua expression at Terra compile time and splices the result into the Terra code as a literal; [bs] where bs is a Lua number variable splices the number’s value as an integer literal in the Terra code; [luafunc] where luafunc is a Lua function that returns a Terra value splices whatever that function returns; the escape mechanism is what makes Terra metaprogramming possible: Lua generates Terra code, and the escape syntax is the bridge from Lua values to Terra literals at compile time. It cannot explain the quote ... in ... end and emit primitives — quote stmt1; stmt2; end produces a Lua value representing a Terra code fragment (a “quote”); emit [quote_value] inside a Terra function inserts the quoted statements at that position; together, they enable generating Terra functions programmatically from Lua loops: a Lua loop that iterates over a list of configurations can emit a specialized Terra function body for each configuration, producing multiple instances of the same kernel specialized for different parameters. The 8 hours of Terra-Lua boundary analysis, compile-time escape restructuring, and metaprogramming design are invisible in the diff.

Terra-Lua metaprogramming: [luaexpr] escape, compile-time value capture, quote/emit code generation, and symbol() naming

The Terra-Lua execution model has two phases. In the Lua phase, the Lua interpreter runs, encounters terra function blocks, and compiles them to native code via LLVM at the current Lua runtime point. In the Terra phase, compiled Terra functions execute as machine code without Lua involvement. The [luaexpr] escape syntax bridges these phases: inside a terra block, [expr] is evaluated by the Lua interpreter at Terra compile time and the resulting Lua value is embedded in the Terra code. If expr evaluates to a number, it becomes a Terra numeric literal; if it evaluates to a Terra type (such as int or double), it can be used as a type annotation; if it evaluates to a Lua function that returns a Terra function, it can be called via escape at a Terra call site. The key constraint is that the Lua expression must be evaluable at Terra compile time (when the Lua interpreter processes the terra block) and must produce a value that has a Terra equivalent. A Lua table is not a Terra value; a Lua number is; a Lua string is not; a Lua boolean mapped to a Terra bool is.

The quote and emit primitives enable programmatic Terra code generation. local q = quote var x = [val]; x = x + 1 in x end produces a Lua object representing the quoted Terra statements; emit [q] inside a Terra function inserts those statements at the emit point. This is the Terra mechanism for generating specialized kernels: a Lua loop iterates over a list of configurations, and for each configuration, emits Terra code that is specialized for that configuration’s parameters (block size, data type, loop unroll factor). The symbol(type) function produces a fresh Terra variable symbol for use in generated code, avoiding name collisions across multiple emitted code fragments. Terra was developed by Zachary DeVito and colleagues at Stanford University as a low-level systems language embedded in Lua, designed for high-performance kernel generation and domain-specific language implementation. Its closest retainer neighbors are Lua developer retainers (Terra is embedded in Lua; the Lua runtime orchestrates Terra compilation) and C++ developer retainers (Terra compiles to native machine code via LLVM with C-compatible data layout and ABI), but Terra’s two-phase execution model where Lua controls the compile-time behavior and Terra provides the runtime-performance behavior, the [luaexpr] escape mechanism for compile-time Lua-to-Terra value bridging, and the quote/emit code generation primitives make the retainer work distinct in Lua-Terra boundary diagnosis, compile-time escape design, and programmatic kernel specialization.

Terra structs, pointer operations, terralib.includec C interop, and terralib.saveobj compilation

Terra structs are declared with the struct keyword: struct Point { x: double; y: double; }. Structs have C-compatible memory layout; a Terra struct pointer can be passed to a C function expecting a pointer to the corresponding C struct with no conversion. Methods on structs use the colon syntax: terra Point:distance(other: &Point): double is a method on Point that takes a pointer to another Point; inside the method, self is the implicit receiver. The & operator takes the address of a Terra value, producing a pointer; the @ operator dereferences a pointer, producing the pointed-to value. Pointer arithmetic is C-style: ptr + n advances the pointer by n elements of the pointed-to type. terralib.types provides access to the Terra type system from Lua: terralib.types.pointer(terralib.types.int) constructs the type &int.

terralib.includec("header.h") imports a C header file and makes its declarations (types, function prototypes, struct definitions) available in Lua as Terra types and function declarations. This is the primary mechanism for calling C libraries from Terra: import the header, then call the declared C functions directly in Terra code. The return value from includec is a Lua table whose keys are declaration names; local C = terralib.includec("stdio.h"); terra printval(x: int) C.printf("%d\n", x) end imports printf from the standard library. terralib.saveobj("output.o", {kernel = compute_kernel}) saves the compiled Terra function compute_kernel to an object file that can be linked into a C project; terralib.saveobj("libkernel.so", ...) produces a shared library; terralib.saveobj("kernel", ...) without an extension produces an executable. terralib.optimize() applies LLVM optimization passes to compiled Terra functions; the optimization level is specified as an argument. Cross-compilation uses a LLVM target triplet string to specify the target architecture, OS, and ABI.

How HourTab tracks Terra developer retainer hours

Terra retainer work carries the invisible-hours problem specific to two-phase language systems: the distinction between “Lua runtime” and “Terra compile time” is the same moment from a wall-clock perspective (Terra compilation happens when Lua processes the terra block), but the semantic distinction between what is a Lua value and what is a Terra value at that moment is the source of the most common Terra development errors. The configuration boundary error described above — where config.block_size inside a terra function body produced a compile-time error because config is a Lua table and not a Terra value — is the error developers encounter within their first hours of writing Terra metaprogramming code. The fix is mechanically simple: capture the Lua value in a local Lua variable before the terra block, then use the [variable] escape syntax inside the Terra function. But the understanding required to apply the fix correctly — knowing which Lua values have Terra equivalents, knowing when the capture happens relative to function invocation (at Lua processing time, not at Terra function call time), and knowing that a [luavalue] escape produces a compile-time constant baked into the Terra function rather than a runtime parameter — is a Terra-specific mental model that takes hours to build from first principles. A retainer engagement typically involves a Terra-Lua boundary audit (all uses of Lua values inside Terra function bodies identified; correct vs incorrect captures verified; escape syntax reviewed), a metaprogramming design review (quote/emit patterns for kernel specialization; symbol naming conventions; Lua loop → Terra code generation correctness), and a terralib.includec interop review (header paths, type mappings, function pointer types, and ABI compatibility with the target C library).

HourTab gives Terra developers a public retainer-hours URL they send to clients — typically high-performance computing teams using Terra to generate specialized numerical kernels, compiler researchers implementing DSLs with Terra as the code generation backend, and systems programmers using Terra’s C interop for embedding JIT-compiled kernels in C applications. For Terra retainers, each work log entry should name the mechanism (boundary category: Lua table field accessed inside terra body — compilation error, local Lua variable capture before terra block, [v] escape syntax, compile-time literal vs runtime parameter; metaprogramming: quote fragment creation, emit insertion, symbol() for generated names, Lua loop over configurations to generate specialized functions; C interop: terralib.includec header import, extern function binding, terralib.saveobj output format, optimization pass; specific function name and before/after compilation error count). Terra retainers are often compared to Lua developer retainers for the embedding relationship, but Terra’s two-phase model where the Lua program controls the compile-time behavior of Terra functions (making Lua the metaprogramming language and Terra the performance language), the [luaexpr] escape that splices Lua values into Terra code at compile time, and the LLVM-backed native compilation that produces machine code with C ABI compatibility make the retainer work distinct in Terra-Lua boundary diagnosis, compile-time capture design, and programmatic kernel generation architecture. HourTab’s work log makes the Lua-Terra boundary restructuring, compile-time escape design, and metaprogramming architecture work visible to clients who would otherwise see only the symptom — a compilation error on a line that looks like straightforward data access — and not understand why the fix required knowing that a Terra function body is compiled to static machine code at the moment Lua processes it, that Lua table fields are not Terra values, and that the [luaexpr] escape is not a way to access Lua values at Terra runtime but a way to embed Lua values as literals at Terra compile time.

FAQ: Terra developer retainers