Blog › ICP guides

Futhark developer on retainer: parallel semantics, GPU compilation, reduction associativity, and Futhark data-parallel programming on monthly retainer

September 26, 2026 · ~15 min read

A Futhark numerical pipeline was built around a series of row-normalization stages that processed a two-dimensional floating-point array. Each stage applied a map over the rows, computing a per-row normalization using reduce (+) 0.0 row to compute the row sum. Futhark’s parallel semantics require that the reduction function be associative with respect to the neutral element: the runtime is permitted to reorder additions arbitrarily across the parallel reduction tree. For integer addition this reordering is exact. For f64 floating-point addition it is not: IEEE 754 defines rounding as occurring at each individual operation, so the order in which partial sums are accumulated affects the final result when intermediate values differ significantly in magnitude. The normalization pipeline produced results that matched the sequential reference implementation when run on the CPU backend (futhark c) but diverged by small but statistically significant amounts when run on the GPU backend (futhark opencl). The difference was not a hardware bug; it was a direct consequence of the parallel reduction reordering additions in a different tree structure than the left-to-right sequential fold. Reduction correctness errors: 2 pipeline stages. The Futhark developer on retainer diagnosed the associativity constraint: reduce in Futhark is permitted to reorder the reduction function’s applications; for operations where reordering changes results, reduce_comm signals that the function is also commutative, which narrows the reordering freedom slightly but does not eliminate it; for operations that require strictly sequential prefix accumulation, scan produces a same-length output where each element is the result of the prefix fold up to that index. The developer restructured the two affected stages: row sums using floating-point addition were annotated with reduce_comm (+) 0.0 row to signal commutativity, and one stage that required computing a running minimum that depended on the previous row’s minimum was restructured from reduce to scan. Reduction correctness errors: 2 stages → 0.

The work log entry read “fixed GPU normalization pipeline numerical errors, 8h.” It names the result and duration. It cannot explain why the error appeared only on the GPU backend and not the CPU backend — the futhark c backend generates sequential C code that evaluates reductions left-to-right, matching the reference implementation exactly, while the futhark opencl backend maps reductions onto GPU workgroups that perform tree reductions within a workgroup and then reduce across workgroups, producing a different summation tree with different intermediate rounding; the same source code, semantically correct for exact arithmetic, produces different results under IEEE 754 depending on the backend. It cannot explain how to choose between reduce, reduce_comm, and scan — reduce requires associativity and a neutral element but makes no commutativity promise; reduce_comm additionally requires commutativity, which enables the compiler to use certain SIMD reduction primitives, but still permits arbitrary reordering; scan computes a sequential prefix scan and is the correct choice whenever the computation depends on an ordered accumulation that cannot be split into independent subranges. It cannot explain Futhark’s irregular parallelism handling — filter produces a variable-length output array whose length is not known at compile time; map and reduce require fixed-shape arrays because GPU parallelism maps array indices to GPU threads; working with irregular data requires flatten to convert a nested [[i32]] jagged array to a flat [i32] array plus a per-segment length array, and unflatten to reconstruct the nested shape after processing; segmented scan applies scan independently per segment of the flat representation. The 8 hours of reduction associativity analysis, scan vs reduce selection, and irregular array restructuring are invisible in the diff.

Futhark parallel semantics: reduce vs reduce_comm vs scan, filter, and flatten/unflatten for irregular arrays

Futhark’s parallel primitives each carry different semantic guarantees that map to different GPU execution patterns. map f arr applies f to each element independently and produces a same-shape output; the shape is fixed at compile time for statically shaped arrays and at runtime for dynamically shaped ones; no element depends on another, so the GPU scheduler assigns elements to threads with no synchronization. reduce f ne arr requires that f be associative with neutral element ne: the runtime is permitted to evaluate f in any tree structure across the input, and the neutral element fills in when the reduction tree has an odd number of elements at any level. reduce_comm f ne arr additionally requires that f be commutative, which the Futhark compiler uses to select different GPU reduction algorithms that may be more efficient for specific hardware; the semantic constraint is stricter (commutativity on top of associativity) and the neutral element requirement remains. scan f ne arr computes the prefix scan: element i in the output is f(f(f(ne, arr[0]), arr[1]), ..., arr[i]), preserving strict left-to-right order; scan is parallelized using the parallel prefix scan algorithm (Blelloch scan), which requires associativity and a neutral element but evaluates the function in a specific tree structure that produces results consistent with sequential left-to-right prefix folding. The choice between reduce and scan is the central design decision in Futhark data pipelines: if you need a single aggregate value, use reduce; if you need the intermediate prefix values, use scan; if the operation is not associative, Futhark cannot express it as a parallel reduction at all, and restructuring to use a sequential loop via loop expressions is the fallback.

Futhark’s type system is array-focused: the element types are i8, i16, i32, i64 for signed integers; u8, u16, u32, u64 for unsigned; f16, f32, f64 for IEEE 754 floating-point; and bool. Arrays are denoted [n]i32 for a one-dimensional array of n signed 32-bit integers, and [m][n]f64 for a two-dimensional array. Futhark supports tuples (a, b, c) and records {field: type} as structural types. The parametric module system uses module type to define interface specifications (analogous to ML signatures) and functor application module M = Functor(Param) to instantiate parameterized modules; this system allows writing generic parallel algorithms (for example, a generic segmented scan that is parameterized on the element type and reduction function) and instantiating them at different types without code duplication. Futhark is a functional array language developed at the University of Copenhagen primarily for high-performance GPU computing. Its closest retainer neighbors are Julia developer retainers (both are used for high-performance numerical computing, though Julia is more general-purpose) and Haskell developer retainers (shared functional paradigm and strong type system), but Futhark’s exclusive focus on GPU-parallel data-parallel computation with strict associativity constraints on reductions, irregular parallelism handling via flatten/unflatten, and multi-backend compilation to OpenCL, CUDA, and WebAssembly make the retainer work distinct in parallel correctness analysis, GPU semantics, and backend-specific performance diagnosis.

Futhark GPU compilation: futhark opencl, futhark cuda, and memory transfer optimization

Futhark provides multiple compilation backends that target different execution environments. futhark c generates sequential C code with no GPU dependency; it is the reference implementation for correctness testing because it evaluates all parallel operations sequentially from left to right, matching the associativity order that most developers intuitively expect. futhark opencl generates OpenCL code that runs on any OpenCL-capable GPU, including AMD, NVIDIA, and Intel integrated graphics; the generated code maps parallel operations onto OpenCL workgroups, and the reduction algorithm used within a workgroup differs from the left-to-right sequential order of the C backend. futhark cuda generates CUDA code for NVIDIA GPUs and can use CUDA-specific primitives such as warp shuffle reductions that are not available in OpenCL; NVIDIA-specific hardware features (tensor cores, cooperative groups) are not directly accessible but the underlying memory and thread hierarchy is exploited by the generated code. futhark wasm generates WebAssembly code for in-browser execution, using WebGPU or WASM SIMD instructions depending on the target environment. The --library flag generates a library rather than an executable, exposing the compiled kernels as callable functions from C, Python, or JavaScript host code; this is the integration path for embedding Futhark parallel kernels inside larger applications that perform the non-parallel work in a host language.

GPU memory transfer is the most common performance bottleneck in Futhark programs that are called from a host language. Each call to a Futhark entry point involves transferring input arrays from host memory to GPU memory, executing the GPU kernels, and transferring output arrays back to host memory. Futhark’s generated library code manages this transfer automatically, but it cannot automatically eliminate redundant round-trips for intermediate results. Retainer work involving GPU memory transfer optimization covers: restructuring multi-stage pipelines so that all stages that can run on the GPU are expressed as a single Futhark entry point, eliminating the host round-trip between stages; using Futhark’s opaque type system to pass GPU-resident arrays between multiple entry points without transferring them back to the host; and profiling with the backend’s native tools (OpenCL profiling events, CUDA nvprof/Nsight) to identify which entry points dominate transfer time. The module system design work — defining module types for common parallel kernel interfaces and using functors to instantiate them for different element types — is the structural component of a Futhark retainer that directly parallels interface and type class design work in conventional functional language retainers.

How HourTab tracks Futhark developer retainer hours

Futhark retainer work carries the invisible-hours problem specific to GPU parallel programming: a correctness bug caused by reduction associativity produces results that are numerically close to correct (the error is at the level of floating-point rounding differences, not catastrophically wrong values), which means the bug may go undetected until a regression test compares against a high-precision reference implementation. The normalization pipeline described above — where reduce (+) produced slightly different results on the GPU backend than the CPU backend — is the most common correctness issue in Futhark programs written by developers accustomed to sequential computation: they write a reduction function that is correct for sequential left-to-right evaluation, which is all that the mathematical definition of “sum” requires, and the GPU backend reorders additions in a way that is mathematically valid under exact arithmetic but produces different IEEE 754 results. Diagnosing this requires understanding Futhark’s parallel semantics contract (associativity is required; commutativity is optional with reduce_comm; sequential prefix order requires scan), knowing that the futhark c and futhark opencl backends evaluate the same source code with different reordering strategies, and knowing how to read a numerical discrepancy between backends as a reduction ordering issue rather than a precision or type error. A retainer engagement typically involves associativity audit (every reduce call verified to use an associative function and correct neutral element), reduce vs scan selection audit (every accumulation that requires ordered prefix semantics verified to use scan), and irregular array audit (filter output fed into downstream map/reduce verified to pass through flatten/unflatten correctly).

HourTab gives Futhark developers a public retainer-hours URL they send to clients — typically research groups running numerical simulations on GPU clusters, data-processing teams embedding Futhark kernels in larger Python or C pipelines, and scientific computing organizations that need high-performance parallel array operations without writing raw CUDA or OpenCL. For Futhark retainers, each work log entry should name the mechanism (parallel semantics: reduce vs reduce_comm vs scan selection, associativity constraint verification, neutral element; irregular: filter, flatten, unflatten, segmented scan; compilation: backend selection, --library generation, memory transfer optimization; module: module type interface, functor parameterization), the specific function name, and the before/after error count. Futhark retainers are often compared to Julia developer retainers for the shared high-performance numerical computing context, but Futhark’s exclusive data-parallel model with explicit associativity constraints on reductions, the divergence between C and GPU backends as a diagnostic tool for reduction ordering bugs, and the flatten/unflatten pattern for irregular parallelism make the retainer work distinct in parallel correctness analysis, backend-specific behavior, and GPU memory transfer optimization. HourTab’s work log makes the associativity audit, scan vs reduce restructuring, and GPU compilation work visible to clients who would otherwise see only the symptom — numerical discrepancies between the CPU and GPU backends — and not understand why the fix required knowing that in Futhark, reduce (+) is a mathematical instruction to compute any associativity-consistent parallel tree sum, not a sequential left-to-right fold, and why switching to reduce_comm or scan is a semantic constraint on the evaluation order rather than a performance annotation.

FAQ: Futhark developer retainers