Blog › ICP guides

Assembly developer on retainer: x86-64 ABI, SIMD intrinsics, microarchitecture analysis, and low-level systems engineering on monthly retainer

September 18, 2026 · ~22 min read

A media processing pipeline written in C ran at 340 MB/s on the development workstation — an AMD Zen 3 with AVX2 — and 180 MB/s on the Intel Haswell production server, also with AVX2. Both machines compiled the same source with -O3 -mavx2 -mfma. The generated assembly was nearly identical: the inner loop on both issued VFMADD231PS against 256-bit YMM registers. The throughput gap was not a missing ISA extension, not a memory bandwidth ceiling, and not a compiler switch. The Assembly developer on retainer diagnosed it: Haswell has one 256-bit floating-point execution port and achieves a reciprocal throughput of 1 cycle per VFMADD231PS instruction; Zen 3 has two such ports and achieves 0.5 cycles per instruction. Both machines could issue one FMA per cycle in the best case, but the loop had a single accumulator — each FMA depended on the previous FMA's result, forming a 5-cycle latency chain. Zen 3 covered the stall by issuing FMAs on the second port for a different loop body; Haswell had nowhere to send the work and waited 5 cycles per iteration.

The fix was software pipelining. The inner loop was unrolled to ten independent accumulator registers — acc0 through acc9 — each accumulating a partial sum from a different stride of the input array. The minimum unroll factor to saturate Haswell's single FP port at full throughput equals latency divided by reciprocal throughput: 5 cycles latency / 0.5 cycles throughput = 10 independent chains. With 10 independent FMA chains in flight simultaneously, Haswell can issue a new VFMADD231PS every cycle without waiting for any chain's result to be available, because only one out of ten depends on the most recent result. The horizontal reduction at loop exit used _mm256_hadd_ps twice (it does not cross the 128-bit lane boundary, so two passes are required) plus _mm256_extractf128_ps and _mm_hadd_ps to reduce the ten YMM accumulators to a scalar sum. Total work: two days with the Intel Architecture Code Analyzer (IACA) and perf stat to identify the bottleneck and validate the fix. The pipeline went from 180 MB/s to 310 MB/s. No new feature shipped.

An Assembly developer on monthly retainer does this category of work continuously: profiling production kernels with perf stat before a throughput regression reaches an incident, auditing AVX2 intrinsics loops for accumulator dependency chains that would stall on a different microarchitecture than the developer's, advising on vzeroupper placement before an AVX-to-SSE call transition penalty appears in a latency-sensitive path, and reviewing inline assembly constraints before a missing clobber annotation lets the compiler silently corrupt a register the asm statement overwrote. The diff is small. The work behind it is not.

x86-64 assembly, System V AMD64 ABI, and calling conventions

The System V AMD64 ABI defines the calling convention used by every Linux and macOS x86-64 binary. Integer and pointer arguments are passed left to right in rdi, rsi, rdx, rcx, r8, r9; floating-point arguments in xmm0 through xmm7; integer return values in rax; floating-point return values in xmm0. Stack arguments beyond the sixth are pushed right to left and accessed at rbp+16 and above after the standard frame prologue. The callee-saved registers are rbx, rbp, r12, r13, r14, r15 — the callee must save and restore them; the caller can rely on their values surviving a call. The caller-saved registers are rax, rcx, rdx, rsi, rdi, r8, r9, r10, r11 — the callee may overwrite them without notice. A retainer engagement reviewing hand-written assembly routines typically finds at least one instance of a callee-saved register used without being saved first, silent during development because the C caller happened not to have a live value in that register at the call site, and catastrophic when the compiler later reuses that register across the call in an optimized build.

Stack alignment is the most common correctness bug in assembly routines that call C functions. The ABI requires rsp to be 16-byte aligned at the point of a CALL instruction — which means at function entry, after the CALL has pushed the 8-byte return address, rsp is misaligned by 8 bytes. The standard prologue (push rbp / mov rbp, rsp) adds another 8 bytes, restoring 16-byte alignment. Any local variable space allocated with sub rsp, N must keep rsp 16-byte aligned; if N is odd multiple of 8, add padding. An assembly routine that allocates 24 bytes of local space with sub rsp, 24 is misaligned — it should allocate 32 bytes. Misaligned stacks are silent until the called function uses an SSE instruction with a 16-byte aligned memory operand (movaps, _mm_load_ps), at which point the CPU raises a general protection fault. The red zone extends 128 bytes below rsp in leaf functions (functions that do not call other functions) and is preserved across signal handlers, which allows leaf functions to use the red zone for temporary storage without adjusting rsp. A function that uses the red zone and then becomes a non-leaf function (because a call is added to it) must have sub rsp, N added to its prologue — a retainer covers this class of silent regression before it ships.

NASM and GAS syntax differences create friction in cross-toolchain codebases. NASM uses Intel syntax: destination operand first, no register prefixes, memory references in square brackets — mov rax, [rbx+8] reads 8 bytes from the address in rbx plus 8 into rax. GAS uses AT&T syntax by default: source operand first, % register prefix, memory references in parentheses with displacement before — movq 8(%rbx), %rax is the same operation. NASM section declarations are section .text / global _start; GAS uses .section .text / .globl _start. Position-independent code for shared libraries requires RIP-relative addressing: NASM uses default rel at the top of the file and then lea rax, [rel data_label]; GAS uses leaq data_label(%rip), %rax. Linking assembly object files with C requires declaring the assembly function in the C header as extern, implementing it in a .asm file compiled with nasm -f elf64 or as, and linking the resulting .o file alongside the C objects. A retainer engagement covering toolchain integration verifies that the assembly symbols are exported with the correct name mangling (C does not mangle names; C++ does), that the calling convention annotations in the C header match the actual register usage in the assembly, and that the build system invokes the assembler with the correct format flag for the target platform.

SIMD intrinsics — SSE2, AVX2, and AVX-512

SSE2 introduced the 128-bit vector types that remain the universal SIMD baseline: __m128 (4 packed single-precision floats), __m128d (2 packed doubles), and __m128i (128-bit integer vector with lane width selected by the operation). The alignment contract for SSE2 is strict: _mm_load_pd and _mm_store_pd require 16-byte aligned addresses and segfault on misaligned accesses; _mm_loadu_pd and _mm_storeu_pd handle any alignment at a 1–3 cycle penalty on cache-line splits. Scalar broadcasts use _mm_set1_pd to fill all lanes with one value. Masked operations compose a comparison (_mm_cmplt_pd returns a mask vector with all bits set in lanes where the comparison holds) with _mm_and_pd to zero lanes where the condition is false — this is the SSE2 equivalent of predicated execution before AVX-512 opmask registers. A retainer engagement covering SSE2 correctness audits alignment assumptions at every load and store, verifies that comparison-based masks are used with the correct bitwise operation, and checks that the __m128i lane width matches the arithmetic operation (a common error is loading bytes with _mm_loadu_si128 and processing them with _mm_add_epi32 when _mm_add_epi8 was intended).

AVX2 doubles the register width to 256 bits: __m256 (8 packed floats), __m256d (4 packed doubles). The most important AVX2 instruction for compute kernels is _mm256_fmadd_ps(a, b, c) — a single instruction that computes a*b+c with only one rounding step instead of the two separate rounding steps that _mm256_mul_ps followed by _mm256_add_ps would produce. This matters both for floating-point accuracy (one fewer rounding error per FMA) and throughput (one instruction instead of two). _mm256_broadcast_ss(&scalar) fills all 8 lanes of a __m256 with one float value. Horizontal reduction from 8 lanes to a scalar requires: _mm256_hadd_ps(v, v) twice (each pass reduces pairs within 128-bit lanes — after two passes each 128-bit half holds a 4-to-2 partial sum), _mm256_extractf128_ps(v, 1) to get the high 128-bit half, _mm_add_ps to sum the two halves, and two more _mm_hadd_ps passes plus _mm_cvtss_f32 to arrive at a scalar. AVX code that calls SSE functions must issue vzeroupper first — without it, Intel CPUs hold the upper 128 bits of the YMM registers in a dirty state, and the transition from AVX execution mode to SSE execution mode costs 50–200 cycles of serialization. Zen CPUs do not have this penalty but it costs nothing to issue vzeroupper regardless.

AVX-512 extends to 512-bit vectors (__m512, 16 packed floats) and introduces opmask registers k1 through k7 for per-lane predicated execution. Masked loads blend lanes from memory with a source vector: _mm512_mask_loadu_ps(src, k, mem) loads lanes where the corresponding bit in the mask k is set and preserves the src value in lanes where it is clear — enabling loop tail handling without a separate scalar cleanup loop. _mm512_fmadd_ps is the 16-lane FMA. _mm512_reduce_add_ps(v) produces a horizontal sum across all 16 lanes in a single intrinsic call. The 64-byte alignment recommended for _mm512_load_ps avoids cache-line split penalties on loads that would otherwise straddle two 64-byte cache lines. The critical operational caveat for AVX-512 in production: mixing _mm512_* intrinsics with non-AVX-512 code on Intel Skylake and Cascade Lake server CPUs triggers a frequency downclocking of the entire logical core — the CPU drops its clock frequency to a sustained AVX-512 turbo level while any AVX-512 instructions are executing. On some SKUs this is a 200–400 MHz reduction. The impact on a mixed-workload server that runs both AVX-512 SIMD kernels and general-purpose C code can be net negative if the kernel occupies only a small fraction of the runtime. A retainer engagement covering AVX-512 advisory measures the frequency transition penalty against the throughput gain before recommending AVX-512 for any given production fleet.

Inline assembly in C, profiling, and microarchitecture analysis

GCC and Clang inline assembly uses the asm volatile("instructions" : outputs : inputs : clobbers) syntax. Output operands specify where the asm statement writes: "=r"(var) means any general-purpose register, write-only; "=m"(*ptr) means a memory location; "+r"(var) means read-write — the compiler loads the current value into a register before the asm statement and stores the result after. Input operands specify what the asm reads: "r"(var) loads the value into any GPR and makes it available; "m"(var) passes a memory address. The clobber list tells the compiler which registers and state the asm statement modifies outside of the declared operands: "cc" when the asm modifies the flags register (any instruction that sets ZF, CF, OF, or SF); "memory" to act as a compiler memory barrier — the compiler cannot reorder loads or stores across the asm statement, and must assume that the asm may read or write any memory location; a specific register name like "rax" when the asm overwrites that register as a side effect. A pure compiler fence — preventing the compiler from reordering loads and stores without generating any instruction — is asm volatile("" ::: "memory"). Omitting a clobber that the asm statement actually requires produces a code generation bug: the compiler may hold a live value in the clobbered register across the asm statement and observe a corrupted value after it, with no warning at any optimization level.

perf stat is the primary first-pass profiling tool for identifying whether a kernel is compute-bound or memory-bound. The essential counter set: perf stat -e cycles,instructions,cache-misses,cache-references,branch-misses ./binary. IPC (instructions per cycle = instructions / cycles) is the primary health metric: above 2.0 on a modern out-of-order CPU suggests the kernel is executing efficiently; below 1.0 suggests either memory-bound stalls (the CPU is waiting for cache-miss loads) or dependency stalls (execution units are idle waiting for a dependent instruction's result to be available). Cache-miss rate (cache-misses / cache-references) above 10% typically indicates that working set size exceeds L2 or L3 cache capacity and that memory access pattern or tiling optimization is warranted. perf record -g captures call-graph samples and perf report presents the hotspot hierarchy; perf annotate shows sample counts per assembly instruction for the selected function, identifying which specific instructions are accumulating the most cycle-wait time. A retainer engagement covering perf profiling runs these tools against the production binary (or a representative synthetic workload if production access is restricted), identifies the hottest function, and delivers a diagnosis that distinguishes compute-bound from memory-bound before recommending which optimization technique to apply.

Agner Fog's instruction tables provide latency and reciprocal throughput for every x86-64 instruction on every major microarchitecture. Latency is the number of cycles from when an instruction's inputs are ready until its output is available for a dependent instruction. Reciprocal throughput is the minimum number of cycles between issuing two independent instances of the same instruction — the inverse of issue rate. VFMADD231PS on Haswell has a latency of 5 cycles and a reciprocal throughput of 0.5 cycles (2 FMAs per cycle from two execution ports). On Zen 3, same values. The key insight: the minimum number of independent instruction chains needed to fully saturate execution units is latency divided by reciprocal throughput — for Haswell's FMA unit it is 5 / 0.5 = 10 independent FMA chains. Software pipelining interleaves these 10 chains across loop iterations so that when one chain stalls waiting for its FMA result, nine other chains have independent work ready to issue. uiCA (available at uica.uops.info) accepts a loop body in assembly text, a microarchitecture selection, and returns a predicted throughput per iteration that can be compared against measured perf stat output to confirm whether the loop is actually achieving its theoretical execution-unit limit or is limited by some other bottleneck — instruction fetch width, register file ports, or store buffer capacity. A loop that is compute-bound with throughput matching the uiCA prediction has no room for further optimization from loop restructuring; a loop that is slower than the uiCA prediction has an unanticipated bottleneck worth investigating.

How HourTab tracks Assembly developer retainer hours

Assembly and SIMD retainers produce some of the starkest work-to-diff ratios in all of software. A session that diagnosed the 180 MB/s to 310 MB/s throughput regression, identified the single-accumulator latency chain as the root cause, derived the 10-chain unroll factor from Agner Fog tables, implemented the restructured inner loop with correct horizontal reduction, and validated with perf stat and IACA produced a diff of roughly 30 lines — an unrolled loop plus a hadd-based horizontal reduction. The diff has no comment explaining that Haswell has one FP port, no comment explaining why 10 chains specifically, and nothing that connects the loop restructuring to the 72% throughput improvement. The log entry “optimized FMA loop, 2d” leaves the client with no way to understand what was done, why it was done on this microarchitecture but not another, or how long this class of work typically takes — which means the retainer renewal conversation starts from scratch every cycle.

HourTab gives Assembly developers a public retainer-hours URL that the client opens to see the current burn-down without asking. For SIMD retainers specifically, the work log format carries more information than the burn-down number: each entry should name the function and the microarchitecture that surfaced the issue (process_frame(), Haswell production vs Zen 3 dev), cite the profiling output that identified the bottleneck (IACA: loop bottleneck Port 1 FP; perf stat: IPC 0.84, confirming compute-stall not memory-bound), state the root cause in terms the client can look up (single FMA accumulator creates 5-cycle latency chain; Haswell issues 1 FMA/cycle instead of 1/0.5 = 2 because the next FMA depends on the previous result), give the fix with the mathematical derivation (unroll factor 10 = latency 5 / throughput 0.5; 10 independent accumulator chains fill Port 1 every cycle), and close with the before/after metric from perf stat (180 MB/s → 310 MB/s; IPC 0.84 → 1.82). That entry takes ten minutes to write from the notes already taken during the profiling session and makes the client check-in a two-sentence confirmation rather than a twenty-minute explanation of what a microarchitecture execution port is and why Haswell and Zen 3 behave differently on the same AVX2 source code.

The retainer model fits Assembly platform engineering because production server fleets are not static. A new instance type in the fleet introduces a different microarchitecture with different execution port configurations and different L3 cache latencies — kernels tuned for the previous fleet may perform differently on the new hardware, and identifying which ones require re-profiling requires the ongoing relationship and codebase context that a monthly retainer provides. ISA extensions evolve similarly: a fleet upgrade from Skylake to Ice Lake adds AVX-512 and VNNI; whether to use them requires an analysis that accounts for the frequency downclocking penalty on the transitional SKUs still in the fleet. A project contract closes when the current optimization milestone is delivered. An Assembly retainer stays open for the next perf stat regression triggered by a new server instance type, the next AVX-SSE transition penalty surfaced by a new call path, and the next inline assembly clobber omission introduced by a developer who added a register use to an existing asm block without updating the constraint list.

FAQ: Assembly developer retainers