Blog › ICP guides

Common Lisp developer on retainer: macro hygiene, CLOS method dispatch, gensym, and Common Lisp on monthly retainer

October 3, 2026 · ~14 min read

A Common Lisp developer was maintaining a rule engine for a financial services risk calculation system in SBCL. The engine evaluated a set of risk rules against trade data, and the central evaluation machinery was encapsulated in a macro named with-rule-result. The macro accumulated the result of evaluating each rule into a symbol called result: it expanded to a let form binding result to nil, evaluated the rule body, and returned the final value of result. This pattern had been in production for over a year, used in dozens of call sites across the system.

The developer added a new category of rule — a composite rule that called the rule evaluation function from within a larger orchestration function. The orchestration function tracked its own progress using a local variable also named result: the function accumulated partial results into this variable before the final composite calculation. The with-rule-result macro was called inside this function, after result had already been bound by the function’s own let. The expected behavior: the macro would establish a fresh binding for its internal result accumulator, evaluate the rule, and return the rule output.

What actually happened: because Common Lisp’s defmacro is not hygienic, the symbol result in the macro’s expansion is the same symbol as the result already bound in the calling function’s lexical environment. The macro’s (let ((result nil)) ...) expansion creates a new inner let binding that shadows the outer result, which appears correct — except that SBCL’s compiler, analyzing the expanded code, sees the inner let’s result binding as shadowing the outer one. For rules where the orchestration function’s outer result had already been set to a non-nil value before the macro call, the macro’s internal logic that conditionally assigned to result produced the correct final value in the macro’s binding — but one code path inside the macro body contained an (or result (compute-default)) form that tested result before the macro had set it in the current execution path. In that path, result was the macro’s fresh nil binding, so compute-default ran correctly. But when the outer result had been set to a non-nil value and the macro’s own binding was still nil at that path — wait. Let me be precise: the actual capture was not the inner let shadowing but a setf inside the macro body that referred to a bare symbol result, one that the developer had added to the macro body after the initial let binding, outside the let’s scope. That setf modified the outer result — the calling function’s accumulator — not the macro’s intended internal state. For four rules where the orchestration function’s result had a non-nil initial value at call time, the macro’s setf result ... overwrote it with the rule output, corrupting the orchestration function’s accumulated state. Wrong rule evaluations: 4 → 0 after replacing the bare result symbol in the defmacro body with (gensym "RESULT") at macro-expansion time, ensuring that every call site gets a fresh, unambiguous symbol.

The reason this class of bug is systematically invisible in Common Lisp is that defmacro introduces symbols by name, not by binding. The macro author writes result meaning “my internal variable”, but the expansion substitutes the literal symbol cl-user::result (or whichever package the macro was defined in) into the expansion. If the call site has a lexical binding for that same symbol, the macro’s references resolve to the call site’s binding. There is no warning, no type error, and no diagnostic: the macro expands successfully, the code compiles, and the behavior is incorrect only at call sites where the naming collision exists, which may be a small fraction of all call sites.

Common Lisp: ANSI CL, SBCL, and the Lisp lineage

Common Lisp is the ANSI-standardized dialect of Lisp produced by the X3J13 committee, ratified in 1994 as ANSI INCITS 226-1994. It descends from McCarthy’s original LISP 1.5 through MacLisp (MIT AI Lab), Interlisp (Xerox PARC), and a collection of commercial Lisp dialects including Symbolics Common Lisp, Franz Common Lisp, and Lucid Common Lisp. The standardization effort produced a language that is substantially larger and more specified than any of its predecessors: ANSI Common Lisp defines a standard object system (CLOS — the Common Lisp Object System), a condition and restart system, a complete numeric tower (integers of arbitrary precision, rationals, floating-point in multiple precisions, complex numbers), a comprehensive sequence and array API, a package (namespace) system, and a rich set of I/O facilities. SBCL (Steel Bank Common Lisp), derived from CMU Common Lisp, is the dominant open-source implementation: it provides a native-code optimizing compiler, a type inference engine that generates SBCL compiler notes when type information is insufficient for optimal code generation, an integrated profiler (sb-sprof), and a thread API built on the OS thread primitive.

The macro system is Common Lisp’s most powerful feature and its most dangerous. A defmacro form defines a compile-time code transformation: the macro receives its arguments as unevaluated S-expressions and returns a new S-expression that replaces the original macro call in the source. The transformation is arbitrary Lisp code, which means macros can inspect and rearrange their arguments in ways that no function can. The backquote (`) and comma (,) syntax provide a quasi-quotation facility for constructing expansion templates: `(let ((x ,val)) ,@body) produces a let form with x bound to the runtime value of val and the forms in body spliced into the body position. This is enormously convenient for writing macros but introduces the capture problem: the symbol x in the expansion is a literal, and if the call site has a binding for x, the macro’s references to the internal variable resolve to the caller’s binding.

The standard fix is gensym: (let ((result-var (gensym "RESULT"))) `(let ((,result-var nil)) ...)). gensym generates a fresh, interned symbol guaranteed to be unique across the entire Lisp image — typically named #:RESULT1234 where the suffix is a global counter. Because the generated symbol has never been used before, no call site can have a binding for it, eliminating the capture risk. The idiomatic pattern for multi-symbol macros is to allocate all gensyms at the top of the macro body with let forms before the backquoted template, naming each gensym variable with a -var suffix: (let ((result-var (gensym "RESULT")) (temp-var (gensym "TEMP"))) ...). Scheme’s define-syntax with syntax-rules is hygienic by design — the macro system automatically renames introduced identifiers to avoid capture — but Common Lisp chose to preserve full macro power (macros can inspect symbols, intern new ones, and do arbitrary computation) at the cost of requiring the programmer to manage hygiene manually with gensym.

The Common Lisp Object System (CLOS) is one of the most sophisticated object systems in any language. It is based on generic functions: methods are not attached to classes but defined on generic functions, and dispatch selects the most specific applicable method based on the runtime types of all arguments (multiple dispatch). A generic function may dispatch on any number of its parameters simultaneously — unlike single-dispatch object systems where only the first argument (the receiver) determines method selection. Method combination controls how multiple applicable methods are combined: the standard method combination calls the primary method (the most specific method with no qualifier) and allows :before and :after methods to run before and after it. :before methods run in most-specific-first order; :after methods run in most-specific-last order; neither can control the return value. :around methods wrap the primary method and call call-next-method to invoke the next method in the combination chain. A retainer bug in CLOS method combination: a developer defines an :around method on a generic function but forgets to call call-next-method; the primary method never runs; all callers receive the :around method’s return value, which may be nil or an incomplete result.

CLOS MOP, condition system, and dynamic binding

The CLOS Metaobject Protocol (MOP) extends CLOS with a reflective API for introspecting and modifying the class system itself. The MOP defines metaobjects — objects that represent classes, generic functions, methods, and slot definitions as first-class Lisp objects — and protocol functions for operating on them: class-name, class-direct-slots, class-precedence-list, generic-function-methods, method-specializers, and the like. The MOP is specified in “The Art of the Metaobject Protocol” (Kiczales, des Rivières, Bobrow, 1991) and implemented in SBCL via the sb-mop package. A retainer bug in MOP usage: a developer iterates over class-direct-slots expecting to see all inherited slots, but class-direct-slots returns only the slots defined directly on the class, not inherited ones; the full slot list requires iterating over class-slots (or walking the class precedence list manually); the developer’s serialization code silently omits inherited slot values. Work log entry: “serialize-instance: iterated class-direct-slots, missed 3 inherited slots from base-entity superclass; deserialized objects missing created-at, updated-at, owner-id fields; fix: switched to sb-mop:class-slots; missing fields: 3 → 0; 2h.”

The Common Lisp condition system is not a try/catch exception mechanism: it is a structured protocol for separating error signaling, error handling, and error recovery. The three layers are: signal/error/warn (signaling a condition), handler-bind (establishing a handler that executes while the signaling call stack is still live), and restart-case/invoke-restart (establishing named restarts that represent recovery options and invoking them from a handler). The critical difference from Java’s try/catch is that handler-bind does not unwind the stack when it handles a condition — the handler runs with the call stack that led to the signal still present, which means the handler can invoke a restart established lower in the call stack (inside the signaling function) to perform recovery without unwinding. A retainer bug: a handler-bind handler calls (invoke-restart 'store-default), but the restart named store-default was established in a restart-case that was already unwound by a prior handler-bind that consumed the condition; invoke-restart raises a control-error because the restart is no longer in scope. Diagnosis requires understanding the dynamic scope of restarts (restarts are only active while the dynamic extent of their restart-case is on the call stack) and restructuring the handler-bind/restart-case nesting so that the handler fires while the restart’s restart-case is still live.

Dynamic binding in Common Lisp is implemented through special variables, declared with defvar or defparameter. A special variable has a global binding (its initial value) and can acquire dynamic (thread-local) bindings via let: (let ((*standard-output* my-stream)) (print "hello")) binds *standard-output* to my-stream for the duration of the let body, restoring the previous binding when the body exits. Special variables are conventionally named with surrounding asterisks (“earmuffs”). A retainer bug: a library function establishes a dynamic binding for a special variable and calls a callback; the callback also binds the same special variable, intending to override the library’s binding for its own purposes; but the library’s code reads the special variable after the callback returns, expecting the library’s binding to still be in effect — which it is, because the callback’s inner let restored the library’s binding on exit. No bug there. The actual bug: the developer calls the library function, sets the special variable with setf (modifying the current binding rather than creating a new one), and expects the modification to be visible only inside their function; but setf on a special variable modifies the most recent dynamic binding, which is the library’s binding if the library called the developer’s function within its own let; the library’s subsequent reads see the developer’s value. Fix: replace (setf *var* val) with (let ((*var* val)) ...) to create a new binding rather than mutating the existing one.

Typical Common Lisp retainer work and what it looks like in a work log

Macro hygiene bugs are the largest category of Common Lisp retainer work that produces no visible artifact. The pattern follows the same shape each time: a defmacro introduces a symbol by name in its expansion; the macro has worked correctly at all existing call sites for months or years; a developer writes a new call site inside a function that happens to use the same symbol name as a local variable; the macro’s expansion’s behavior changes at that one call site, producing incorrect results; every other call site continues to work correctly. The only way to detect the bug is to macroexpand the macro at the specific call site ((macroexpand-1 '(with-rule-result ...)) in the REPL) and inspect the expansion for the introduced symbol name. Work log entry: “with-rule-result: setf result inside macro body (outside the let introduced by the macro’s (let ((result nil)) ...)) references caller-scope result in evaluate-composite-rule; macro expansion’s setf result overwrites orchestration function’s result accumulator; 4 rules where result was non-nil at call time returned wrong composite output; fix: moved dangling setf inside let body; replaced result with gensym; wrong evaluations: 4 → 0; 3h.”

CLOS method resolution failures are the second category. A generic function has primary methods defined on multiple classes in a hierarchy; a new subclass is added; the new subclass inherits a primary method from one superclass but the developer expected it to inherit from a different superclass; CLOS’s class precedence list (CPL) determines the order, and the CPL is computed by the C3 linearization algorithm. The developer’s expectation about which method is most specific is wrong because the CPL order for multiple inheritance is not always intuitive. Work log entry: “render-item generic function: SpecialWidget inherits from both BaseWidget and Renderable; expected Renderable’s render-item method to be primary (developer added it last); CPL computed as (SpecialWidget BaseWidget Renderable Standard-Object T) because BaseWidget appears before Renderable in defclass superclass list; BaseWidget’s method is primary; wrong rendering for 6 item types; fix: reordered superclass list to (Renderable BaseWidget) in SpecialWidget’s defclass; wrong renders: 6 → 0; 2.5h.”

Package system visibility bugs are the third category. Common Lisp’s package system provides namespace management: a package is a collection of symbols, and a symbol’s printed name is always relative to the current package. Accessing a symbol from another package requires either importing it or using a fully qualified name (package-name:symbol-name). A retainer bug: a developer use-package’s two packages that both export a symbol with the same name; loading the second use-package form raises a package-error: name conflict and SBCL drops into the debugger asking which symbol to use; the developer selects one; the system compiles; but a third package that was already using the first symbol by that name now gets the second one (or vice versa), producing wrong behavior at runtime. Work log entry: “config-system package uses both :utils and :schema; both export validate; conflict resolved interactively in favor of schema:validate; call sites in config-system that intended utils:validate now call schema:validate; 7 validation failures for config keys that schema:validate rejects with wrong error type; fix: removed use-package :schema, replaced with explicit schema:validate qualified calls; failures: 7 → 0; 2h.”

How HourTab tracks Common Lisp developer retainer hours

Common Lisp retainer work is invisible by the same mechanism that makes macro hygiene bugs dangerous: the SBCL compiler sees no inconsistency between the macro’s definition and its expansion at a call site where a variable capture occurs. The expansion is syntactically valid Lisp, the symbol references are well-formed, and no type error or warning is produced. The code compiles and runs; the incorrect behavior appears only at specific call sites where the naming collision exists, which may be one out of dozens of call sites for the same macro. The symptom — a rule engine returning wrong values for a small subset of inputs — looks like a logic bug in the rule definitions, not a macro expansion issue, which sends initial investigation in the wrong direction.

The work log needs to name the mechanism: which macro, which introduced symbol, which call site, why the symbol resolved to the caller’s binding rather than the macro’s intended binding, and what the rule count change was before and after the fix. A log entry that says “fixed macro bug in rule engine, 3h” is not auditable. A log entry that says “with-rule-result: bare result symbol in setf outside macro’s let body resolves to caller’s result accumulator in evaluate-composite-rule; macro expansion’s setf result overwrites orchestration function’s partial accumulator for 4 rules where result was non-nil at entry; fix: moved setf inside let body, replaced bare symbol with gensym; wrong evaluations: 4 → 0; 3h” is auditable.

HourTab gives Common Lisp developers a public retainer-hours URL they send to clients — financial services firms running ANSI CL risk engines, knowledge representation research groups building inference systems in CLOS, digital humanities projects using Lisp for text analysis pipelines, and game studios maintaining Common Lisp AI scripting systems. For Common Lisp retainers, each work log entry should name the mechanism: which macro or generic function, which call site or specializer combination, what the capture or dispatch failure was, and what inputs demonstrated the incorrect behavior. Comparative context: Common Lisp retainer work has structural overlap with other Lisp and functional paradigms — Scheme retainers cover define-syntax hygiene and closure binding capture, where Scheme’s hygienic macro system prevents variable capture automatically but closure-over-binding bugs (capturing a binding cell rather than a value) produce analogous invisible errors; Haskell retainers involve type class instance selection where overlapping instances produce the same kind of “which binding does this reference resolve to?” question that Common Lisp macro hygiene poses, expressed at the type level rather than the value level; and PicoLisp retainers cover dynamic scoping semantics, where PicoLisp’s dynamic scoping means all variable references use the most recent dynamic binding — a system where every variable is always potentially “captured” by any caller in the dynamic scope, making the Common Lisp hygiene problem visible as the default behavior rather than as an exception.

FAQ: Common Lisp developer retainers