Blog › ICP guides

dBASE developer on retainer: SET DELETED ON/OFF, dBASE IV, dBASE Plus, and xBase on monthly retainer

October 8, 2026 · ~15 min read

A dBASE developer was maintaining a legacy mailing list management application written in dBASE IV for a regional trade association. The application managed member contact records and produced monthly summary reports: total active members by region, total addresses by state, and total renewals due. The developer ran a cleanup routine each quarter to mark outdated member records as deleted — records for members who had not renewed in 3 or more years. After the cleanup run, the developer generated the monthly summary reports. On three consecutive monthly report cycles, the reports showed totals that were higher than expected: the “total active members by region” count included members who had been marked for deletion. The cleanup routine had run successfully, but the summary reports were not filtering out the deleted records. The association’s membership coordinator flagged the discrepancy when the region totals did not match a manually verified list, and the developer was called in to investigate why three separate summary sections were overstating active membership.

The root cause was SET DELETED OFF — the dBASE default setting. In dBASE IV, the DELETE command marks a record with a logical deletion flag (an asterisk in byte 0 of the record buffer), but does not physically remove the record from the DBF file. The record remains in the file and is accessible to all read operations unless the application explicitly sets SET DELETED ON. When SET DELETED is OFF (the default in older dBASE versions), commands like COUNT, SUM, LIST, DISPLAY, and REPORT FORM include logically-deleted records in their results. The cleanup routine correctly marked the outdated member records with DELETE — but the summary reports ran with SET DELETED OFF, so COUNT returned the full record count including the deleted records. Three regional summary sections showed totals 8–12% higher than the true active-member count because they included deletion-flagged records that the cleanup routine had processed but the reporting routines had never been told to exclude.

The root of the problem is the dBASE logical deletion model. In dBASE (and xBase in general), DELETE is a two-phase operation: the first phase is logical deletion, which sets a deletion flag in the record’s first byte (the deletion marker character *, ASCII 0x2A) while leaving all field data intact and the record addressable by GOTO and SKIP; the second phase is physical removal, accomplished by PACK (which rewrites the entire DBF file, omitting all logically-deleted records, and rebuilds all open index files). Between DELETE and PACK, the record exists in a logically-deleted state that is visible by default to all dBASE read operations. SET DELETED ON changes this: with SET DELETED ON, logically-deleted records are skipped by GOTO, SKIP, LIST, COUNT, SUM, AVERAGE, REPLACE ALL, DELETE ALL, and REPORT FORM. They remain physically in the file (GOTO RECNO() can still position directly to a deleted record’s record number if you know it), but they are invisible to normal navigation and aggregation commands. SET DELETED OFF (the default in older dBASE versions) restores full visibility. RECALL reverses the deletion flag for the current record — or RECALL FOR condition for bulk recall — but only before PACK; once PACK has rewritten the DBF, logically-deleted records are gone permanently and cannot be recalled.

The fix was to add SET DELETED ON before the three summary report routines, and SET DELETED OFF after (to restore the application’s default behavior for interactive browsing, which relied on the user being able to see deletion-flagged records in BROWSE before deciding whether to RECALL or PACK). After the fix, the COUNT, SUM, and REPORT FORM commands skipped all logically-deleted records and returned correct active-member totals. Wrong report totals: 3 → 0 after adding SET DELETED ON before each summary report routine. The investigation — reading the dBASE IV source code to find that SET DELETED was never set explicitly anywhere in the application, confirming that the dBASE default is OFF, understanding the two-phase DELETE / PACK model, and adding SET DELETED ON before each of the three summary report sections — took 1.5 hours.

dBASE, the xBase record model, and logical deletion

The xBase family begins with Vulcan (later called JPLDIS), a database system developed at the Jet Propulsion Laboratory in the mid-1970s. Wayne Ratliff commercialized a descendant of that work as Vulcan, which was then licensed to Ashton-Tate and released as dBASE II in 1980 — the first commercially successful xBase product. dBASE III followed in 1984, adding a faster engine and improved networking support; dBASE III+ (1985) added the RunTime environment and improved multi-user support; dBASE IV (1988) introduced SQL passthrough, multi-tag MDX compound index files, and the dBASE IV Applications Generator. Ashton-Tate was acquired by Borland International in 1991. The dBASE brand changed hands several times; dBASE Inc. (later dBASE LLC) eventually acquired the dBASE brand and continues to develop dBASE Plus — the current, actively maintained product for Windows, which supports 32-bit and 64-bit Windows and provides a modern IDE while retaining compatibility with legacy .PRG, .DBF, .FRM, and .LBL files. Meanwhile, the dBASE III+ and dBASE IV codebase remained in production at thousands of organizations — government agencies, non-profits, small professional associations, and regional membership organizations that built mission-critical applications in dBASE during the 1980s and 1990s and have never migrated. Clipper (by Nantucket, later Computer Associates) and FoxPro (by Fox Software, later Microsoft) were dBASE-compatible compilers that diverged into their own ecosystems with extended language features and higher performance. FoxBase was an early dBASE III-compatible alternative. The term xBase refers to the entire family: dBASE, Clipper, FoxPro, FoxBase, Visual FoxPro, and compatible derivatives that share the DBF file format and core command set.

The DBF file format is the shared foundation of the xBase family. A DBF file is a fixed-length record file: the file header (32 bytes) stores the record count, field count, field descriptor array, and a version byte (0x03 for dBASE III, 0x83 for dBASE III with memo, 0x8B for dBASE IV with memo). Each data record is stored as a fixed-width row prefixed by a single deletion-flag byte: 0x20 (space) means the record is active; 0x2A (asterisk) means the record is logically deleted. All field data remains intact in the record after DELETE — only the first byte changes from space to asterisk. The record count in the DBF header counts all records, including logically-deleted ones; RECCOUNT() returns this total. RECCOUNT() - COUNT() (with SET DELETED ON active so COUNT() returns only active records) gives the number of deletion-flagged records. Index files in the xBase world come in two major formats: NDX (single-tag, one index expression per file, used in dBASE III and dBASE III+) and MDX (multi-tag compound index, introduced in dBASE IV, storing up to 47 named index tags in a single .MDX file associated with the DBF by the same base name). The production MDX file — the production index — is automatically opened whenever its associated DBF is opened with USE; all active tags are maintained on every APPEND, DELETE, RECALL, PACK, and field-level REPLACE. The Visual FoxPro CDX format is structurally similar to MDX and fills the same role in that branch of the family.

The logical deletion model in depth: DELETE marks the current record; DELETE FOR condition marks all matching records (with SET DELETED OFF so that previously-deleted records matching the condition are also evaluated). RECALL reverses the current record’s deletion flag; RECALL FOR condition bulk-recalls all matching deletion-flagged records. ZAP removes all records from the DBF — it is equivalent to DELETE ALL followed by PACK, but much faster because it rewrites only the DBF header (resetting the record count to 0) rather than rewriting the entire file. PACK physically removes all records flagged with * and rewrites the entire DBF, then rebuilds all open MDX and NDX index files. PACK is irreversible — there is no undo after PACK completes. On large DBF files, PACK can take minutes because it reads every record, writes the non-deleted ones to a new file, and then replaces the original. PACK requires exclusive file access: the DBF must be opened with USE filename EXCLUSIVE; attempting PACK on a shared-access DBF in a multi-user environment produces an error (or, in some dBASE versions, silent index corruption). SET DELETED ON and SET DELETED OFF are session-level settings — they affect only the current dBASE session and do not affect other concurrent dBASE sessions accessing the same DBF over a network share. Multi-user dBASE used file-level and record-level locking via SET EXCLUSIVE ON/OFF, FLOCK() (file lock), RLOCK() / LOCK() (record lock), and UNLOCK to coordinate concurrent access.

dBASE programming constructs that appear throughout legacy application code: STORE value TO variable for assignment (equivalent to variable = value, which is also valid); PRIVATE varname and PUBLIC varname for variable scope declarations (PRIVATE variables are local to the current procedure and invisible to called procedures; PUBLIC variables are global across all procedures and remain in memory after the procedure returns); PROCEDURE name and RETURN for subroutine definitions stored in .PRG program files; DO WHILE condition / ENDDO for loops; IF condition / ELSE / ENDIF for conditionals; DO CASE / CASE condition / OTHERWISE / ENDCASE for multi-branch logic; DO procedurefile.prg to call an external program; SET PROCEDURE TO filename to load an external procedure library, making all named PROCEDURE blocks in that file callable via DO procedurename. Report and label generation: REPORT FORM reportname TO PRINTER to print a formatted report defined in a .FRM report definition file; REPORT FORM reportname TO FILE outputfile to redirect to a text file; LABEL FORM labelname for mailing label output from a .LBL definition; CREATE REPORT / MODIFY REPORT for the visual report designer; MODIFY STRUCTURE for interactive schema changes (caution: removing or narrowing a field in MODIFY STRUCTURE destroys the data in that field for all records); INDEX ON expr TO ndxfile for NDX single-tag index creation; SET INDEX TO ndxfile to open an existing NDX index; SET ORDER TO tagname in dBASE IV to select the active tag from the MDX compound index; SEEK value to position to the first matching record in the active index (returns FOUND() = .T. on match, .F. on no match).

Typical dBASE retainer work and what it looks like in a work log

SET DELETED ON/OFF bugs are the most common and most invisible category of dBASE retainer work. The pattern is consistent: a cleanup or archival routine marks records for deletion using DELETE or DELETE FOR condition, confirms that the records are flagged by visually inspecting BROWSE (which shows the deletion asterisk in the leftmost column), and then closes out the cleanup task. The summary and reporting routines — COUNT, SUM, AVERAGE, and REPORT FORM — continue to run with SET DELETED OFF because the application never explicitly sets SET DELETED ON anywhere, and the dBASE default is OFF. The aggregate results include all deletion-flagged records, inflating totals by however many records were marked. The inflation grows each quarter as the cleanup routine marks more records but PACK is never called (because the developer wants the ability to RECALL records if a mistake was made). After several cleanup cycles, the DBF accumulates dozens or hundreds of deletion-flagged records that appear in every COUNT and SUM. Fix: add SET DELETED ON immediately before every summary and reporting routine, and SET DELETED OFF immediately after if the rest of the application depends on the default behavior for interactive browsing. Work log entry: “rpt_summary.prg: SET DELETED OFF (default, never explicitly set); COUNT and REPORT FORM monthly_summary included 47 deletion-flagged records from quarterly cleanup; wrong-total count: 3 regional sections → 0 after adding SET DELETED ON before report routines; PACK not called (recall window still needed); 1.5h.”

PACK timing and exclusivity bugs are the second most common dBASE retainer pattern. PACK is a destructive, file-rewriting operation that requires exclusive access to the DBF and all its associated index files. In a multi-user dBASE environment where the DBF is stored on a network share and accessed by multiple workstations, calling PACK without first opening the file with USE filename EXCLUSIVE produces either a dBASE error (“File is in use by another” or similar) or, in some dBASE IV configurations, executes partially and leaves the MDX index file in an inconsistent state. The most common scenario: a nightly maintenance script runs DELETE FOR INACTIVE = .T. followed immediately by PACK to keep the DBF compact, but the script starts at 6:00 AM when early-arriving office staff are already logged in and have the DBF open for read access. PACK begins, rewrites a portion of the DBF, and then encounters the shared lock held by the early user’s open USE handle, aborting mid-rewrite. The DBF record count in the header is now inconsistent with the actual number of records written, and the MDX index file contains stale tag entries pointing to record numbers that have shifted. Subsequent SEEK operations return wrong records. Fix: move the PACK call to a dedicated maintenance window (e.g., 11 PM Sunday) with a pre-check that confirms no other sessions have the DBF open, and always wrap PACK in USE filename EXCLUSIVE / PACK / USE filename. Work log entry: “nightly_maint.prg: PACK MEMBERS.DBF called without USE EXCLUSIVE; early-session conflict at 6:02 AM; MDX MEMBER_STATE tag returned wrong RECNO() for SEEK on 3 state codes; REINDEX rebuilt tags; added USE MEMBERS EXCLUSIVE before PACK; rescheduled to 11 PM Sunday; 2.5h.”

MDX/NDX index corruption after power failure is the third common dBASE retainer pattern. dBASE IV MDX multi-tag index files are not crash-safe. When dBASE exits normally (via QUIT), it flushes all pending index writes to disk and closes all open DBF, MDX, and NDX file handles cleanly. When the application exits abnormally — power failure, workstation freeze, operating system crash, or a hard reboot — the MDX file may be left in a partially-written state. The MDX B-tree structure for one or more tags is internally inconsistent: some leaf nodes have been flushed to disk with new record number entries, while the root node or intermediate nodes still point to the pre-update structure. On the next application startup, SEEK operations on the corrupted tag traverse the stale B-tree path and land on wrong record numbers, returning incorrect records or no record at all for a value that is present in the DBF. The symptom is SEEK value returning FOUND() = .F. for a key that visually exists in LIST output, or worse, FOUND() = .T. followed by field values that do not match the sought key. The fix is always REINDEX, which rebuilds all open MDX and NDX tags from scratch by scanning every record in the DBF and inserting each active record’s key expression into a freshly-written B-tree structure. On a large DBF, REINDEX requires exclusive access and can take several minutes. Organizations that run dBASE IV on Windows workstations with UPS protection and graceful shutdown procedures rarely encounter this; organizations that run dBASE IV on older hardware without UPS, or where users routinely close the terminal window instead of using QUIT, encounter MDX corruption regularly. Work log entry: “MEMBERS.MDX: REGION_IDX tag corrupted after workstation power failure; SEEK on “NORTHEAST” returned FOUND()=.F.; LIST showed record present; REINDEX rebuilt all 4 tags; SEEK now returns correct RECNO(); recommended UPS and graceful-shutdown procedure documented for client; 1.5h.”

How HourTab tracks dBASE retainer hours

dBASE SET DELETED retainer work is invisible by the same mechanism that makes the bug dangerous: in development and the manual confirmation step after running a cleanup routine, the developer opens BROWSE and sees the asterisk (*) deletion marker in the leftmost column for each newly-flagged record. The asterisk is visible confirmation that DELETE worked. The developer does not also run COUNT or REPORT FORM to verify that the aggregate commands are excluding those same records, because the assumption — reasonable if you are not familiar with the two-phase dBASE deletion model — is that a deleted record is excluded from everything. In a test environment with a handful of sample records, the inflation from including a few deleted records in COUNT may be invisible in a casual visual check: a count of 23 versus the expected 21 is not obviously wrong if you do not have an independent reference. No dBASE error is raised; SET DELETED OFF is a valid and intentional setting. dBASE does not warn that COUNT is including deletion-flagged records. The developer must independently know to ask whether SET DELETED ON is active before every aggregation routine — and for applications written in the late 1980s and early 1990s where the original author may no longer be available, this question is not always asked.

The work log needs to name the mechanism: which .PRG routine, the SET DELETED state before the fix (OFF — default, never explicitly set in the application), the commands affected (COUNT, SUM, REPORT FORM), the DBF name (MEMBERS.DBF), the deleted-record count (47 records), the wrong-total count before and after (3 regional sections → 0), whether PACK was called or not (not called — logical-deletion-only policy to preserve recall window), and the fix type (SET DELETED ON placement before summary routines). A log entry that says “fixed report wrong totals, 1.5h” is not auditable. A log entry that names the routine, the SET DELETED default state, the 47 deletion-flagged records inflating three summary sections, and the exact fix is auditable and defensible. HourTab gives dBASE developers a public retainer-hours URL they send to clients — regional trade associations, government offices, non-profits, and small membership organizations running legacy contact management and mailing list applications built in dBASE III/IV in the 1980s and 1990s, maintained today on dBASE IV on Windows or dBASE Plus. Comparative context: dBASE SET DELETED bugs have structural overlap with adjacent xBase family issues on other legacy platforms. Clipper retainers cover the same xBase/DBF logical deletion model alongside Clipper-specific SEEK index bugs in NDX and NTX index files, where the B-tree traversal returns wrong record positions after index file drift between the DBF data and the index key entries. Visual FoxPro retainers cover xBase variable scope bugs and the FoxPro/VFP variant of the deletion model, where SET DELETED ON is also session-level and where PACK requires exclusive access — the same two-phase model, different CDX multi-tag index format.

FAQ: dBASE developer retainers