How to Build a Hallucination-Free NotebookLM Knowledge Base for Legacy COBOL Systems

How to Build a Hallucination-Free Noteb

Enterprise legacy modernization projects frequently stall for one fundamental reason: teams feed raw mainframe code into a Large Language Model (LLM) and ask it to guess what the business logic does.

When an LLM lacks deterministic boundaries, it blends what the code says, what the data actually looks like, and what changed over the last decade into a single plausible-sounding narrative. To make AI reliable on mission-critical systems, you must strictly separate empirical facts from AI interpretation.

NotebookLM excels at grounded synthesis when you feed it structured, verifiable artifacts rather than unorganized code dumps. Below is a four-pillar methodology for assembling a deterministic COBOL repository in NotebookLM—covering source code, copybooks, metadata, runtime logs, and version deltas.

Pillar 1: Observed Data & Structural Facts (COBOL Source & Copybooks)

A COBOL program cannot be read accurately in isolation from its data definitions. If an LLM reads a PROCEDURE DIVISION without resolved COPY members, it cannot verify byte offsets, precision, or overlapping memory structures.

Ingestion Standard:

  • Expanded Source Listings: Pair every .cbl program with its fully expanded copybooks (.cpy) so every PIC clause, COMP-3 packed decimal, OCCURS table, and REDEFINES overlay is explicitly visible in the same context window.
  • Explicit File Headers: Prefix every uploaded source file with deterministic metadata tags so NotebookLM can cite exact origins:

t
[ARTIFACT_TYPE: COBOL_SOURCE]
[PROGRAM_ID: FIN204B]
[VERSION: PROD-CURRENT]
[EFFECTIVE_DATE: 2026-10-14]
[COPYBOOKS_INCLUDED: CUSTMAST.CPY, ACCTBAL.CPY]

## Pillar 2: Deterministic Metadata Profiles

Before asking NotebookLM to explain business rules, preload hard statistical and structural facts about the underlying files or VSAM/DB2 datasets. Never rely on generative guesses to infer data behavior.

### Ingestion Standard:

* **Schema & Layout Definitions:** Exact byte lengths, data types, decimal scaling, and primary/foreign key relationships.

* **Empirical Data Profiling:** Actual null/space rates, low/high values, cardinality, and pattern masks extracted from production datasets.

* **CRUD Cross-Reference Matrices:** A deterministic table mapping which programs read, insert, update, or delete specific copybook fields.

| Field Name | Copybook | PIC Clause | Byte Offset | Null/Space Rate | Cardinality | Governing Program(s) |

|—|—|—|—|—|—|—|

| `ACCT-STATUS-CD` | `ACCTBAL.CPY` | `X(02)` | `45-46` | `0.00%` | `6 distinct` | `FIN204B` (R/W), `RPT110A` (R) |

| `CURR-BAL-AMT` | `ACCTBAL.CPY` | `S9(11)V99 COMP-3` | `47-53` | `0.00%` | `1,420,890` | `FIN204B` (R/W) |

## Pillar 3: Operational Reality (Execution Logs & JCL)

Source code represents *potential* execution paths; operational logs represent *actual* production behavior. Dead code, unreachable `PERFORM` paragraphs, and deprecated branches clutter mainframe estates.

### Ingestion Standard:

* **JCL Execution Flows:** Include the Job Control Language (`JCL`) steps, `DD` statements, and dataset dispositions (`DISP=SHR`, `OLD`, `MOD`) to establish execution order.

* **Run Telemetry & Return Codes:** Load sanitized `SYSOUT` logs, step return codes (`RC=0000`, `RC=0004`), abend histories (`S0C7`, `S0C4`), and record input/output counts.

* **Empirical Grounding:** With runtime logs loaded alongside source code, NotebookLM can verify which branches actually execute during nightly batch windows versus dead code from 2008.

## Pillar 4: Version Diffs & Current State (The Change Ledger)

The single biggest failure point in legacy documentation is version drift—mixing up historical behavior with current production logic. Rather than forcing the AI to compare two massive files from scratch, provide explicit diff artifacts alongside the active baseline.

### Ingestion Standard:

1. **Current Production Baseline (`[VERSION: PROD-CURRENT]`):** The single authoritative snapshot currently running in production.

2. **Prior Baseline (`[VERSION: PROD-PREVIOUS]`):** The immediate predecessor version for historical traceability.

3. **Deterministic Diff File (`[ARTIFACT_TYPE: VERSION_DIFF]`):** A unified line-level diff (`diff -u`) annotated with change-ticket IDs, timestamps, and modified paragraphs.

“`text

[ARTIFACT_TYPE: VERSION_DIFF]

[PROGRAM_ID: FIN204B]

[COMPARE: PROD-V2024.3 -> PROD-CURRENT (2026-10-14)]

[CHANGE_TICKET: CHG-88412]

— FIN204B (PROD-V2024.3)

+++ FIN204B (PROD-CURRENT)

@@ -412,6 +412,9 @@

     IF ACCT-STATUS-CD = ’01’

        PERFORM 2100-CALC-STANDARD-INTEREST

+    ELSE IF ACCT-STATUS-CD = ’09’

+       PERFORM 2150-CALC-HOLD-ADJUSTMENT

     END-IF.

Context & Interpretation: How to Query Your NotebookLM Repository

Once your Observed Data / Facts are indexed across these four pillars, structure your NotebookLM prompts to maintain strict epistemic discipline:

  • Trace Lineage Deterministically: “Using only FIN204B, ACCTBAL.CPY, and the SYSOUT logs, trace every paragraph that modifies CURR-BAL-AMT and state the exact input/output record counts from the last batch run.”
  • Isolate Version Deltas: “Compare [VERSION: PROD-CURRENT] against [ARTIFACT_TYPE: VERSION_DIFF] for FIN204B. List every modified conditional branch and cite the corresponding change ticket without speculating on business intent.”
  • Separate Facts from Hypotheses: “Separate what is directly proven by the copybook layouts and runtime logs from any assumptions that still require subject-matter expert confirmation.”

By enforcing metadata discipline, log verification, and explicit version deltas before generating a single summary, you turn NotebookLM from a speculative chatbot into an auditable engineering assistant.

Leave a Reply