Codebase Impact Graph for Safer Changes in ObjectScript Systems

Summary

As engineers increasingly use AI-powered coding agents, the quality of the agent’s context becomes a deployment risk. If an agent changes one symbol without understanding the other symbols, macros, generated methods, inherited methods, or tests that depend on it, the workflow becomes fast but unsafe.

This project addresses that gap for ObjectScript-heavy IRIS codebases. It builds deterministic dependency context that can be inspected by humans or queried by coding agents, so AI-assisted feature work, bug fixes, test updates, and remediation workflows have a better map of where to look.

Context

ObjectScript is a specialized language used by InterSystems and InterSystems customers. Its ecosystem does not have the same public tooling coverage as mainstream languages, which limits the usefulness of generic codebase graph builders for ObjectScript-heavy systems.

ObjectScript source can include macros and preprocessor directives that act as textual substitutions, constants, or code snippets defined elsewhere. Some relationships are only visible after IRIS compilation expands those constructs into compiler-generated intermediate artifacts.

Problem

Without a language-aware dependency graph, engineers and AI coding agents are pushed toward manual tracing, grep, or overloading an LLM context window with source snippets. That is error-prone, token-inefficient, and particularly weak around ObjectScript macro expansion and generated behavior.

The core problem was to build a context engine that can answer: if this symbol changes, what other symbols may be affected, what edges explain that relationship, and where should a human or coding agent inspect next?

Constraints

• Use public-safe architecture descriptions only.
• Do not include InterSystems source code, internal repo paths, raw compiler output, or proprietary class names.
• Do not use an LLM to infer graph edges during graph construction.
• Preserve macro provenance from source parsing while also using compiler-generated artifacts to observe expanded dependencies.
• Require an accessible IRIS instance for compilation-backed graph construction.
• Describe the tool as locally used, not as a hosted service or automated pipeline.

My role

I designed and built the tool as an internal ObjectScript dependency context engine for engineering and AI coding workflows. The work combined source parsing, IRIS compilation, compiler-derived signals, graph construction, command-line querying, and MCP access for coding agents.

System design

The tool analyzes ObjectScript-related artifacts such as `.xml`, `.cls`, `.mac`, `.rtn`, `.inc`, and compiler-generated `.int` files. It builds a complete call-dependency graph for a target codebase at the symbol level.

Source parsing is used to preserve information that compilation can erase, such as where a macro came from. IRIS compilation is used to expose relationships that static parsing alone cannot reliably see, including macro-expanded calls, generated methods, inherited methods, and configuration-sensitive expansions.

Graph model

• Nodes represent source artifacts and symbols, including method-level and property-level abstractions.
• Edges represent call dependencies, includes, macro-derived relationships, inheritance-derived relationships, generated methods, and compiler-observed relationships.
• The graph is designed to explain dependencies, not to claim behavioral understanding by itself.
• A coding agent can use the graph to decide which symbols to inspect, then read those symbols directly for behavior-level reasoning.

Static analysis

Static source parsing extracts symbols, source-defined relationships, includes, macro references, and provenance that would be lost after compiler expansion. This is necessary because compiler-generated artifacts can show the expanded dependency while no longer preserving the original macro source in a form that is sufficient for explainable context.

Static analysis alone is not enough. ObjectScript macros, conditional compilation, generated methods, inherited methods, and environment- or version-sensitive definitions mean the real dependency surface can differ from what a source-only parser can see.

Compiler-derived signals

IRIS compilation is part of the graph construction loop because compiler-generated INT artifacts contain expanded macro output. That expansion can reveal the actual symbols being called after preprocessing.

Compiler-derived signals also help with relationships that do not exist plainly in source code, including generated methods, inherited methods, context-sensitive macros, and calls introduced by compile-time behavior. Without compilation in the loop, a graph can miss exactly the dependencies that make ObjectScript impact analysis difficult.

Query workflow

Users can query individual symbols, inspect individual edges between symbols, and compare generated call-dependency graphs before and after a change. The same graph can be queried directly from the command line or exposed through an MCP server so coding agents can request targeted dependency context.

A typical coding-agent workflow is: ask the graph where to look, inspect the returned symbols and edges, then use source context to reason about the actual behavioral implications of a proposed change.

AI boundary

AI was useful in building the system, but not in graph construction itself. The graph is constructed deterministically because using an LLM to infer dependencies in a complex ObjectScript codebase would introduce non-determinism and hallucination risk.

The tool is meant to serve AI coding agents, not to delegate dependency extraction to them. That boundary is important: coding agents should consume the graph as structured context and then reason over the source, rather than inventing the graph from partial context.

Reliability and uncertainty

The design separates dependency surfacing from behavioral interpretation. A call edge tells an engineer or agent that one symbol may need inspection because of a relationship; it does not, by itself, explain the full behavior of that symbol or prove that a change is safe.

That separation keeps the tool useful without overstating what static and compiler-derived analysis can know.

Impact

The tool provides deterministic dependency surfacing for ObjectScript-heavy systems and makes AI coding workflows more token-efficient. Instead of loading broad source regions or relying on grep, agents and engineers can query targeted symbols and edges.

The practical impact is faster dependency context gathering, better reviewer orientation, clearer blast-radius reasoning, safer test targeting, and reduced risk of missed dependency gaps when using AI coding agents.

Tradeoffs

The tool trades convenience for correctness by requiring IRIS compilation in the loop. That makes setup heavier than a pure offline parser, but it is necessary for a complete ObjectScript dependency picture.

It also deliberately avoids using AI for dependency extraction. That makes the graph less magical, but more auditable and repeatable.

Limitations

The main limitation is environmental: complete graph construction requires access to an IRIS instance so the tool can compile relevant artifacts and observe compiler-derived relationships. A fully accurate offline graph is not realistic for this codebase.

The graph also does not claim to explain actual symbol behavior. It identifies dependency context and blast-radius candidates; humans or coding agents still need to inspect the relevant symbols to understand behavior-level consequences.

What I learned

The safest AI coding workflows often need deterministic context systems around the model. For ObjectScript, that means respecting the language, compiler, and generated artifacts instead of pretending a generic parser or a large prompt can recover all relevant dependencies.

The project reinforced a broader AI deployment lesson: when the domain has specialized semantics, the highest-leverage work may be building the context engine that lets the AI system operate with less guesswork.

What I would improve next

• Continue validating against historical changes, SME review, manual comparison, and known dependency gaps.
• Improve graph-diff workflows for comparing dependency surfaces before and after a change.
• Add more feedback loops from coding-agent usage to identify missing edges, noisy edges, and high-value query patterns.