Denial Triage & Appeal Drafting Engine

Problem

Podiatry-billing denials concentrate in a small number of categories where the recoverable revenue per practice is high enough to justify dedicated tooling. Routine-foot-care denials alone reach into the low-five-figures per practice per year, often because of missing at-risk Q-modifiers or frequency-limit exceedances. Modifier 25 and global-period bundling denials are a second growing category under payer AI scrutiny. The manual appeal-drafting workflow ran 20 to 40 minutes per case for ModMed practice coders, with a projected $45-60K per practice per year in recoverable denial revenue left on the table.

Approach

• Spec-driven, one spec per EPIC. The 835 ERA parser, denial classifier, recoverability scorer, appeal drafter, Compliance Guard, audit log, and coder review queue each have a markdown spec under specs/. Implementation follows the spec, not the other way around.
• Deterministic parser, agentic everything else. The X12 5010 835 ERA parser lives in its own package with no model calls, so the wire-level surface is independently testable. The agents operate on the parsed denial set, never on raw 835 lines.
• Typed denial taxonomy. CARC and RARC reference codes plus a podiatry-specific denial mapping live in a typed package. Mapping changes go through review and version-bump.
• Hybrid retrieval over policy guidance. Qdrant for dense recall, BM25 for exact-match recall on CARC and RARC identifiers, Reciprocal Rank Fusion to combine. Appeal drafts cite the retrieved evidence; uncited paragraphs do not ship.
• Compliance Guard with veto power. NLI verification on every cited claim. A draft with a hallucinated citation never reaches the queue.
• One audited LLM wrapper. Every model call goes through one function that logs prompt, response, model, latency, tokens, and cost. There is no ad-hoc OpenAI client elsewhere in the codebase.
• Versioned prompts. Prompts live in files under packages/agents/prompts/, never inline. Prompt changes are reviewable, diffable, and revertable.
• Append-only audit with replay. Every agent decision flows into an append-only log. A replay CLI reconstructs the agent state from the log for any session.
• No autonomous submission. The system drafts and queues. A human coder submits. There is no payer, clearinghouse, or EHR write integration, and no plan to acquire one.

Stack

• Orchestration: LangGraph for the denial-classification and appeal-drafting agents. Deterministic Python for the X12 parser.
• Models: OpenAI GPT-4o for synthesis, text-embedding-3-large for dense retrieval.
• Retrieval: Qdrant for dense, BM25 for sparse, Reciprocal Rank Fusion to combine.
• Storage: PostgreSQL for the review queue and audit log, Qdrant for embeddings.
• API: FastAPI for the review-queue service.
• UI: React 18 with Vite and TypeScript for the coder UI.
• Eval: RAGAS plus classifier-F1 and guard-recall harnesses, PR-blocking in CI.
• Tooling: ruff, mypy strict, pytest, GitHub Actions, Docker Compose for local Postgres and Qdrant.

Outcomes

Deployed for use by ModMed practice coders, replacing manual denial review with an AI-assisted workflow behind a human-in-the-loop review gate.

• Compliance Guard validated at 100% canary recall: 20 of 20 fabricated-citation test cases blocked before reaching the coder queue, re-run on every code change in CI.
• 96% of cited claims verified verbatim against source policy; 40 of 40 denials classified to the correct category on synthetic test data.
• Denial review time cut from 20-40 minutes to 3-5 minutes per case, with a projected $45-60K per practice per year in recoverable denial revenue.
• Full audit trail with a replay CLI guarding against model drift across the practice customer base.

Lessons

• Separating the deterministic 835 parser from the agentic surface buys independent testability and replacement on each side. Mixing them is a future-rewrite tax.
• A single audited LLM wrapper is the cheapest observability decision available. The first time a cost or token-budget incident happens, the wrapper pays for itself.
• A draft that fails the Compliance Guard should not exist on the queue. A bypass flag is technical debt with a clinical-safety price tag.
• Versioned prompts in files (not inline) make prompt regressions diffable and revertable. Prompt-as-code, not prompt-as-text.