SIMULATION LAYER 01 / 04
Between your export and a panel there are eight stages, and none of them throw anything away.
A spreadsheet written for a human reader is not a dataset. It has a title block, merged cells, subtotals in the middle, a scale note in prose and a fiscal calendar that is not January to December. Every stage below exists because one of those things breaks a naive read, and each stage records what it did so a figure can be walked back to the cell it came from.
SIMULATION LAYER 02 / 04
The eight stages
stage 01
Land
The raw bytes are stored exactly as received, before any code interprets them. Everything downstream is a derivation, so the original is always available to re-derive from when a parser improves.
Why it matters: a fix to the parser six months from now must be able to reproduce every historical figure. That is only possible if the input was never mutated in place.
stage 02
Parse
A full-fidelity read: every sheet, every row, text cells retained alongside numeric ones. Bracketed negatives, currency symbols and grouped digits are handled as notation rather than left to a type guesser.
This is deliberately ours rather than the engine's. The engine's ingester caps sheets and rows and keeps numeric cells only — correct for a language-model digest, fatal for a general ledger.
stage 03
Discover
Where the table actually begins, which columns are headers, which rows are group subtotals rather than data, and which blocks are separate tables sharing one sheet.
A subtotal counted as a row double-counts the group. This is the most common way a plausible-looking wrong number gets made.
stage 04
Profile
What each column is, established per column: an account code, a date, an amount, a vendor name, a free-text narration. Fan-out is per column rather than one array shared by every consumer.
One shared numeric array is enough to summarise a document. It is not enough to state a fact about a specific account in a specific period.
stage 05
Calendar
Your fiscal year, your period ends, your labels. An Indian financial year runs April to March; a quarter labelled Q1 means different months in different companies, and a projection three quarters out is meaningless without knowing which.
Native granularity is carried through — day, week, month or quarter — because a model trained on the wrong granularity produces a confident, wrong band.
stage 06
Normalize
Units, signs and scale factors resolved. "Figures in ₹ lakh", written once in a title cell three rows above the numbers it governs, is a hundred-thousand-fold error if it is missed.
INR is the primary currency and the notation follows it: lakh and crore grouping, and the rupee symbol in panel prefixes.
stage 07
Reconcile
Does it add up. Where two documents disagree, or a total does not equal its parts, the discrepancy is surfaced with both readings rather than silently averaged.
A reconciliation difference is information about your books. Hiding it to make a dashboard look clean destroys the most useful thing in the upload.
stage 08
Assemble
Measures become panels: one snapshot, one shape, one schema version. Panels without the measures they need are locked, with the document that would unlock them named.
The client refuses a payload whose schema version differs from the one it was built against, rather than rendering undefined into a figure.
SIMULATION LAYER 03 / 04
The engine seam
One module touches the engine. Every import inside it is function-local.
The forecasting engine is vendored as a read-only mirror of the platform monorepo, not forked. A bug found in it is fixed upstream and pulled down; an in-place edit fails a blocking CI gate. Deliberate divergence is allowed, but it has to be declared and moved out of the mirror.
Nothing in the product imports the engine at module level, so starting the API does not load a machine-learning stack. That is enforced three ways, including an import test run with the vendored tree renamed away.
Bridged capabilities
- Forecast a seriesReturns the vendored payload verbatim. The gate that decides whether to trust it lives on our side, because it is a product decision.
- Classify and ingest a documentRight for non-tabular files and for working out what a document is. Not the path for accounting-grade extraction.
- Extract numeric panels from textA quick numeric read. Facts come from our own per-column fan-out instead.
- Check figures against groundingSubstitution-aware: catches a number drifting, not just a missing citation.
- ProbeReports which capabilities are importable rather than raising. A missing capability is a degraded product, not a dead one.
bi/engine_bridge.py
SIMULATION LAYER 04 / 04
Next: what each figure carries with it.
Provenance, formula, drivers, line items and sources — and the five-stage trace that puts them in order.