srdatalog.ir.hir.lower

HIR -> MIR Lowering (Pass 6). Phase 1+2.

Phase 1 (shipped): single-clause variant case + helpers. Phase 2 (this file): multi-clause lowering (ColumnJoin per join var + one CartesianJoin for the remaining independent vars), negation patterns, and the four maintenance generators (rebuild/merge indices, simple + loop maintenance).

Phase 3 (future): stratum wrapping (ExecutePipeline / Block / FixpointPlan / Program, parallel groups, schema arities, before/after hooks) and a Nim-side tool that emits MIR S-expr so we can do end-to-end byte-diff.

Deliberately NOT ported (no Python DSL equivalent or deferred optimization):

• Binary-join / materialized-join dispatch (alternate dialects)

• Balanced-scan (balancedRoot / balancedSources pragmas)

• Split rule lowering (SplitClause + tempRelName)

• IfClause (Filter) / LetClause (ConstantBind) / AggClause handling

• InnerPipeline / debug-hook injection

Module Contents

Functions

generate_column_source	Mirror Nim generateColumnSource. Used by Phase 2 ColumnJoin lowering.
generate_insert_into	Mirror Nim generateInsertInto. Emits to NEW_VER (always) with the stratum’s canonical index for the head relation.
generate_loop_maintenance	Maintenance at the end of a fixpoint iteration.
generate_merge_indices
generate_rebuild_indices
generate_scan	Mirror Nim generateScan. Produces the Scan node used as the outer iteration driver when a variant has a single body clause.
generate_simple_maintenance	Maintenance for a non-recursive (simple) SCC: build canonical NEW, size-check, compute delta, clear NEW, rebuild non-canonical DELTAs, merge every index into FULL.
lower_hir_to_mir	Top-level lowering entry: HirProgram -> MIR Program.
lower_hir_to_mir_steps	Assemble per-stratum FixpointPlan + PostStratumReconstructInternCols steps. Returns the flat [(node, is_recursive)] sequence consumed by lower_hir_to_mir (which wraps them in a Program).
lower_split_above	Mirror lowerSplitAbove in lowering.nim. Supports single-positive-clause above-split (Scan + negations + filter/let), which covers the negation- pushdown use case. Returns empty list if multi-positive above-split is encountered (caller falls back to full pipeline).
lower_split_below	Mirror lowerSplitBelow: Scan(temp) + CartesianJoin(below sources, prefix = temp vars that they share) + below negations + below filters
lower_variant_to_pipeline	Lower a rule variant to an MIR pipeline.
wrap_in_execute_pipeline	Wrap a pipeline body in an ExecutePipeline node, extracting source specs (flattened through ColumnJoin/CartesianJoin) and dest specs (InsertInto nodes).

API

srdatalog.ir.hir.lower.generate_column_source(pattern: srdatalog.ir.hir.types.AccessPattern) → srdatalog.ir.mir.types.ColumnSource

Mirror Nim generateColumnSource. Used by Phase 2 ColumnJoin lowering.

srdatalog.ir.hir.lower.generate_insert_into(head: srdatalog.dsl.Atom, canonical_index: list[int]) → srdatalog.ir.mir.types.InsertInto

Mirror Nim generateInsertInto. Emits to NEW_VER (always) with the stratum’s canonical index for the head relation.

srdatalog.ir.hir.lower.generate_loop_maintenance(rel_name: str, indices: list[list[int]], canonical_index: list[int], arity: int, full_needed: set[tuple[int, ...]] | None = None) → list[srdatalog.ir.mir.types.MirNode]

Maintenance at the end of a fixpoint iteration.

full_needed is the set of (completed) indices whose FULL-version is actually read by joins in this SCC — only those (plus the canonical index) need MergeIndex into FULL. Others just get their DELTA rebuilt.

srdatalog.ir.hir.lower.generate_merge_indices(rel_name: str, indices: list[list[int]]) → list[srdatalog.ir.mir.types.MirNode]

srdatalog.ir.hir.lower.generate_rebuild_indices(rel_name: str, indices: list[list[int]], version: srdatalog.ir.hir.types.Version) → list[srdatalog.ir.mir.types.MirNode]

srdatalog.ir.hir.lower.generate_scan(pattern: srdatalog.ir.hir.types.AccessPattern, bound_vars: list[str]) → srdatalog.ir.mir.types.Scan

Mirror Nim generateScan. Produces the Scan node used as the outer iteration driver when a variant has a single body clause.

bound_vars is accepted for API parity with Nim; the Nim implementation does not use it either (all binding context comes from the access pattern’s own access_order / prefix_len).

srdatalog.ir.hir.lower.generate_simple_maintenance(rel_name: str, indices: list[list[int]], canonical_index: list[int], arity: int) → list[srdatalog.ir.mir.types.MirNode]

Maintenance for a non-recursive (simple) SCC: build canonical NEW, size-check, compute delta, clear NEW, rebuild non-canonical DELTAs, merge every index into FULL.

srdatalog.ir.hir.lower.lower_hir_to_mir(hir: srdatalog.ir.hir.types.HirProgram) → srdatalog.ir.mir.types.Program

Top-level lowering entry: HirProgram -> MIR Program.

Does NOT run the MIR optimization passes (pre_reconstruct_rebuild, clause_order_reorder, etc.) that Nim’s compileToMir runs afterwards. The Nim-side tool used for golden fixtures also dumps pre-pass MIR, so byte-diff lines up.

srdatalog.ir.hir.lower.lower_hir_to_mir_steps(hir: srdatalog.ir.hir.types.HirProgram) → list[tuple[srdatalog.ir.mir.types.MirNode, bool]]

Assemble per-stratum FixpointPlan + PostStratumReconstructInternCols steps. Returns the flat [(node, is_recursive)] sequence consumed by lower_hir_to_mir (which wraps them in a Program).

Mirrors lowerHirToMirSteps in lowering.nim. Deliberately not ported from Nim (see module docstring): before/after hooks, split-rule dispatch, InjectCppHook for debug code, balanced scan.

srdatalog.ir.hir.lower.lower_split_above(variant: srdatalog.ir.hir.types.HirRuleVariant, stratum: srdatalog.ir.hir.types.HirStratum) → list[srdatalog.ir.mir.types.MirNode]

Mirror lowerSplitAbove in lowering.nim. Supports single-positive-clause above-split (Scan + negations + filter/let), which covers the negation- pushdown use case. Returns empty list if multi-positive above-split is encountered (caller falls back to full pipeline).

srdatalog.ir.hir.lower.lower_split_below(variant: srdatalog.ir.hir.types.HirRuleVariant, stratum: srdatalog.ir.hir.types.HirStratum, temp_version: srdatalog.ir.hir.types.Version = Version.FULL) → list[srdatalog.ir.mir.types.MirNode]

Mirror lowerSplitBelow: Scan(temp) + CartesianJoin(below sources, prefix = temp vars that they share) + below negations + below filters

• InsertInto(head).

temp_version switches between the non-recursive default (FULL — the temp is merged into FULL by standard maintenance before the Scan) and the recursive inner-loop variant (NEW — temp is repopulated each iteration and consumed directly from NEW).

srdatalog.ir.hir.lower.lower_variant_to_pipeline(variant: srdatalog.ir.hir.types.HirRuleVariant, stratum: srdatalog.ir.hir.types.HirStratum) → list[srdatalog.ir.mir.types.MirNode]

Lower a rule variant to an MIR pipeline.

For a single-clause variant: Scan (+ Negation*) + InsertInto. For a multi-clause variant: ColumnJoin* + CartesianJoin? + Negation* + InsertInto.

srdatalog.ir.hir.lower.wrap_in_execute_pipeline(pipeline: list[srdatalog.ir.mir.types.MirNode], clause_order: list[int], rule_name: str, use_fan_out: bool = False, work_stealing: bool = False, block_group: bool = False, count: bool = False, dedup_hash: bool = False) → srdatalog.ir.mir.types.ExecutePipeline

Wrap a pipeline body in an ExecutePipeline node, extracting source specs (flattened through ColumnJoin/CartesianJoin) and dest specs (InsertInto nodes).