ADR-0021: Native storage for ordinary node foreign keys

Status: native writes, root reads, storage witnesses, direct Include and Blogging BLOG-07/08 verified in hosted run 38118649416: 927 total tests, including nine provider Include and three sample relationship/Include facts. Restricted-user deployment and M7 remain separate open gates.

Blogging's ordinary Blog.Posts / Post.Blog relationship is not a many-to-many join. An explicit join entity could demonstrate existing native association support, but would not enforce one Blog per Post or satisfy the sample contract. Preserve ordinary EF relationship metadata and the Post entry as the only owner of its persisted state.

ToGraphReference(relationshipType) annotates an EF ReferenceCollectionBuilder foreign key. Direction is principal to dependent: (blog)-[:HAS_POST]->(post). The first contract allows one required, non-owning, nonunique many-to-one FK per dependent, references the principal node's primary key, and requires compatible ordered model/wire keys and converter instances. FK slots cannot belong to dependent identity, concurrency or generated state. Delete behavior is explicitly Restrict or NoAction. Optional/unique references, ambiguous relationship-type sharing and cycles reject rather than acquire accidental semantics.

The finalized EF model remains authoritative. NodeBinding.Properties retains all EF slots, including Post.BlogId; StoredProperties excludes derived FK slots. An immutable NodeReferenceBinding links the real IForeignKey, principal/dependent node bindings, relationship type and ordered property mappings. The FK is read from the relationship's principal key, never independently copied onto the dependent node. EF still performs identity resolution, relationship snapshots and navigation fixup.

The write planner excludes derived FKs from node storage and composes owned membership operations with existing scalar node commands. Root reads now derive those slots from the immediate principal key; schema witnesses describe the same physical contract. The original whole-model execution gate is removed. Dependent-root aggregates reject before transport; direct Include has separate live acceptance. The explicit native DTO lane remains separate and does not interpret these mappings.

Runtime acceptance contract

Create endpoint nodes before their owned links in the same transaction. A Blog and two Posts must save three EF entries, three nodes and two native relationships; graph operation counts must not replace EF entry counts. Validate the complete plan before transport, require endpoint key constraints and existing principals, and retain deferred generated values until success. Serialize membership creation using dependent/endpoint locks before checking cardinality. Provider/provider races must not create multiple parents; failures roll back the whole save.

The implementation adds a membership phase after all node creates. Updates/deletes lock the dependent and original principal, then assert/delete the bounded incoming membership before the existing scalar command. Any later concurrency or restrictive-deletion failure rolls back the whole transaction, including link deletion. Node deletes sort deepest dependents first along the validated acyclic reference chain. Distinct EF entry reference identity drives save counts and failure entry lists; only node operations carry generated values. Lock order can still deadlock under competing multi-entry transactions; failure/rollback is supported, automatic retries are not.

The initial runtime contract rejects FK reassignment, including navigation-driven changes, before writes. A payload update must verify the original principal and exactly one incoming mapped relationship before applying normal optimistic concurrency. Deleting a Post validates and removes only its own incoming relationship and then the node atomically; unrelated relationships still make plain node deletion fail. Deleting a Blog remains restrictive and must not silently delete Posts or their links. No hidden join entries, second tracker, implicit cascade or DETACH DELETE is introduced.

Dependent queries begin with the dependent node so a required orphan cannot disappear through an inner relationship match. Inspect at most two incoming physical links before principal-label filtering; zero links, duplicates, multiple parents, wrong-label principals and malformed/noncanonical principal keys fail before EF shaping. Preserve these checks for scalar projections and paging. Count/Any need their own aggregate corruption diagnostic or explicit rejection because the existing aggregate path bypasses ordinary row decoding. Diagnostics cover observed/returned data under read-committed semantics, not a whole-graph integrity guarantee.

The read implementation exposes no arbitrary parent when raw membership is invalid: the parent alias is null unless exactly one physical link has the expected source label. FK predicates may consequently exclude malformed dependents, and pagination may exclude them as well. Dependent-ID or payload queries that return those rows still fail before EF shaping, including scalar projections and tracked identity reuse. Immediate principal key validation does not recursively audit that principal's own incoming reference chain.

The manifest must describe dependent ownership and derived FK storage, change the fingerprint when this relationship contract changes, and exclude indexes/keys that would falsely imply a stored FK property. Neo4j node-key uniqueness does not enforce one-parent relationship cardinality; schema comparison must report the application-enforced invariant explicitly.

The witness now uses DependentNode identity, principal-to-dependent endpoint keys, requiredIncomingLinks: 1 and remove-with-dependent-restrict-principal deletion. Logical derived-FK slots remain immutable review metadata outside canonical JSON; ordered physical principal keys and relationship semantics determine the fingerprint. Existing node/edge canonical serialization is preserved. Comparison warns that link cardinality, direction, labels and key completeness have not been established by schema inspection, and proposes no fictional relationship cardinality constraint.

Include acceptance contract

Readonly backing-field correction (2026-10-11): valid finalized EF navigations can select an init-only field for materialization. Include fixup must use EF's provider-facing expression assignment helper, not raw Expression.Assign, which rejects such fields. The existing supported mapping contract is preserved; no field restriction is added. Both Include directions and all three tracking modes receive compilation and live materialization/identity regression coverage; execution status is in the tracker.

Inspect the actual EF 10.0.12 navigation-expanded expression before implementing translation. The pinned source at .research/efcore/src/EFCore/Query/Internal/NavigationExpandingExpressionVisitor.ExpressionVisitors.cs emits correlated Where plus MaterializeCollectionNavigationExpression for collections and a Join for the required dependent reference, wrapped in IncludeExpression. Existing GraphBridge subquery/Join translation and shaping do not support these forms.

The matching EF InMemory projection/shaper visitors provide source examples for collection shaping, loaded flags and tracking/no-tracking fixup; their private helpers are not provider APIs and the InMemory provider is not a runtime dependency. Reuse EF materializers and identity services rather than attaching manually populated objects or maintaining a second tracker.

First prove unfiltered Blogs.Include(blog => blog.Posts) and Posts.Include(post => post.Blog). Root ordering and pagination apply before collection expansion; empty collections survive; each root is yielded once without duplicate children or silently truncated collections. Test inverse fixup, navigation loaded state, tracking/no-tracking/identity resolution, malformed cardinality, cancellation and cleanup. Required-reference Include must not silently drop orphan dependents through its expanded Join. Unsupported filtered/nested Include shapes reject explicitly. Only after these gates pass should the executable expose its HAS_POST mode and BLOG-07/08 be marked complete.

The implementation extracts one direct typed Include before navigation expansion, preserves base EF preprocessing and carries a provider annotation into translation. One Cypher command selects/paginates roots before collecting children. Collection targets are matched without a label filter so wrong-label targets fail diagnostics instead of disappearing. Each root and its entire included collection are decoded before either EF materializer runs for that root. Root streaming remains; memory consumption is proportional to a complete collection, with no silent cap or truncation. Previously yielded roots can already be tracked when a later root fails.

Separate EF materializers share the query's EF state manager. Tracked fixup preserves pending local values and FK changes; explicitly marking a still-unresolved tracked reference loaded would be wrong, so reference loaded state follows actual EF fixup. No-tracking modes use finalized metadata accessors for CLR navigation fixup. Reference Include does not mark its partial inverse collection loaded. Initial support rejects multiple, nested, filtered and string Includes, nonentity results, target query filters/AutoInclude and reference targets requiring a further parent diagnostic. These limits remain until separately implemented and verified.