Skip to content

Formal evaluation semantics aligned with REST-VKG: flat sequences, XSLT focus, SPARQL/XPath conformance - #25

Merged
namedgraph merged 15 commits into
mainfrom
feat/formal-semantics-evaluation
Oct 3, 2026
Merged

namedgraph merged 15 commits into
mainfrom
feat/formal-semantics-evaluation

Conversation

@namedgraph

@namedgraph namedgraph commented Jul 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Turns formal-semantics.md from a type catalog into an actual semantics, aligns it with the Java/XML implementation in REST-VKG, and brings the Python implementation and test suite up to it. The evaluation model is stated as XSLT/XPath applied to RDF, with SPARQL contributing the data model and the term-level function library.

Spec

  • Formal definition (§3.8): abstract syntax, semantic domains (environment, focus, world), the big-step judgment ρ, φ ⊢ ⟨e, σ⟩ ⇓ ⟨v, ρ′, σ′⟩ with the full rule set, operator effect classes, and metatheory (termination, determinism, when concurrent ForEach iterations are equivalent to sequential ones). §§3.2–3.7 restate it in prose.
  • Document model (§2): syntactic form discrimination, the {"@id": …} URI reference form, scalar coercion, the closed JSON-LD reserved-key list, the base-IRI rule.
  • Flat sequences (XDM): Unit is the empty sequence; the sequence form, ForEach and Iterate yield the concatenation of their values; a Result is one item.
  • Effects (§3.6): iterations of a ForEach are unordered. Two iterations updating the same URI are an error, the algebra's XTDE1490, compared on the URI the write reports and counted through nested ForEaches. A read of what another iteration writes is implementation-dependent (XTRE1500).
  • SPARQL 1.1 / XPath conformance, no divergences: Str, Concat, Replace (with flags and XPath $N), EncodeForURI and STRUUID follow their W3C signatures exactly.
  • XSLT focus: ForEach establishes (item, position, size); Position and Last per fn:position()/fn:last().
  • Iterate (xsl:iterate): params, quoted operation/next-iteration, structured break, normative 1000-iteration cap.
  • Filter: by position from a sequence or result ($seq[2]), or by name from a row ($row?url).
  • SPARQL operations (§4.3): SELECT/CONSTRUCT/DESCRIBE take exactly one of endpoint and graph; the graph case is pure. SPARQLString takes endpoint, question, projection (the as of xsl:evaluate) and context (explorations shown to the model), with bounded repair.
  • Linked Data operations (§4.4): RDF-only and symmetric, with transparent conneg. A write returns (status, url), where url is the response's Location, otherwise the effective request URI. A write answered outside 2xx is an error. If-Match from a HEAD is transport, and the spec says what it does and does not protect against.
  • Schema operations (§4.6): an optional bindings scope (?subject → VALUES); default and named graphs are read.
  • §5: what each serialization still owes the spec. Appendix A: the extension-family table (XML waldh: ↔ JSON ldh-; JSON keeps the prefix because its names are also MCP tool names) and ldh-AddConstruct.
  • Execute is removed; the document envelope is removed.

Implementation

  • The interpreter follows §3.8: flat concatenation; scopes per sequence and per iteration; no mutable default arguments; the URI reference form.
  • ForEach gate for the same-target rule (SameTargetError); Filter by name; SELECT/CONSTRUCT/DESCRIBE over a graph via rdflib.
  • Writes: If-Match from HEAD; Location-aware url; WriteRefusedError (a ValueError) outside 2xx.
  • SPARQLString ported from REST-VKG: projection and parse repair, context excerpts, an ASK check (for SELECT queries), AGENTS.md.
  • Extraction queries are REST-VKG's, scoped by bindings, with a fallback for endpoints that refuse GRAPH.
  • Iterate, Position, Last and ldh-AddConstruct added. The ldh-* string checks accept simple literals (RDF 1.1).
  • Code health: an exception taxonomy (WebAlgebraError subclasses that also inherit the §3.7 built-ins) and an HTTP-client mixin.
  • prompts/system.md and the README follow the catalog.
  • Also included, though not used by the algebra: groundwork for an HTTP service, which sits in the same files. That is OperationKind (read/write/destructive) on operations, and an optional write recorder and CA bundle on the clients.

⚠️ Breaking changes

Deliberate and conformance-driven; hence 2.0.0.

  • Sequences are flat: a JSON array, ForEach and Iterate concatenate their values. A ForEach/Iterate with an array body yields all its values, not the last one.
  • ForEach: two iterations writing the same URI raise ValueError.
  • Writes: a non-2xx answer raises ValueError (WriteRefusedError), not urllib.error.HTTPError. url is the Location when the server sends one. Writes to existing resources are now conditional (If-Match).
  • SPARQLString: endpoint is required. The result is a simple literal; it is checked and may be regenerated.
  • Execute is removed (unknown operation → ValueError).
  • Str: language tags are no longer preserved; Str(BNode) raises TypeError; the result is a simple literal.
  • EncodeForURI, STRUUID: return simple literals.
  • Concat: the result kind follows SPARQL CONCAT rules.
  • Replace: XPath replacement syntax ($1; bare $/\ are errors). Zero-length-matching patterns and invalid flags raise ValueError; language-tagged pattern/replacement raise TypeError.
  • URI(BNode), Substitute with a BNode binding: TypeError.
  • Value: focus-item lookup is closed to Binding + mapping. Current/Value outside ForEach: ValueError.
  • Scalar coercion: booleans → xsd:boolean; null → TypeError.
  • {"@id": "..."}: an object whose only member is @id is a URI.
  • ForEach context: operations receive a Focus, not the bare item.
  • Filter: a string expression is a name lookup on a row; other non-integer expressions raise TypeError.
  • HTTP/SPARQL responses: a non-RDF or unparseable response raises ValueError.

Test plan

  • pytest tests/unit: 443 passed, 8 skipped (live-service or LLM-dependent). Network/sparql/ldh markers are deselected. The tests are derived from the spec alone; HTTP is stubbed at the urllib boundary (tests/http_stub.py).
  • ruff check src tests: clean.
  • New suites: document model and scoping, write contract (Location, If-Match, refused writes), same-target rule, graph-queried SPARQL, extraction bindings, Filter by name, Position/Last, Iterate, Concat, ldh-AddConstruct.
  • Open spec questions are tracked in tests/SPEC_GAPS.md. One example: what a relative IRI does inside a graph data form.

🤖 Generated with Claude Code

namedgraph and others added 15 commits July 12, 2026 17:29
…plan

Evaluates the composed-operations premise, the formal semantics, the JSON
DSL, the Python codebase and the rdflib data model; diffs the operation
set and execution semantics against the Java/XML sibling (REST-VKG) and
lays out a three-tier plan: spec unification, two-way operation parity,
and Python code health.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d tests

Spec (formal-semantics.md rewrite):
- Add evaluation semantics: eager depth-first evaluation, quoted operands
  (ForEach.operation, Execute.operation), sequence forms with fresh variable
  scopes, context rules, effect classes and ordering guarantees, and a
  normative error taxonomy.
- Add the document model: optional envelope ({"@web-algebra": "1", "program":
  [...]}), the URI reference form ({"@id": ...}), normative JSON-LD
  discrimination keys, scalar coercion table, base-IRI rule.
- Rebuild the catalog with per-operation JSON argument tables; add missing
  Concat and ExtractOntology entries; fix Unit vs bottom on Variable; pin
  every decision tracked in tests/SPEC_GAPS.md (Str datatype, EncodeForURI
  charset, Replace dialect, STRUUID format, Substitute and Values rules,
  Merge set-union, Bindings order, Filter positional signature, ForEach
  output shape, Value/Current context rules, Execute narrative, Extract*
  endpoint role). ldh-* moves to an informative appendix.

Implementation alignment:
- Fix mutable default arguments across the interpreter and all operations
  (variable_stack/context leak between documents in one process, e.g. the
  MCP server).
- Sequence forms and ForEach iterations now push/pop a variable scope
  (replacing copy() semantics that leaked writes when an outer scope existed).
- Implement the URI reference form and Operation.unwrap_document; unwrap the
  envelope in main.py.
- Execute threads the variable environment (was silently dropped); Current
  errors without an iteration context; Value supports mapping context items;
  URI and Substitute reject BNodes (Substitute previously emitted invalid
  "_: N" syntax); Filter raises TypeError for non-integer expressions;
  boolean scalars coerce to xsd:boolean (bool-before-int); null forms raise
  TypeError; remove dead _serialize_for_json_context.

Tests: 42 UNCLEAR(spec) skips un-skipped and authored from the new spec;
new test_document.py covers the envelope, URI reference form, coercions and
scoping. 221 passed, 8 skipped (was 145/50).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The spec no longer documents divergences from W3C functions; operations
named after SPARQL/XPath functions follow those definitions by normative
reference, signatures included. Simple literals are materialized as plain
rdflib literals (no datatype) exactly as rdflib's own SPARQL engine does.

- Str: per `simple literal STR(literal ltrl)` / `STR(IRI rsrc)` — returns
  the lexical form / codepoint representation as a simple literal; language
  tags are no longer carried over; BNode is a type error.
- Replace: per REPLACE()/fn:replace — adds the optional flags argument
  (s m i x q), XPath replacement syntax ($N group references, \$ and \\
  escapes), err:FORX000* conditions as ValueError (invalid flags/pattern/
  replacement, zero-length-matching pattern), result kind follows the first
  argument, and pattern/replacement/flags must be simple literals.
- Concat: per CONCAT() result-kind rules — all xsd:string inputs yield
  xsd:string, a shared language tag is carried, anything else yields a
  simple literal.
- EncodeForURI, STRUUID: return simple literals per their signatures.
- SELECT/CONSTRUCT/DESCRIBE accept simple-literal queries via a new
  Operation.is_string_literal predicate (RDF 1.1 equivalence), so Str
  output composes into query arguments.

tests/SPEC_GAPS.md records the one honest implementation gap: XPath-only
regex constructs unsupported by Python re surface as ValueError. New
test_concat.py; Str/Replace suites rewritten against the W3C behavior.
238 passed, 8 skipped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 1.5.0 bump (510f4dd) updated pyproject.toml without regenerating the
lockfile's own package entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the optional {"@web-algebra": ..., "program": [...]} prolog: a
document is simply a single form or a program array. Removes
Operation.unwrap_document, the main.py unwrap step, the spec section and
error-table row, the system-prompt mention, and the envelope tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
formal-semantics.md §3.8 now contains the formal definition the document's
title promised: abstract syntax, semantic domains (Env, Focus, World), the
big-step judgment ρ, φ ⊢ ⟨e, σ⟩ ⇓ ⟨v, ρ′, σ′⟩ with the full rule set
(scalar/URI-ref/object/data/sequence/call plus the state-accessing forms
Variable, Value, Current, Position, Last, ForEach, Execute), the operator
interpretation δ_op with effect classes, and metatheory notes (termination
by structural induction, determinism modulo declared non-determinism, why
concurrent ForEach iterations are observationally sound). The prose of
§§3.2–3.7 is now the restatement; §3.8 wins on conflict.

The context (§3.5) generalizes to the focus — the triple (item, position,
size), exactly XSLT's dynamic context. ForEach establishes it per
iteration; new operations Position and Last expose it per XPath
fn:position()/fn:last() as xsd:integer literals; Current and unprefixed
Value lookups read the focus item. Outside any focus all three raise
ValueError.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the getattr fallback: "attribute of that name" was host-language
reflection, not a defined path step — it had no possible Java translation
and no focus-item shape that ForEach can produce needs it. The item shapes
are now closed (Binding → bound term, mapping → member value; anything
else raises ValueError), which makes Value fully well-defined and
REST-VKG-portable.

Also note in the catalog that Value has xsl:sequence semantics (value
as-is, no string conversion) — xsl:value-of is expressible as
Str(Value(...)).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…sistence, Value-domain precision

Linked Data operations (§4.4) are now specified as RDF-specific and
symmetric: they read and write RDF graphs; content negotiation is handled
transparently by the implementation; a non-RDF response — unsupported
media type, missing Content-Type, or a body that does not parse as the
negotiated format — raises ValueError. §4.3 states the counterpart for
the SPARQL operations. client.py conforms: missing Content-Type no longer
crashes with AttributeError, and RDF-labelled bodies that fail to parse
raise ValueError instead of leaking rdflib parser errors; new stub-opener
tests pin the contract offline.

Also: Result values declared materialized and re-iterable (§1.1); the
Value domain's former JSON summand split into Object (generic-object
results over Values) and Data (RDF data forms: Term holes + raw JSON)
with their conversion boundaries stated (§3.1); scalar coercion row
reworded to "number parsed as floating-point"; Position type/operation
name overload noted in §1.1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The one operation the Java/XML Web Algebra had that the JSON side lacked,
inspired by XSLT 3.0's xsl:iterate. JSON serialization:

  {"@op": "Iterate", "args": {
    "params": {name: form, ...},           eager, bound as loop variables
    "operation": <quoted form or array>,   the loop body
    "next-iteration": {name: form, ...},   quoted; evaluated after each body
                                           in the iteration's environment
                                           (loop params + body bindings) and
                                           rebinds the parameters
    "break": {"name": ..., "equals"|"not-equals": ...}}}

Without next-iteration exactly one iteration runs; break compares the
named loop variable's lexical form after rebinding (missing variable
compares as "", REST-VKG parity); the result is the sequence of iteration
values (Units dropped); Iterate establishes no focus, so an enclosing
ForEach focus stays visible — matching the Java implementation, which
preserves the current binding across withVariable.

The iteration count is capped at a normative 1000 (as in REST-VKG), which
keeps the algebra terminating: §3.8 gains the (ITERATE) rule with a loop
helper measured by CAP − k, and the metatheory note is updated. §5 records
the Java implementation's fused merged-graph return and totalLimit as its
specialization (Merge(Iterate(...)) expresses the fusion here).

Spec §3.3/§3.6/§4.1/§3.8/§5, implementation, spec-derived tests (11 cases
incl. the cap and cursor threading), README and system prompt updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Major release: the formal evaluation semantics, SPARQL/XPath conformance
for the string operations, the XSLT focus (Position/Last), Iterate, the
@id URI reference form, and the interpreter fixes in this branch are
deliberately breaking (see PR #25 for the full list).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tracts

Implements the remaining tractable Tier 3 items from architecture-evaluation.md
(3.1 mutable defaults and the _serialize_for_json_context deletion already
landed in the semantics work).

Exception taxonomy (3.3): new exceptions.py with WebAlgebraError and the
interpreter-level subclasses UnknownOperationError, InvalidFormError,
VariableNotFoundError, NoFocusError. Each ALSO inherits the built-in the spec's
error table (§3.7) mandates, so the normative contract and every existing
pytest.raises(TypeError|ValueError) assertion still hold, while callers gain
`except WebAlgebraError` to tell an ill-formed document from an unrelated bug.
Applied at the dispatch/evaluation/variable/focus sites (operation.py, value,
current, position, last); per-operation argument-type validation stays plain
TypeError per §3.7. HTTP transport failures are deliberately NOT wrapped —
§3.7 pins them unwrapped. §3.7 gains a sentence documenting the hierarchy.

HTTP-client dedup (3.4): new ClientOperation mixin builds the cert-configured
client once; GET/POST/PUT/PATCH (LinkedDataClient), SELECT/CONSTRUCT/DESCRIBE
(SPARQLClient) and ldh-AddFile (FileClient) drop their duplicated
model_post_init. SPARQLString keeps its own (OpenAI, different constructor).

Honest contracts (3.5): removed nine dead mcp_run methods from operations that
never claimed MCPTool (Bindings, Current, Execute, Filter, ForEach, Str, URI,
Value, Variable) — the server only dispatches mcp_run on MCPTool instances, so
those were unreachable. Establishes the invariant "mcp_run exists iff MCPTool
is claimed." (If Str/URI should be MCP tools, that is a deliberate feature add:
claim MCPTool + implement mcp_run.) ForEach/Iterate.execute now state plainly
that they are interpreter-level special forms with no pure form (§4.1).

Deferred to their own PR (large refactors coupled to the future parallel-ForEach
work, not rushed onto the 2.0 branch): extracting the interpreter into an
immutable ExecutionContext object (3.2), and generating inputSchema from
pydantic argument models (3.5 tail).

265 passed (5 new exception tests), ruff clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…BE, result-document rule; drop Execute

The evaluation model is stated as XSLT/XPath applied to RDF, with SPARQL
contributing the data model and the term-level function library.

- Sequences are flat (XDM): Unit is the empty sequence, and the sequence
  form, ForEach and Iterate yield the concatenation of their values.
- Filter: (Sequence α + Result + Binding) × (Position + Literal) → α; a
  Literal on a Binding is the lookup operator ($row?url).
- SELECT/CONSTRUCT/DESCRIBE: (URI + Graph) × Literal, exactly one of
  endpoint/graph; the graph case is pure.
- §3.6: updates inside ForEach are unordered; two updates to one URI within
  one ForEach are an error, as two result documents to one href are.
- Execute is removed; it was the MCP-era entry point and no document uses it.
- Appendix A pins the ldh-* returns; Position and Values.vars typed in §1.1
  terms; §5 lists what each serialization still owes the spec.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A PUT, POST or PATCH answered outside 2xx raises, as an xsl:result-document
that cannot be written does, so nothing after it runs on a document that did
not change. Sending the resource's entity tag as If-Match, where a server
requires a write to name the state it was written against, is transport, like
content negotiation, and stays out of the signatures.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e is per iteration

Appendix A: the LDH operations are an extension family in the XML serialization,
elements in https://w3id.org/atomgraph/web-algebra/linkeddatahub with the prefix
waldh, their arguments in the algebra's namespace; JSON, which has no namespaces,
keeps the ldh- names. §3.6/§3.7: two iterations of one ForEach updating the same
URI are the error, and within one iteration the sequence form orders the writes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ization owes

Spec:
- §3.6: the same-target rule compares the URI a write reports, counts nested
  ForEach writes for every enclosing iteration, and says why it is stated on
  targets; a read of what another iteration writes is implementation-
  dependent (XTRE1500), and the metatheory's concurrency claim is narrowed
  to match.
- §4.3: SELECT/CONSTRUCT/DESCRIBE take `endpoint` or `graph`, no longer
  slated; SPARQLString takes `endpoint`, `projection` and `context` - the
  `as` of xsl:evaluate - with bounded repair and an empty context an error.
- §4.4: `url` is the response's Location, resolved against the effective
  request URI; If-Match from HEAD satisfies a 428 but is not lost-update
  protection.
- §4.6: the schema operations take an optional `bindings` scope.
- §5 rewritten: it described REST-VKG as it was; it now lists what the XML
  serialization still owes. Appendix A: the family table (waldh <-> ldh-,
  which JSON keeps since its names are also MCP tool names) and
  ldh-AddConstruct.

Implementation:
- Flat sequences; ForEach and Iterate concatenate their iteration values.
- ForEach refuses a URI two iterations write (SameTargetError).
- Filter looks a variable up on a row by name.
- SELECT/CONSTRUCT/DESCRIBE run over a graph in hand.
- Writes send If-Match from a HEAD, report Location, and raise
  WriteRefusedError (a ValueError) outside 2xx.
- SPARQLString ported from REST-VKG: projection, context, parse/projection
  repair, the ASK check for SELECT, AGENTS.md.
- Extract* run REST-VKG's queries, scoped by `bindings`, without GRAPH where
  an endpoint refuses it.
- ldh-AddConstruct; the ldh-* string checks accept simple literals.
- Execute removed; prompts/system.md and the README follow.

Tests derived from the spec alone: write contract, same-target rule, graph
queries, extraction scope, Filter by name, flat sequences.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@namedgraph namedgraph changed the title Formal evaluation semantics, SPARQL/XPath conformance, XSLT focus, Iterate Formal evaluation semantics aligned with REST-VKG: flat sequences, XSLT focus, SPARQL/XPath conformance Oct 3, 2026
@namedgraph
namedgraph merged commit 0dcb53d into main Oct 3, 2026
9 checks passed
@namedgraph
namedgraph deleted the feat/formal-semantics-evaluation branch October 3, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant