Repository navigation
Formal evaluation semantics aligned with REST-VKG: flat sequences, XSLT focus, SPARQL/XPath conformance - #25
Merged
Conversation
…plan Evaluates the composed-operations premise, the formal semantics, the JSON DSL, the Python codebase and the rdflib data model; diffs the operation set and execution semantics against the Java/XML sibling (REST-VKG) and lays out a three-tier plan: spec unification, two-way operation parity, and Python code health. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d tests
Spec (formal-semantics.md rewrite):
- Add evaluation semantics: eager depth-first evaluation, quoted operands
(ForEach.operation, Execute.operation), sequence forms with fresh variable
scopes, context rules, effect classes and ordering guarantees, and a
normative error taxonomy.
- Add the document model: optional envelope ({"@web-algebra": "1", "program":
[...]}), the URI reference form ({"@id": ...}), normative JSON-LD
discrimination keys, scalar coercion table, base-IRI rule.
- Rebuild the catalog with per-operation JSON argument tables; add missing
Concat and ExtractOntology entries; fix Unit vs bottom on Variable; pin
every decision tracked in tests/SPEC_GAPS.md (Str datatype, EncodeForURI
charset, Replace dialect, STRUUID format, Substitute and Values rules,
Merge set-union, Bindings order, Filter positional signature, ForEach
output shape, Value/Current context rules, Execute narrative, Extract*
endpoint role). ldh-* moves to an informative appendix.
Implementation alignment:
- Fix mutable default arguments across the interpreter and all operations
(variable_stack/context leak between documents in one process, e.g. the
MCP server).
- Sequence forms and ForEach iterations now push/pop a variable scope
(replacing copy() semantics that leaked writes when an outer scope existed).
- Implement the URI reference form and Operation.unwrap_document; unwrap the
envelope in main.py.
- Execute threads the variable environment (was silently dropped); Current
errors without an iteration context; Value supports mapping context items;
URI and Substitute reject BNodes (Substitute previously emitted invalid
"_: N" syntax); Filter raises TypeError for non-integer expressions;
boolean scalars coerce to xsd:boolean (bool-before-int); null forms raise
TypeError; remove dead _serialize_for_json_context.
Tests: 42 UNCLEAR(spec) skips un-skipped and authored from the new spec;
new test_document.py covers the envelope, URI reference form, coercions and
scoping. 221 passed, 8 skipped (was 145/50).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The spec no longer documents divergences from W3C functions; operations named after SPARQL/XPath functions follow those definitions by normative reference, signatures included. Simple literals are materialized as plain rdflib literals (no datatype) exactly as rdflib's own SPARQL engine does. - Str: per `simple literal STR(literal ltrl)` / `STR(IRI rsrc)` — returns the lexical form / codepoint representation as a simple literal; language tags are no longer carried over; BNode is a type error. - Replace: per REPLACE()/fn:replace — adds the optional flags argument (s m i x q), XPath replacement syntax ($N group references, \$ and \\ escapes), err:FORX000* conditions as ValueError (invalid flags/pattern/ replacement, zero-length-matching pattern), result kind follows the first argument, and pattern/replacement/flags must be simple literals. - Concat: per CONCAT() result-kind rules — all xsd:string inputs yield xsd:string, a shared language tag is carried, anything else yields a simple literal. - EncodeForURI, STRUUID: return simple literals per their signatures. - SELECT/CONSTRUCT/DESCRIBE accept simple-literal queries via a new Operation.is_string_literal predicate (RDF 1.1 equivalence), so Str output composes into query arguments. tests/SPEC_GAPS.md records the one honest implementation gap: XPath-only regex constructs unsupported by Python re surface as ValueError. New test_concat.py; Str/Replace suites rewritten against the W3C behavior. 238 passed, 8 skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 1.5.0 bump (510f4dd) updated pyproject.toml without regenerating the lockfile's own package entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the optional {"@web-algebra": ..., "program": [...]} prolog: a
document is simply a single form or a program array. Removes
Operation.unwrap_document, the main.py unwrap step, the spec section and
error-table row, the system-prompt mention, and the envelope tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
formal-semantics.md §3.8 now contains the formal definition the document's title promised: abstract syntax, semantic domains (Env, Focus, World), the big-step judgment ρ, φ ⊢ ⟨e, σ⟩ ⇓ ⟨v, ρ′, σ′⟩ with the full rule set (scalar/URI-ref/object/data/sequence/call plus the state-accessing forms Variable, Value, Current, Position, Last, ForEach, Execute), the operator interpretation δ_op with effect classes, and metatheory notes (termination by structural induction, determinism modulo declared non-determinism, why concurrent ForEach iterations are observationally sound). The prose of §§3.2–3.7 is now the restatement; §3.8 wins on conflict. The context (§3.5) generalizes to the focus — the triple (item, position, size), exactly XSLT's dynamic context. ForEach establishes it per iteration; new operations Position and Last expose it per XPath fn:position()/fn:last() as xsd:integer literals; Current and unprefixed Value lookups read the focus item. Outside any focus all three raise ValueError. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the getattr fallback: "attribute of that name" was host-language reflection, not a defined path step — it had no possible Java translation and no focus-item shape that ForEach can produce needs it. The item shapes are now closed (Binding → bound term, mapping → member value; anything else raises ValueError), which makes Value fully well-defined and REST-VKG-portable. Also note in the catalog that Value has xsl:sequence semantics (value as-is, no string conversion) — xsl:value-of is expressible as Str(Value(...)). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…sistence, Value-domain precision Linked Data operations (§4.4) are now specified as RDF-specific and symmetric: they read and write RDF graphs; content negotiation is handled transparently by the implementation; a non-RDF response — unsupported media type, missing Content-Type, or a body that does not parse as the negotiated format — raises ValueError. §4.3 states the counterpart for the SPARQL operations. client.py conforms: missing Content-Type no longer crashes with AttributeError, and RDF-labelled bodies that fail to parse raise ValueError instead of leaking rdflib parser errors; new stub-opener tests pin the contract offline. Also: Result values declared materialized and re-iterable (§1.1); the Value domain's former JSON summand split into Object (generic-object results over Values) and Data (RDF data forms: Term holes + raw JSON) with their conversion boundaries stated (§3.1); scalar coercion row reworded to "number parsed as floating-point"; Position type/operation name overload noted in §1.1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The one operation the Java/XML Web Algebra had that the JSON side lacked,
inspired by XSLT 3.0's xsl:iterate. JSON serialization:
{"@op": "Iterate", "args": {
"params": {name: form, ...}, eager, bound as loop variables
"operation": <quoted form or array>, the loop body
"next-iteration": {name: form, ...}, quoted; evaluated after each body
in the iteration's environment
(loop params + body bindings) and
rebinds the parameters
"break": {"name": ..., "equals"|"not-equals": ...}}}
Without next-iteration exactly one iteration runs; break compares the
named loop variable's lexical form after rebinding (missing variable
compares as "", REST-VKG parity); the result is the sequence of iteration
values (Units dropped); Iterate establishes no focus, so an enclosing
ForEach focus stays visible — matching the Java implementation, which
preserves the current binding across withVariable.
The iteration count is capped at a normative 1000 (as in REST-VKG), which
keeps the algebra terminating: §3.8 gains the (ITERATE) rule with a loop
helper measured by CAP − k, and the metatheory note is updated. §5 records
the Java implementation's fused merged-graph return and totalLimit as its
specialization (Merge(Iterate(...)) expresses the fusion here).
Spec §3.3/§3.6/§4.1/§3.8/§5, implementation, spec-derived tests (11 cases
incl. the cap and cursor threading), README and system prompt updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Major release: the formal evaluation semantics, SPARQL/XPath conformance for the string operations, the XSLT focus (Position/Last), Iterate, the @id URI reference form, and the interpreter fixes in this branch are deliberately breaking (see PR #25 for the full list). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tracts Implements the remaining tractable Tier 3 items from architecture-evaluation.md (3.1 mutable defaults and the _serialize_for_json_context deletion already landed in the semantics work). Exception taxonomy (3.3): new exceptions.py with WebAlgebraError and the interpreter-level subclasses UnknownOperationError, InvalidFormError, VariableNotFoundError, NoFocusError. Each ALSO inherits the built-in the spec's error table (§3.7) mandates, so the normative contract and every existing pytest.raises(TypeError|ValueError) assertion still hold, while callers gain `except WebAlgebraError` to tell an ill-formed document from an unrelated bug. Applied at the dispatch/evaluation/variable/focus sites (operation.py, value, current, position, last); per-operation argument-type validation stays plain TypeError per §3.7. HTTP transport failures are deliberately NOT wrapped — §3.7 pins them unwrapped. §3.7 gains a sentence documenting the hierarchy. HTTP-client dedup (3.4): new ClientOperation mixin builds the cert-configured client once; GET/POST/PUT/PATCH (LinkedDataClient), SELECT/CONSTRUCT/DESCRIBE (SPARQLClient) and ldh-AddFile (FileClient) drop their duplicated model_post_init. SPARQLString keeps its own (OpenAI, different constructor). Honest contracts (3.5): removed nine dead mcp_run methods from operations that never claimed MCPTool (Bindings, Current, Execute, Filter, ForEach, Str, URI, Value, Variable) — the server only dispatches mcp_run on MCPTool instances, so those were unreachable. Establishes the invariant "mcp_run exists iff MCPTool is claimed." (If Str/URI should be MCP tools, that is a deliberate feature add: claim MCPTool + implement mcp_run.) ForEach/Iterate.execute now state plainly that they are interpreter-level special forms with no pure form (§4.1). Deferred to their own PR (large refactors coupled to the future parallel-ForEach work, not rushed onto the 2.0 branch): extracting the interpreter into an immutable ExecutionContext object (3.2), and generating inputSchema from pydantic argument models (3.5 tail). 265 passed (5 new exception tests), ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…BE, result-document rule; drop Execute The evaluation model is stated as XSLT/XPath applied to RDF, with SPARQL contributing the data model and the term-level function library. - Sequences are flat (XDM): Unit is the empty sequence, and the sequence form, ForEach and Iterate yield the concatenation of their values. - Filter: (Sequence α + Result + Binding) × (Position + Literal) → α; a Literal on a Binding is the lookup operator ($row?url). - SELECT/CONSTRUCT/DESCRIBE: (URI + Graph) × Literal, exactly one of endpoint/graph; the graph case is pure. - §3.6: updates inside ForEach are unordered; two updates to one URI within one ForEach are an error, as two result documents to one href are. - Execute is removed; it was the MCP-era entry point and no document uses it. - Appendix A pins the ldh-* returns; Position and Values.vars typed in §1.1 terms; §5 lists what each serialization still owes the spec. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A PUT, POST or PATCH answered outside 2xx raises, as an xsl:result-document that cannot be written does, so nothing after it runs on a document that did not change. Sending the resource's entity tag as If-Match, where a server requires a write to name the state it was written against, is transport, like content negotiation, and stays out of the signatures. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e is per iteration Appendix A: the LDH operations are an extension family in the XML serialization, elements in https://w3id.org/atomgraph/web-algebra/linkeddatahub with the prefix waldh, their arguments in the algebra's namespace; JSON, which has no namespaces, keeps the ldh- names. §3.6/§3.7: two iterations of one ForEach updating the same URI are the error, and within one iteration the sequence form orders the writes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ization owes Spec: - §3.6: the same-target rule compares the URI a write reports, counts nested ForEach writes for every enclosing iteration, and says why it is stated on targets; a read of what another iteration writes is implementation- dependent (XTRE1500), and the metatheory's concurrency claim is narrowed to match. - §4.3: SELECT/CONSTRUCT/DESCRIBE take `endpoint` or `graph`, no longer slated; SPARQLString takes `endpoint`, `projection` and `context` - the `as` of xsl:evaluate - with bounded repair and an empty context an error. - §4.4: `url` is the response's Location, resolved against the effective request URI; If-Match from HEAD satisfies a 428 but is not lost-update protection. - §4.6: the schema operations take an optional `bindings` scope. - §5 rewritten: it described REST-VKG as it was; it now lists what the XML serialization still owes. Appendix A: the family table (waldh <-> ldh-, which JSON keeps since its names are also MCP tool names) and ldh-AddConstruct. Implementation: - Flat sequences; ForEach and Iterate concatenate their iteration values. - ForEach refuses a URI two iterations write (SameTargetError). - Filter looks a variable up on a row by name. - SELECT/CONSTRUCT/DESCRIBE run over a graph in hand. - Writes send If-Match from a HEAD, report Location, and raise WriteRefusedError (a ValueError) outside 2xx. - SPARQLString ported from REST-VKG: projection, context, parse/projection repair, the ASK check for SELECT, AGENTS.md. - Extract* run REST-VKG's queries, scoped by `bindings`, without GRAPH where an endpoint refuses it. - ldh-AddConstruct; the ldh-* string checks accept simple literals. - Execute removed; prompts/system.md and the README follow. Tests derived from the spec alone: write contract, same-target rule, graph queries, extraction scope, Filter by name, flat sequences. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Turns
formal-semantics.mdfrom a type catalog into an actual semantics, aligns it with the Java/XML implementation in REST-VKG, and brings the Python implementation and test suite up to it. The evaluation model is stated as XSLT/XPath applied to RDF, with SPARQL contributing the data model and the term-level function library.Spec
ρ, φ ⊢ ⟨e, σ⟩ ⇓ ⟨v, ρ′, σ′⟩with the full rule set, operator effect classes, and metatheory (termination, determinism, when concurrentForEachiterations are equivalent to sequential ones). §§3.2–3.7 restate it in prose.{"@id": …}URI reference form, scalar coercion, the closed JSON-LD reserved-key list, the base-IRI rule.ForEachandIterateyield the concatenation of their values; aResultis one item.ForEachare unordered. Two iterations updating the same URI are an error, the algebra's XTDE1490, compared on the URI the write reports and counted through nestedForEaches. A read of what another iteration writes is implementation-dependent (XTRE1500).Str,Concat,Replace(withflagsand XPath$N),EncodeForURIandSTRUUIDfollow their W3C signatures exactly.ForEachestablishes (item, position, size);PositionandLastperfn:position()/fn:last().Iterate(xsl:iterate):params, quotedoperation/next-iteration, structuredbreak, normative 1000-iteration cap.Filter: by position from a sequence or result ($seq[2]), or by name from a row ($row?url).SELECT/CONSTRUCT/DESCRIBEtake exactly one ofendpointandgraph; the graph case is pure.SPARQLStringtakesendpoint,question,projection(theasofxsl:evaluate) andcontext(explorations shown to the model), with bounded repair.(status, url), whereurlis the response'sLocation, otherwise the effective request URI. A write answered outside 2xx is an error.If-Matchfrom aHEADis transport, and the spec says what it does and does not protect against.bindingsscope (?subject→VALUES); default and named graphs are read.waldh:↔ JSONldh-; JSON keeps the prefix because its names are also MCP tool names) andldh-AddConstruct.Executeis removed; the document envelope is removed.Implementation
ForEachgate for the same-target rule (SameTargetError);Filterby name;SELECT/CONSTRUCT/DESCRIBEover a graph via rdflib.If-MatchfromHEAD;Location-awareurl;WriteRefusedError(aValueError) outside 2xx.SPARQLStringported from REST-VKG: projection and parse repair, context excerpts, anASKcheck (forSELECTqueries),AGENTS.md.bindings, with a fallback for endpoints that refuseGRAPH.Iterate,Position,Lastandldh-AddConstructadded. Theldh-*string checks accept simple literals (RDF 1.1).WebAlgebraErrorsubclasses that also inherit the §3.7 built-ins) and an HTTP-client mixin.prompts/system.mdand the README follow the catalog.OperationKind(read/write/destructive) on operations, and an optional write recorder and CA bundle on the clients.Deliberate and conformance-driven; hence 2.0.0.
ForEachandIterateconcatenate their values. AForEach/Iteratewith an array body yields all its values, not the last one.ForEach: two iterations writing the same URI raiseValueError.ValueError(WriteRefusedError), noturllib.error.HTTPError.urlis theLocationwhen the server sends one. Writes to existing resources are now conditional (If-Match).SPARQLString:endpointis required. The result is a simple literal; it is checked and may be regenerated.Executeis removed (unknown operation →ValueError).Str: language tags are no longer preserved;Str(BNode)raisesTypeError; the result is a simple literal.EncodeForURI,STRUUID: return simple literals.Concat: the result kind follows SPARQL CONCAT rules.Replace: XPath replacement syntax ($1; bare$/\are errors). Zero-length-matching patterns and invalid flags raiseValueError; language-taggedpattern/replacementraiseTypeError.URI(BNode),Substitutewith a BNode binding:TypeError.Value: focus-item lookup is closed to Binding + mapping.Current/ValueoutsideForEach:ValueError.xsd:boolean;null→TypeError.{"@id": "..."}: an object whose only member is@idis a URI.ForEachcontext: operations receive aFocus, not the bare item.Filter: a string expression is a name lookup on a row; other non-integer expressions raiseTypeError.ValueError.Test plan
pytest tests/unit: 443 passed, 8 skipped (live-service or LLM-dependent). Network/sparql/ldh markers are deselected. The tests are derived from the spec alone; HTTP is stubbed at the urllib boundary (tests/http_stub.py).ruff check src tests: clean.Location,If-Match, refused writes), same-target rule, graph-queried SPARQL, extractionbindings,Filterby name,Position/Last,Iterate,Concat,ldh-AddConstruct.tests/SPEC_GAPS.md. One example: what a relative IRI does inside agraphdata form.🤖 Generated with Claude Code