Define parameter values and decision outcome validation for the DSL #117

Current state

Literal parameter values are passed to yaml.safe_load, while DSD syntax is split first using commas for chains, + for parameters, : for key/value separation, and // for comments. That splitting does not recognize quoted strings or YAML collections. Values containing spaces are also rejected.

For example, a quoted value containing a comma can be split into another chain element; a URL containing :// conflicts with comment/key-value handling. YAML decoding therefore does not imply support for arbitrary YAML values. ROS parameter references (%name) and subtree references (*name) add separate value sources. #119 types parameter values as object; #120 improves structural validation and source diagnostics but leaves these delimiter rules unchanged.

Decisions return arbitrary strings. Binding verifies that implementation classes exist, but classes do not declare parameter schemas or possible outcomes. An unmapped outcome is discovered at runtime unless an ELSE branch handles it.

Problem and relevance

The accepted parameter language is difficult to predict: a value valid in YAML can be rejected or split incorrectly before YAML sees it. Quoting does not protect delimiters. Authors need a precise grammar and useful errors rather than trial and error.

Parameter typos/types and missing decision branches can survive loading and fail only when the relevant path executes. Earlier validation would help behavior authors and tooling, but mandatory schemas or exhaustive outcomes could break dynamic decisions and existing behaviors.

Options for parameter syntax

1. Specify a restricted scalar grammar. Document supported scalar values and delimiters, and reject collections or unsupported quoting with source diagnostics.
   • Benefits: small implementation and clear limits; predictable compatibility for simple existing files.
   • Caveats: restricts configuration expressiveness; ROS/subtree values need the same validation rules or an explicit exception.
2. Use a lexer that understands quoting and nesting. Recognize comments and DSD delimiters only outside quoted values and collections, then decode values with YAML. Specify escaping, whitespace, and supported YAML constructs.
   • Benefits: quoted strings and structured values work consistently; existing simple syntax can be preserved.
   • Caveats: more parser complexity; ambiguous legacy input and YAML implicit conversions need migration tests and documented rules.

Options for load-time validation

1. Keep open parameters and string outcomes. Improve runtime diagnostics only.
   • Benefits: maximum flexibility and no metadata burden.
   • Caveats: misspelled parameters and missing branches remain runtime errors.
2. Add optional class metadata/validators. Validate required/unknown parameters and decoded value types after reference resolution. Allow a class to declare possible string outcomes; check branch coverage, accounting for ELSE.
   • Benefits: opt-in early errors without requiring behavior migration; dynamic decisions can omit outcome declarations.
   • Caveats: undeclared classes remain unchecked; ownership of schemas, default values, and diagnostics must be defined.
3. Require schemas and enum-based outcomes. Make declarations part of every element's API.
   • Benefits: stronger guarantees and better editor/tooling support.
   • Caveats: broad migration; an enum in Python still needs runtime metadata to validate a loaded behavior, and computed/open-ended outcomes need an escape hatch.

Recommendation

Choose syntax option 2 with an explicit grammar and compatibility tests for current behavior files. Start with quoted scalar values and clearly specified collections; reject unsupported YAML constructs rather than implying full YAML support. Add validation option 2 independently, after resolving ROS/subtree references, and keep string outcomes as the public contract initially.

Test delimiters inside strings/collections, escapes, whitespace, YAML conversions, reference values, and ELSE coverage. Mandatory schemas/enums should follow demonstrated consumer needs. Related: #64 concerns grouping multiple outcome labels on one branch, which is a separate syntax extension; #23 concerns indentation.

I am working on behalf of @jaagut.