Skip to content

fix(agentic-ai): stale activity references and a dependency-level fix - #84

Open
dzeusking-dev wants to merge 3 commits into
devsecopsmaturitymodel:mainfrom
dzeusking-dev:review/agentic-ai-dimension-20260903
Open

fix(agentic-ai): stale activity references and a dependency-level fix#84
dzeusking-dev wants to merge 3 commits into
devsecopsmaturitymodel:mainfrom
dzeusking-dev:review/agentic-ai-dimension-20260903

Conversation

@dzeusking-dev

@dzeusking-dev dzeusking-dev commented Sep 3, 2026

Copy link
Copy Markdown

Context

This is a focused review of the Agentic AI dimension, concentrating on the
areas highlighted for it: activity titles, maturity levels, dependency structure
and the human-verification ladder. The dimension introduced in #80 (and the
activities added afterwards) is in good shape; this PR does not rewrite it. It
corrects two concrete internal-consistency defects found during a focused
review of the Agentic AI dimension and its dependencies, while leaving the
substantive security guidance untouched. It does not claim to close #80.

Review method

  • Built a complete inventory of the 40 Agentic AI activities directly from the
    source YAML (title, uuid, level, subdimension, tags, dependencies,
    implementation refs, SAMM/ISO mappings), rather than trusting the
    documentation.
  • Ran the repository generator and a focused integrity audit of all 40 Agentic
    AI activities and their dependencies: uuid and activity-name consistency,
    dependency resolution and level progression, cycle detection, implementation
    $ref resolution, required fields / numeric ranges / tags, and dependsOn
    inline comments vs. canonical activity titles.
  • Validated the four Agentic AI files against
    schemas/dsomm-schema-agentic-ai.json.
  • Compared the ARCHITECTURE.md inventory against the canonical YAML.
  • Ran the documented generator before and after the change and diffed the
    generated model.

Changes

Two themes, both internal-consistency corrections. No activity content, wording
or UUIDs are otherwise changed.

1. Stale activity references synchronized to canonical titles.
Several references named activities by titles that do not exist in the model.
The generator already normalizes dependsOn comments from the canonical title,
so these source comments had drifted from the generated output.

  • Language and framework specific security rules and Dynamic load of security rules (Guidance) referenced uuid 51ebc877 as "Basic secure coding rules
    for AI assistants"
    → corrected to its canonical title "Static load of
    security rules"
    .
  • Dynamic load of security rules (Guidance) and Security test generation with AI (Verification) referenced uuid 1caad69c as "Security requirements for
    AI-assisted development"
    → corrected to its canonical title "Threat
    modeling rule"
    .
  • The Threat modeling of AI components description contained the same stale
    cross-reference in prose; reworded to reference the Threat modeling rule
    activity by its canonical title.

2. Dependency-level progression fixed for one activity.
Static and dynamic analysis of AI generated code (Verification) was level 2
but depends on Static analysis for important server side components (Test and
Verification, level 3), so a prerequisite sat one level above the activity that
requires it. Raised the activity to level 3. This also aligns it with its
sibling No verification bypass for AI generated code (level 3), which shares
the same level-3 prerequisite. ARCHITECTURE.md updated to match.

Files: ARCHITECTURE.md,
src/assets/YAML/default/AgenticAI/Guidance.yaml,
src/assets/YAML/default/AgenticAI/Verification.yaml.

Compatibility

No activity UUIDs change and no activities are added, removed, moved or renamed,
so assessment continuity is fully preserved. Only dependsOn comment labels,
one prose cross-reference, and one activity's level (2 → 3) change. Generated
artifacts (generated/model.yaml, generated/dependency-tree.md) are
.gitignored and regenerated/committed by CI, so they are intentionally not
part of this PR.

Validation

Observed results of what was actually run locally:

  • Integrity audit (post-change): no stale Agentic AI dependsOn comments
    remain, no dependency-level violations remain, no dependency cycles involving
    the reviewed Agentic AI activities, all Agentic AI dependencies resolve, all
    implementation $refs used by the Agentic AI dimension resolve, all required
    fields / ranges / tags present, ARCHITECTURE.md matches the canonical YAML
    (40/40 activities).
  • JSON schema: all four Agentic AI files validate against
    dsomm-schema-agentic-ai.json (0 errors).
  • Generator (generateDimensions.bash, dockerized): runs without errors
    before and after; the only meaningful diff in generated/model.yaml is the
    intended level: 2 → 3 and the reworded cross-reference, plus the
    CI-injected version/publisher metadata.
  • Local application smoke test: started the wurstbrot/dsomm app on the
    generated model, received HTTP 200, confirmed the Agentic AI dimension loads
    and the activity is served at level 3, with no error entries in the container
    log.
  • URL testing (--test-urls): not run, because no implementation references
    or URLs are added or changed by this PR.
  • CI checks: the repository's workflow triggers on push to main, schedule
    and manual dispatch, not on pull_request, so no PR-triggered status checks
    run on this branch.

Source basis

The changes are structural/consistency corrections derived from the model's own
rules (canonical titles, the prerequisite semantics of dependsOn described in
ARCHITECTURE.md) rather than from external standards, so no new references were
added.

Scope

Intentionally not changed: activity descriptions, risks, measures, assessments,
SAMM/ISO mappings, implementation references, subdimension taxonomy, and every
activity in the other dimensions. Potential consistency observations outside the
Agentic AI dimension were intentionally left untouched to keep this pull request
focused.

@wurstbrot

wurstbrot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Hi @dzeusking-dev ,

thank you for the PR. I agree with everything except the level of static and dynamic tests.
The reason to have it on level 2 and not 3 is because it is more important with AI development.
Specally SAST with live verification of an AI is power full.
The argument to move it to level 3 just because the dependency in DevOps-based development is 2 doesn't fit here.

Kind regards
Timo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants