Skip to content

FEAT: Rename and delete Operations in the GUI - #3275

Open
Adrian Gavrila (adrian-gavrila) wants to merge 2 commits into
microsoft:mainfrom
adrian-gavrila:adrian-gavrila-operation-rename-delete
Open

Adrian Gavrila (adrian-gavrila) wants to merge 2 commits into
microsoft:mainfrom
adrian-gavrila:adrian-gavrila-operation-rename-delete

Conversation

@adrian-gavrila

Copy link
Copy Markdown
Contributor

Description

Follow-up to #3066, which listed renaming Operations as a follow-up. Without rename or delete, a typo in an Operation name is permanent. The upgrade also registers every Operation name already used in history, so misspelled or unused names became Operations nobody could remove.

What changed

  • Rename. PATCH /api/operations/{id} now takes name as well as description, and leaves omitted fields unchanged. Names stay unique, ignoring case and surrounding spaces: a taken name returns 409 with the existing Operation, as create does, and fixing only the capitalization is allowed. The Operation page's Edit dialog covers both fields and sends only the ones you changed. The last save wins, as with descriptions in FEAT: Findings dashboard with persistent Operations in the GUI #3066.
  • Delete. DELETE /api/operations/{id} removes an Operation only when no findings, attacks or scanner runs reference it by ID, and returns 204, 404, or 409 with the counts. Activity saved with only a legacy name doesn't count, so names registered by the upgrade can be cleaned up; that activity keeps its name. Like create, it needs no admin rights.
  • Integrity on SQLite. PyRIT's SQLite doesn't enforce foreign keys, so the delete is one conditional DELETE ... WHERE NOT EXISTS (...), and attack, scanner-run and finding inserts check their Operation again after inserting, while they hold the write lock. A delete that races a new attack either sees it and returns 409, or the save fails as not found; nothing is orphaned. On SQL Server the foreign keys catch these races, and those failures map to the same 404 and 409.
  • GUI. GET /api/operations/{id} adds usage counts. Delete sits next to Edit. When the Operation is in use it's greyed out and its tooltip says why, for example "Has 3 findings and 12 linked attacks or runs." Otherwise it asks for confirmation, deletes, and returns to the Operations list. The labels bar re-checks its selected Operation after a rename or delete.
  • Memory API. update_operation_async gains an optional name, and there are new get_operation_usage_async and delete_operation_async methods; the latter raises OperationInUseError. No schema change or migration.

What a rename doesn't update

Findings, evidence and activity linked by ID show the new name. Anything that stores the old name doesn't follow:

  • Analytics groups and filters Operations by name (FEAT: Attack analytics REST #3149, FEAT: Add saved attack results analytics GUI #3168), so saved views and #analytics= links that selected the old name stop matching its activity.
  • A deprecated name in configuration, GLOBAL_MEMORY_LABELS or --memory-labels stops resolving. It's stored as a plain name, and the labels bar shows it as not registered.
  • History saved before Operation IDs keeps the old name.

Follow-ups

  • Keying the analytics Operation dimension by ID, so a rename doesn't narrow saved views.
  • Moving activity between Operations, and run counts on the Operations list.

Tests and Documentation

New tests cover rename (capitalization-only, taken, invalid and missing names), delete (each kind of reference, legacy names, 404 and 409), the insert/delete and rename/delete races, the REST layer, both dialogs, and a seeded Playwright test of the whole flow. SQL Server's foreign-key failures are tested with a simulated error. Results on de63b7594, based on 893404f29:

  • pytest -n 4 --dist=loadfile --cov=pyrit --cov-fail-under=78 tests/unit: 29,025 passed, 57 skipped, 96.03% coverage.
  • diff-cover coverage.xml --compare-branch=893404f29 --fail-under=90: 97% of 124 changed lines.
  • pre-commit run --all-files and ty check pyrit tests/unit: passed.
  • npm run test:coverage: 3,240 passed.
  • npm run lint, npx tsc --noEmit, npm run build and npx playwright test --list: passed.
  • npx playwright test --project=seeded: 98 passed. --project=mock: 188 passed; accessibility.spec.ts:406 ("render without layout shifts") fails the same way on main. npm run test:e2e:analytics: 9 passed.
  • Not run: integration tests and anything against a live SQL Server. SQL Server is compile-checked only.

Docs: doc/gui/0_gui.md renames its section to "Operations" and adds two short paragraphs on renaming and deleting. No notebooks changed, so JupyText wasn't needed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment on lines +765 to +768
for table in (FindingEntry, AttackResultEntry, ScenarioResultEntry)
]
try:
async with await self.get_session_async() as session, session.begin():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Keep in-flight SDK attacks from being treated as empty

Could we protect Operations that an SDK attack is already using? AttackStrategy.execute_with_context_async resolves the Operation before execution, but saves its AttackResult only after the target finishes. These checks therefore consider the Operation empty throughout that first attack.

I reproduced this with PromptSendingAttack.execute_async(objective="Say hello.", operation_id=operation.id) and a delayed offline target: usage was all zero, deletion succeeded, and releasing the target made execution raise LookupError: Operation ... not found. No attack result was saved, although both messages remained in memory. A GUI user can therefore delete an apparently unused Operation and break another user's running execution after the target work has completed.

Please retain a durable reference for the execution lifetime, or otherwise prevent deletion until it finishes, and cover this case with an in-flight SDK test. The post-insert checks prevent a dangling Operation ID, but do not preserve the completed result.

Comment on lines +5572 to +5573
if new_attack is not None:
self._recheck_activity_operations(session=session, entries=[entry])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Translate a missing Operation FK failure on activity inserts

Could we also handle the foreign-key failure when deletion wins just after the first Operation check? On SQL Server, the pending attack INSERT can raise IntegrityError during flush or an earlier autoflush, before the new existence check runs. The surrounding handler only checks for a duplicate conversation-save retry and otherwise rethrows, so POST /api/attacks returns 500 instead of the intended missing-Operation 404. The scenario insert path has the same gap.

I checked the REST path with an isolated SQLite database and skipped the first check to model that race: with foreign keys off it returns 404; with foreign keys enforced it raises FOREIGN KEY constraint failed and returns 500. This is a constraint-enabled control, not a live SQL Server run.

Please check for the missing Operation after rolling back these insert failures and translate that case to LookupError, as add_finding_async already does, while preserving unrelated integrity errors. A constraint-enforced insert/delete test would cover the backend difference that the current SQLite tests miss.

@romanlutz

Copy link
Copy Markdown
Contributor

Please attach screenshots or a video to the description.

<DialogBody>
<DialogTitle>Delete operation?</DialogTitle>
<DialogContent className={styles.form}>
<Text>Permanently delete “<span translate="no">{operation.name}</span>”? This cannot be undone.</Text>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we explain what will be deleted and what will remain in this confirmation? “Permanently delete” does not tell users whether this removes just the operation entry or also its findings and execution history. The tooltip helps, but the confirmation should make the consequences clear without requiring users to read the docs.

For example:

This removes the operation entry, including its name and description. It does not delete historical activity. Older history saved under this name will remain unchanged.

Operations with linked findings, attacks, or scanner runs cannot be deleted. This cannot be undone.

That would also explain why an operation can count as “empty” even when older name-only history exists.

@ManoharPaturi

Copy link
Copy Markdown
Contributor

The backend reference validation in OperationService.delete_operation_async is well constructed. Checking unreferenced existence across FindingEntry, AttackResultEntry, and ScenarioResultEntry directly in the delete statement guarantees that SQLite handles referential integrity properly even without foreign key enforcement, while returning HTTP 409 with full usage counts gives clear actionable feedback when an operation is still linked. Echoing Roman's comment above, having a quick UI screen capture or video in the description will make reviewing the frontend delete confirmation dialog and rename flow straightforward.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants