[BREAKING] FEAT: Add scenario dataset controls to CLI, REST, and GUI

.github/instructions/scenarios.instructions.md

@@ -169,8 +169,24 @@ Options:
 - Ingredients must remain complete. Use `max_per_dataset="all"` and no finite source
   `max_size`; cap the assembled attack groups with `max_total`, not ingredient rows.
 - Use `with_overrides()` instead of reconstructing a subclass or mutating its defaults.
+- Clients and Python use `ScenarioDatasetRequest` through the common `dataset_overrides`
+  parameter. `dataset_config` still replaces the default; overrides apply on top.
+- Resolve parameter/technique-dependent names in `_resolve_default_dataset_config()`,
+  before overrides, validation, and preparation. Do not rewrite user selections in validators.
+- Declare `DATASET_OPTIONS` (or parameter-aware `get_dataset_options()`): `ANY` only for
+  generic named populations, `SUBSET` for approved alternatives, `FIXED` for ingredients.
+  Disable limits that would truncate ingredients. Compound source edits are ambiguous;
+  preserve child shaping/caps and allow only supported global edits.
+- Estimates expose `dataset_defaults`, `effective_datasets`, and `dataset_options`.
+  These are response projections, not incoming edit state. Never read seeds for projections.
+  The catalog reads options and defaults from registry metadata, never from an estimate.
+- Resume replays the saved launch request; the stored run plan fixes the selected seed groups,
+  so caps and fetch policies cannot change a resumed run. Dataset names are checked by the
+  stored scenario identity. Resume never prepares datasets.
 - Keep custom shaping in `_build_attack_groups()` or `_build_groups_by_dataset_async()`.
   Reads use `_collect_seeds_for_dataset_async()` and must not fetch.
+- Build synthesized prompts from those filtered seeds, not a second unfiltered memory query.
+  Scenario code must not call memory seed-read methods directly; the scenario contract test enforces this.
 - `dataset_names`, `max_dataset_size`, `auto_fetch`, and `per_dataset()` are deprecated.
 
 `max_dataset_size` is an exact alias for `max_total`; it does not disable source caps.

doc/code/scenarios/0_scenarios.ipynb

@@ -66,10 +66,11 @@
     "\n",
     "3. **Default Dataset**: Pass `default_dataset_config=` to `super().__init__()` to specify the datasets your scenario uses out of the box.\n",
     "   - Returns a `DatasetAttackConfiguration` with named sources (e.g., `sources=[DatasetSource(name=\"my_dataset\")]`)\n",
-    "   - Users can override this at runtime via `--dataset-names` in the CLI or by passing a custom `dataset_config` programmatically\n",
+    "   - Users select sources with `--datasets name[:N|:all]`, or pass `dataset_overrides=ScenarioDatasetRequest(...)` in `set_params_from_args`. A custom `dataset_config` replaces the default; overrides apply on top.\n",
     "   - Named sources select at most 5 attack groups per dataset by default; `max_total` caps the union. For any limit, omitted/`None`/empty/`\"default\"` uses the default; `\"all\"` removes that limit.\n",
     "   - The deprecated `max_dataset_size` is an exact alias for `max_total`, not a way to disable source caps. To keep the old total-only selection, use `max_per_dataset=\"all\", max_total=10`; to remove both caps, set both to `\"all\"`. `None` now uses the default, not unlimited selection.\n",
     "   - New runs prepare missing registered datasets once; reads, estimates, and resume never fetch\n",
+    "   - Scenarios declare allowed source selections and limit controls. Estimates return resolved `dataset_defaults` and `effective_datasets` separately from requested edits. Name-only selection preserves existing source caps and fetch policies.\n",
     "\n",
     "4. **Constructor**: Use `@apply_defaults` decorator and call `super().__init__()` with scenario metadata:\n",
     "   - `name`: Descriptive name for your scenario\n",

doc/code/scenarios/0_scenarios.py

@@ -68,10 +68,11 @@
 #
 # 3. **Default Dataset**: Pass `default_dataset_config=` to `super().__init__()` to specify the datasets your scenario uses out of the box.
 #    - Returns a `DatasetAttackConfiguration` with named sources (e.g., `sources=[DatasetSource(name="my_dataset")]`)
-#    - Users can override this at runtime via `--dataset-names` in the CLI or by passing a custom `dataset_config` programmatically
+#    - Users select sources with `--datasets name[:N|:all]`, or pass `dataset_overrides=ScenarioDatasetRequest(...)` in `set_params_from_args`. A custom `dataset_config` replaces the default; overrides apply on top.
 #    - Named sources select at most 5 attack groups per dataset by default; `max_total` caps the union. For any limit, omitted/`None`/empty/`"default"` uses the default; `"all"` removes that limit.
 #    - The deprecated `max_dataset_size` is an exact alias for `max_total`, not a way to disable source caps. To keep the old total-only selection, use `max_per_dataset="all", max_total=10`; to remove both caps, set both to `"all"`. `None` now uses the default, not unlimited selection.
 #    - New runs prepare missing registered datasets once; reads, estimates, and resume never fetch
+#    - Scenarios declare allowed source selections and limit controls. Estimates return resolved `dataset_defaults` and `effective_datasets` separately from requested edits. Name-only selection preserves existing source caps and fetch policies.
 #
 # 4. **Constructor**: Use `@apply_defaults` decorator and call `super().__init__()` with scenario metadata:
 #    - `name`: Descriptive name for your scenario

doc/code/scenarios/1_common_scenario_parameters.ipynb

@@ -15,7 +15,7 @@
     "\n",
     "> **Two selection axes**: *Techniques* select attack techniques (*how* attacks run — e.g., prompt\n",
     "> sending, role play, TAP). *Datasets* select objectives (*what* is tested — e.g., harm categories,\n",
-    "> compliance topics). Use `--dataset-names` on the CLI to filter by content category.\n",
+    "> compliance topics). Use `--datasets` on the CLI to filter by content category.\n",
     "\n",
     "> **Running scenarios from the command line?** See the [Scanner documentation](../../scanner/0_scanner.md).\n",
     "\n",

doc/code/scenarios/1_common_scenario_parameters.py

@@ -19,7 +19,7 @@
 #
 # > **Two selection axes**: *Techniques* select attack techniques (*how* attacks run — e.g., prompt
 # > sending, role play, TAP). *Datasets* select objectives (*what* is tested — e.g., harm categories,
-# > compliance topics). Use `--dataset-names` on the CLI to filter by content category.
+# > compliance topics). Use `--datasets` on the CLI to filter by content category.
 #
 # > **Running scenarios from the command line?** See the [Scanner documentation](../../scanner/0_scanner.md).
 #

doc/code/scenarios/3_adaptive_scenarios.ipynb

@@ -804,7 +804,7 @@
     "# Use specific datasets and limit size\n",
     "pyrit_scan --scenario TextAdaptive --target openai_chat \\\n",
     "    --datasets airt_hate airt_violence \\\n",
-    "    --max-dataset-size 10\n",
+    "    --max-total 10\n",
     "```"
    ]
   }

doc/code/scenarios/3_adaptive_scenarios.py

@@ -301,5 +301,5 @@ def _technique_label(result) -> str:
 # # Use specific datasets and limit size
 # pyrit_scan --scenario TextAdaptive --target openai_chat \
 #     --datasets airt_hate airt_violence \
-#     --max-dataset-size 10
+#     --max-total 10
 # ```

doc/getting_started/install_local_dev.md

@@ -114,6 +114,15 @@ Use `uv run` to execute Python with the virtual environment:
 uv run python your_script.py
 ```
 
+Remote dataset loaders use `HF_DATASETS_CACHE` when it is set. Otherwise they cache
+Hugging Face data under `dbdata/huggingface`. On Windows, a long worktree path can
+cause `WinError 206` when Hugging Face creates a cache lock file. Set a shorter
+cache path before you start the server or script:
+
+```powershell
+$env:HF_DATASETS_CACHE = Join-Path $env:TEMP "pyrit-hf"
+```
+
 #### Running Tests
 
 ```bash

doc/gui/0_gui.md

@@ -34,7 +34,7 @@ To deploy an isolated instance for an external team, see [Deploy a New Instance]
 
 ## Views
 
-CoPyRIT has three main views, accessible from the left sidebar: **Chat**, **Attack History**, and **Target Configuration**. The **Theme** menu is available at the bottom of the sidebar.
+CoPyRIT has three main views, accessible from the left sidebar: **Technique.execute** (Chat), **Attack History**, and **Target Configuration**. The **Theme** menu is available at the bottom of the sidebar.
 
 ### Themes
 
@@ -511,7 +511,59 @@ Defaults are saved in this browser, separately for each signed-in account. They
 
 A missing or changed default is shown as unavailable. Select a new default or clear it; CoPyRIT does not silently substitute a different target. If browser storage is unavailable, a warning states that the choice applies only in the current session.
 
-Scanner forms let you override either selection for one run. **Objective Target** is the target the scenario runs against. **Adversarial Target**, under **Parameters** above **Dataset override**, is the target used to generate attacks. The adversarial selector appears only for scenarios whose available techniques use the shared default; scenarios with no such use, including the explicit-target adversarial benchmark, do not show it or send an override. **Use server default** clears the per-run adversarial override. Changes to your defaults do not change existing chats or queued/running scans. Explicit adversarial targets in a scenario or technique still take priority. Scorer targets are unchanged.
+Scanner forms let you override either selection for one run. **Objective Target** is the target the scenario runs against. **Adversarial Target**, under **Parameters**, is the target used to generate attacks. The adversarial selector appears only for scenarios whose available techniques use the shared default; scenarios with no such use, including the explicit-target adversarial benchmark, do not show it or send an override. **Use server default** clears the per-run adversarial override. Changes to your defaults do not change existing chats or queued/running scans. Explicit adversarial targets in a scenario or technique still take priority. Scorer targets are unchanged.
+
+### Scenario datasets
+
+**Datasets** is a separate section below **Techniques**. Its table has **Dataset**,
+**Size**, and **Include Criteria** columns. **Size** shows an upper bound, such as **Up to 4
+of 200**, when the stored total is known, or **Up to 4** before loading. This is not
+an exact selected count: filters and the combined objective limit can reduce it.
+Click a supported size to edit its limit, use all available examples, or reset to
+the displayed default. Required reference data shows its stored size without a
+limit editor.
+
+Select a name from **Add dataset** to add it immediately. Provider names remain
+available before their data is stored. Selecting a name never loads it. The remove
+icon is available only when the scenario permits removal and another source will
+remain. A scenario that permits only one source uses **Choose dataset** to replace
+it. **Browse datasets**, beside **Reset datasets**, opens the dataset browser.
+**Reset datasets** restores source defaults and row limits without clearing
+the combined objective limit or filters.
+
+**Load datasets** appears only while selected data is missing. It loads missing
+sources and keeps existing data, including manual edits. It may download data or
+use a configured generator, but does not start a scan. A missing source without a
+provider must be imported first. When all selected data is loaded, the load button
+and loading controls are omitted.
+
+Loaded rows show harm categories and data types as differently colored chips.
+All labels start checked when there is no configured filter. Uncheck labels to
+narrow the existing filters for **all selected datasets**, not just the row clicked.
+Checking all labels restores the default request when no filter was configured.
+Partial category selections require every checked label and retain substring
+matching; data types match any checked type. An empty selection blocks launch
+instead of silently restoring all data. Selection is shown on the metadata chips;
+there is no separate filter list below the table. No preview of data values is provided.
+
+**Maximum objectives** shows its resolved value. Click it to enter a positive
+integer, use all available objectives, or reset to the displayed default. This
+limits objectives, not prompts sent. Four objectives with three techniques produce
+12 attacks, plus four more if the baseline is enabled. Each attack can send several
+prompts. The run estimate shows the resulting attacks.
+
+Scenarios that build objectives from templates or reference data apply the total
+limit to the completed objectives, not to the required reference data. Their
+generation limits still apply. A scenario that requires one objective per selected
+technique rejects a total limit smaller than the number of techniques.
+
+**Load missing data when the scan starts** is one switch for the selected sources.
+Its initial state reflects their resolved loading rules, which normally load
+missing data. Turning it off prevents automatic loading on scan start without
+disabling the manual load action. Existing source-specific rules are not silently
+replaced until the switch is changed. Closing a limit editor does not clear an
+invalid draft. Estimate and launch use the same dataset edits.
+Resume uses the saved request and original selection and never loads datasets.
 
 #### Core Adversarial Default Override
 

doc/scanner/1_pyrit_scan.ipynb

@@ -1194,9 +1194,17 @@
     "on demand, so a full dataset preload is not required:\n",
     "\n",
     "```shell\n",
-    "pyrit_scan run garak.figstep --target openai_chat --dataset-names figstep_pro --max-dataset-size 1\n",
+    "pyrit_scan run garak.figstep --target openai_chat --datasets figstep_pro --max-total 1 --dataset-fetch if_missing\n",
     "```\n",
     "\n",
+    "Scanner and shell share `--datasets name[:N|:all]`, `--max-per-dataset`, `--max-total`,\n",
+    "and `--dataset-fetch never|if_missing`. Source limits apply before the combined total.\n",
+    "Omitted, empty, or `default` keeps the scenario setting; `all` removes only its own cap.\n",
+    "`list-scenarios` displays allowed sources and defaults. Ingredient/compound scenarios\n",
+    "restrict controls to preserve their shaping. `--dataset-filters` works with either form.\n",
+    "Deprecated `--dataset-names` and `--max-dataset-size` cannot be mixed with the new flags.\n",
+    "Discovery and estimates never fetch; only a new run prepares absent registered datasets.\n",
+    "\n",
     "Custom initialization scripts are loaded by the backend at startup: list them in the\n",
     "`initialization_scripts` section of the config the server loads (paths are relative to your\n",
     "working directory, but full paths avoid confusion). Once the server is running with that\n",

doc/scanner/1_pyrit_scan.py

@@ -125,9 +125,17 @@
 # on demand, so a full dataset preload is not required:
 #
 # ```shell
-# pyrit_scan run garak.figstep --target openai_chat --dataset-names figstep_pro --max-dataset-size 1
+# pyrit_scan run garak.figstep --target openai_chat --datasets figstep_pro --max-total 1 --dataset-fetch if_missing
 # ```
 #
+# Scanner and shell share `--datasets name[:N|:all]`, `--max-per-dataset`, `--max-total`,
+# and `--dataset-fetch never|if_missing`. Source limits apply before the combined total.
+# Omitted, empty, or `default` keeps the scenario setting; `all` removes only its own cap.
+# `list-scenarios` displays allowed sources and defaults. Ingredient/compound scenarios
+# restrict controls to preserve their shaping. `--dataset-filters` works with either form.
+# Deprecated `--dataset-names` and `--max-dataset-size` cannot be mixed with the new flags.
+# Discovery and estimates never fetch; only a new run prepares absent registered datasets.
+#
 # Custom initialization scripts are loaded by the backend at startup: list them in the
 # `initialization_scripts` section of the config the server loads (paths are relative to your
 # working directory, but full paths avoid confusion). Once the server is running with that

doc/scanner/adaptive.ipynb

@@ -26,8 +26,8 @@
     "pyrit_scan run adaptive.text_adaptive \\\n",
     "  --initializers target \\\n",
     "  --target openai_chat \\\n",
-    "  --dataset-names airt_hate \\\n",
-    "  --max-dataset-size 2 \\\n",
+    "  --datasets airt_hate \\\n",
+    "  --max-total 2 \\\n",
     "  --max-attempts-per-objective 2\n",
     "```"
    ]

doc/scanner/adaptive.py

@@ -25,8 +25,8 @@
 # pyrit_scan run adaptive.text_adaptive \
 #   --initializers target \
 #   --target openai_chat \
-#   --dataset-names airt_hate \
-#   --max-dataset-size 2 \
+#   --datasets airt_hate \
+#   --max-total 2 \
 #   --max-attempts-per-objective 2
 # ```
 

doc/scanner/airt.ipynb

@@ -102,8 +102,8 @@
     "  --initializers target \\\n",
     "  --target openai_chat \\\n",
     "  --techniques role_play_movie_script \\\n",
-    "  --dataset-names airt_hate \\\n",
-    "  --max-dataset-size 1\n",
+    "  --datasets airt_hate \\\n",
+    "  --max-total 1\n",
     "```\n",
     "\n",
     "**Available techniques:** ALL, DEFAULT, SINGLE_TURN, MULTI_TURN, role_play_movie_script, many_shot, tap"
@@ -255,8 +255,8 @@
     "\n",
     "Each sub-harm escalates a simulated multi-turn conversation toward the objective, then layers the\n",
     "selected converter techniques on top (natural-language rewrites that preserve emotional framing;\n",
-    "obfuscation converters like base64/morse are excluded). Datasets are bound to the sub-harms, so\n",
-    "`--dataset-names` is ignored (`--max-dataset-size` still applies).\n",
+    "obfuscation converters like base64/morse are excluded). `--datasets` can select a subset of the\n",
+    "selected sub-harms' datasets; `--max-total` must cover each selected sub-harm.\n",
     "\n",
     "**Available techniques:** ALL, DEFAULT, tone, language, persuasion, deterministic, crescendo"
    ]
@@ -413,7 +413,7 @@
     "  --initializers target \\\n",
     "  --target openai_chat \\\n",
     "  --techniques role_play_movie_script \\\n",
-    "  --max-dataset-size 1\n",
+    "  --max-total 1\n",
     "```\n",
     "\n",
     "**Available techniques:** Use `pyrit_scan run airt.cyber --list-scenario-parameters` to inspect\n",
@@ -570,8 +570,8 @@
     "pyrit_scan run airt.jailbreak \\\n",
     "  --initializers target \\\n",
     "  --target openai_chat \\\n",
-    "  --dataset-names harmbench \\\n",
-    "  --max-dataset-size 1\n",
+    "  --datasets harmbench \\\n",
+    "  --max-total 1\n",
     "```\n",
     "\n",
     "**Available technique selectors:** ALL, DEFAULT, and SINGLE_TURN currently select both\n",
@@ -730,8 +730,8 @@
     "pyrit_scan airt.multilingual \\\n",
     "  --initializers target \\\n",
     "  --target openai_chat \\\n",
-    "  --dataset-names harmbench \\\n",
-    "  --max-dataset-size 1\n",
+    "  --datasets harmbench \\\n",
+    "  --max-total 1\n",
     "```\n",
     "\n",
     "**Available techniques:** `prompt_sending` is the default. Every registry technique (`role_play_*`,\n",
@@ -870,7 +870,7 @@
     "plagiarism detection.\n",
     "\n",
     "```bash\n",
-    "pyrit_scan run airt.leakage --target openai_chat --techniques first_letter --max-dataset-size 1\n",
+    "pyrit_scan run airt.leakage --target openai_chat --techniques first_letter --max-total 1\n",
     "```\n",
     "\n",
     "**Available techniques:** ALL, SINGLE_TURN, MULTI_TURN, IP, SENSITIVE_DATA, FirstLetter, Image, RolePlay, Crescendo\n",
@@ -1049,7 +1049,7 @@
     "  --initializers target \\\n",
     "  --target openai_chat \\\n",
     "  --techniques context_compliance \\\n",
-    "  --max-dataset-size 1\n",
+    "  --max-total 1\n",
     "```\n",
     "\n",
     "**Available techniques:** ALL, DEFAULT, SINGLE_TURN, MULTI_TURN, ContextCompliance, RolePlay,\n",