Expand python findings

.agents/skills/alpine.md

@@ -145,6 +145,85 @@ Apptainer image. For NF1, prefer grouping compatible ZedProfiler feature familie
 by image-set when possible so each task loads image data once and writes validated
 outputs.
 
+## uv For ZedProfiler
+
+CURC documents a `uv` module for Python environments. The documented flow is:
+
+```bash
+module load uv
+uv venv "$UV_ENVS/mycustomenv" --python 3.12
+source "$UV_ENVS/mycustomenv/bin/activate"
+uv pip install <packages>
+```
+
+The documentation says `module load uv` creates and sets `UV_ENVS` to
+`/projects/$USER/software/uv/envs`. During live validation on `2026-08-04`,
+neither the regular login node nor `Persistence1` exposed an `uv` module via
+`module avail uv` or `module spider uv`.
+
+The tested fallback is a project-owned `uv` install that keeps the same CURC
+environment layout:
+
+```bash
+export UV_HOME="/projects/$USER/software/uv"
+export UV_INSTALL_DIR="$UV_HOME/bin"
+export UV_ENVS="$UV_HOME/envs"
+export UV_CACHE_DIR="/scratch/alpine/$USER/uv-cache"
+export PATH="$UV_INSTALL_DIR:$PATH"
+mkdir -p "$UV_INSTALL_DIR" "$UV_ENVS" "$UV_CACHE_DIR"
+
+if [[ ! -x "$UV_INSTALL_DIR/uv" ]]; then
+  curl -LsSf https://astral.sh/uv/install.sh | sh
+fi
+
+uv venv "$UV_ENVS/zedprofiler-simple" --python 3.12
+source "$UV_ENVS/zedprofiler-simple/bin/activate"
+uv pip install zedprofiler
+python - <<'PY'
+import importlib.metadata as metadata
+import zedprofiler
+print(metadata.version("zedprofiler"))
+print(zedprofiler.__file__)
+PY
+```
+
+Observed validation:
+
+- `uv 0.12.1` installed under `/projects/$USER/software/uv/bin`
+- `uv venv --python 3.12` used `/usr/bin/python3.12`
+- ZedProfiler `0.1.1` installed and imported
+- heavy dependencies resolved as wheels, including `mahotas`, `pyarrow`,
+  `numpy`, `pandas`, and `scikit-image`
+- with `UV_CACHE_DIR` on scratch and the env under `/projects`, `uv` warned
+  that hardlinking was not available and fell back to copying; set
+  `UV_LINK_MODE=copy` to make that explicit
+
+Validated simple experiment using the `uv` environment:
+
+```bash
+cd /scratch/alpine/$USER/formascute-codex-test
+module load nextflow/25.10.2
+source "/projects/$USER/software/uv/envs/zedprofiler-simple/bin/activate"
+make run EXPERIMENT=zp_synthetic_features ITEMS=4 ACCOUNT=amc-general PROFILE=alpine RUN_ID=zp-sim-uv-01
+```
+
+Observed result:
+
+- Nextflow ran from `/curc/sw/install/bio/nextflow/25.10.2_env/bin/nextflow`
+- `python3` resolved to the `uv` environment
+- preflight recorded Python `3.12.13`
+- `completion_status.txt` reported `nextflow_exit_status: 0`
+- `validation.json` reported `"valid": true`
+- 4 feature tasks plus 1 validation task were submitted to Slurm
+- 16 simulated feature rows were emitted
+
+For current repository behavior, prefer this direct `Persistence1` run pattern
+when testing `uv`: load the Nextflow module first, then activate the `uv`
+environment before invoking `make run`. The generated `make submit` coordinator
+currently treats `FORMASCUTE_ENV_DIR` as a replacement environment and skips
+module loading when it exists, so it is not yet the right interface for a
+combined Nextflow-module plus `uv` Python environment.
+
 ## Important Failure Mode
 
 Submitting directly from the regular Alpine login node failed because the batch

docs/alpine-findings.md

@@ -14,6 +14,9 @@ image data were used.
   not expose that module during testing.
 - `/usr/bin/apptainer` and `/usr/bin/singularity` are available on the tested
   compute environment.
+- A project-owned `uv` install under `/projects/$USER/software/uv` can build a
+  Python 3.12 ZedProfiler environment when the documented CURC `uv` module is
+  not visible.
 
 The validated smoke command was:
 
@@ -48,6 +51,7 @@ All runs validated successfully.
 | sequential | `nf1_featurization_batched` | 16 | 4 | 00:00:41 | 5 | 1 s |
 | sequential | `nf1_queue_size_4` | 16 | 1 | 00:01:02 | 17 | 1 s |
 | burst | `zp_synthetic_features` | 4 | 1 | about 00:00:42 | 5 | 0-1 s |
+| direct `uv` env | `zp_synthetic_features` | 4 | 1 | about 00:00:16 task span | 5 | 0-1 s |
 
 ## ZedProfiler-Shaped Probe
 
@@ -87,6 +91,61 @@ present in the tested base environment. A production NF1/ZedProfiler run should
 therefore use a project-owned environment or container instead of the default
 `Persistence1` Python.
 
+## uv Environment Probe
+
+CURC documents `module load uv`, with `$UV_ENVS` set to
+`/projects/$USER/software/uv/envs`, as the preferred `uv` environment location.
+During validation on `2026-08-04`, `module avail uv` and `module spider uv` did
+not find that module on either the regular login node or `Persistence1`.
+
+A user-space fallback worked:
+
+```bash
+export UV_HOME="/projects/$USER/software/uv"
+export UV_INSTALL_DIR="$UV_HOME/bin"
+export UV_ENVS="$UV_HOME/envs"
+export UV_CACHE_DIR="/scratch/alpine/$USER/uv-cache"
+export PATH="$UV_INSTALL_DIR:$PATH"
+export UV_LINK_MODE=copy
+
+uv venv "$UV_ENVS/zedprofiler-simple" --python 3.12
+source "$UV_ENVS/zedprofiler-simple/bin/activate"
+uv pip install zedprofiler
+```
+
+Observed result:
+
+- `uv 0.12.1` installed under `/projects/$USER/software/uv/bin`.
+- The environment used `/usr/bin/python3.12`, reporting Python `3.12.13`.
+- ZedProfiler `0.1.1` installed and imported.
+- Wheels resolved for previously missing dependencies including `mahotas`,
+  `pyarrow`, `numpy`, `pandas`, and `scikit-image`.
+- Keeping `UV_CACHE_DIR` on scratch and the environment under `/projects` causes
+  cross-filesystem hardlink fallback; `UV_LINK_MODE=copy` should be set
+  intentionally for this layout.
+
+The simple Nextflow experiment also completed with the `uv` environment active:
+
+```bash
+cd /scratch/alpine/$USER/formascute-codex-test
+module load nextflow/25.10.2
+source "/projects/$USER/software/uv/envs/zedprofiler-simple/bin/activate"
+make run EXPERIMENT=zp_synthetic_features ITEMS=4 ACCOUNT=amc-general PROFILE=alpine RUN_ID=zp-sim-uv-01
+```
+
+Result:
+
+- Nextflow exit status was `0`.
+- validation passed for 4 expected items.
+- the run emitted 16 simulated feature rows.
+- `trace.tsv` contained 4 feature tasks and 1 validation task.
+- `slurm.tsv` was collected for account `amc-general`.
+
+This upgrades the ZedProfiler runtime recommendation from "project-owned
+environment or container" to "`uv` environment first, Apptainer if native wheels
+or system libraries fail." It still does not validate real NF1 image I/O or a
+real ZedProfiler feature call inside a workflow task.
+
 ## Interpretation
 
 - Nextflow can successfully submit and track Slurm jobs on Alpine when submitted
@@ -102,6 +161,12 @@ therefore use a project-owned environment or container instead of the default
   control knob for NF1-style work.
 - The ZedProfiler-shaped probe behaved like the earlier tiny fan-out tests:
   scheduler wait was negligible, and task overhead dominated actual work.
+- `uv` is a practical first runtime path for ZedProfiler on Alpine, even though
+  the documented module was not visible during testing.
+- The current `make submit` coordinator is not yet the best `uv` interface
+  because it treats `FORMASCUTE_ENV_DIR` as a replacement for module loading.
+  For now, direct `Persistence1` runs should load Nextflow first and then
+  activate the `uv` environment.
 
 ## Recommended NF1 Orchestration Direction
 
@@ -114,7 +179,10 @@ Recommended first implementation:
 - Use one Nextflow process for CPU featurization work items.
 - Keep the existing Python feature scripts as the task payload.
 - Use Slurm executor with `acpu`, `cpu-normal`, and account configuration.
-- Build an explicit ZedProfiler runtime environment before real data tests.
+- Use a project-owned `uv` environment as the first ZedProfiler runtime before
+  real data tests.
+- Update the coordinator bootstrap so it can load the Nextflow module and
+  activate a separate Python runtime environment in the same submitted job.
 - Start with independent jobs for real feature tasks when each task loads a
   single image-set and runs enough feature work to amortize Slurm overhead.
 - Add optional batching when individual feature calls are very short or repeatedly
@@ -135,6 +203,10 @@ Avoid for the first production-facing iteration:
 - Mixing CPU and GPU work in one executor profile.
 - Running real NF1 image data before a tiny real-work probe is designed.
 - Relying on the base `Persistence1` Python environment for ZedProfiler.
+- Assuming the documented CURC `uv` module is available without checking
+  `module spider uv`.
+- Pointing `FORMASCUTE_ENV_DIR` at the `uv` environment before the submit
+  wrapper can separately load Nextflow.
 
 ## Questions For CURC
 
@@ -150,5 +222,7 @@ Ask CURC these before scaling:
   independent submission easier for accounting and retries?
 - Should CPU featurization and SAMMed3D GPU work be split into separate Nextflow
   profiles and runs?
-- Should ZedProfiler be installed as a project conda/uv environment or as an
-  Apptainer image for production NF1 runs?
+- Is CURC's documented `uv` module expected to be available on Alpine now, and
+  if so, which module tree or host should expose it?
+- For production NF1 runs, is a project-owned `uv` environment acceptable, or
+  should CURC prefer an Apptainer image once image I/O dependencies are known?

docs/nf1-pipeline-target.md

@@ -56,6 +56,15 @@ make submit EXPERIMENT=nf1_submit_rate_4 ITEMS=16 ACCOUNT=<allocation> SUBMIT_HO
 make submit EXPERIMENT=zp_synthetic_features ITEMS=8 ACCOUNT=<allocation> SUBMIT_HOST=Persistence1
 ```
 
+For `uv` runtime checks, use a direct `Persistence1` run until the submit
+wrapper can load the Nextflow module and activate a separate Python environment:
+
+```bash
+module load nextflow/25.10.2
+source "/projects/$USER/software/uv/envs/zedprofiler-simple/bin/activate"
+make run EXPERIMENT=zp_synthetic_features ITEMS=4 ACCOUNT=<allocation> PROFILE=alpine
+```
+
 Interpretation:
 
 - `nf1_minimal_overhead` estimates Nextflow + Slurm coordinator overhead before
@@ -94,9 +103,17 @@ tasks. On Alpine, the base `Persistence1` Python environment is not ready for
 real ZedProfiler work because it provides Python 3.9 and lacks required packages
 such as `mahotas`, `pyarrow`, and `zedprofiler`.
 
+A project-owned `uv` environment is now the preferred first runtime path:
+Python 3.12.13, ZedProfiler `0.1.1`, `mahotas`, `pyarrow`, `numpy`, `pandas`,
+and `scikit-image` installed and imported successfully from
+`/projects/$USER/software/uv/envs/zedprofiler-simple`. The documented CURC `uv`
+module was not visible during validation, so the tested fallback installs `uv`
+under `/projects/$USER/software/uv/bin` and uses scratch for `UV_CACHE_DIR`.
+
 For the NF1 refactor, prefer:
 
-- a project-owned Python 3.11+ environment or Apptainer image for ZedProfiler
+- a project-owned `uv` Python 3.11+ environment first, with Apptainer as the
+  fallback if native wheels or system libraries fail
 - a Nextflow process that consumes a manifest of image-set work items
 - task payloads that can compute multiple compatible feature families after one
   image load
@@ -106,9 +123,9 @@ For the NF1 refactor, prefer:
 ## What This Does Not Answer Yet
 
 These experiments do not measure image-processing runtime, GPU memory, CellPose,
-SAMMed3D, DuckDB, parquet performance, or real ZedProfiler dependency
-installation. They only characterize scheduler, submission, retry, validation,
-and artifact behavior with a small controlled workload.
+SAMMed3D, DuckDB, parquet performance, or real ZedProfiler feature execution.
+They only characterize scheduler, submission, retry, validation, artifact
+behavior, and the basic viability of installing/importing ZedProfiler with `uv`.
 
 Do not use NF1 image data in this phase. Data-based testing belongs in a later
 phase, after synthetic runs have shown stable submission, accounting,

docs/north-star.md

@@ -26,8 +26,10 @@ This should be a thin workflow layer around real work, not a broad rewrite.
   separate profiles or separate runs.
 - Do not rely on the base `Persistence1` Python for ZedProfiler. It is Python
   3.9 and lacks required packages.
-- Use a project-owned Python 3.11+ environment or an Apptainer image for
-  ZedProfiler.
+- Use a project-owned `uv` Python 3.11+ environment as the first ZedProfiler
+  runtime path; use Apptainer if native wheels or system libraries fail.
+- The documented CURC `uv` module was not visible during validation, but a
+  user-space `uv` install under `/projects/$USER/software/uv/bin` worked.
 
 ## Target Shape
 
@@ -140,7 +142,7 @@ feature extraction with a tiny controlled manifest.
 Recommended slice:
 
 1. Add a manifest builder for a few known-safe image-sets.
-2. Add a ZedProfiler runtime environment definition.
+2. Add a ZedProfiler `uv` runtime environment definition.
 3. Add one Nextflow process for CPU feature extraction.
 4. Add per-task validation and summary output.
 5. Run one image-set with real ZedProfiler on Alpine.
@@ -155,7 +157,7 @@ or production data movement in the first slice.
 Scale only after these gates pass:
 
 - formascute synthetic Alpine checks pass from a fresh clone
-- ZedProfiler environment builds reproducibly on Alpine
+- ZedProfiler `uv` environment builds reproducibly on Alpine
 - one real image-set completes and validates
 - 4-8 real image-sets complete and validate
 - `trace.tsv` and `slurm.tsv` show acceptable queue wait and memory use
@@ -178,7 +180,7 @@ Avoid these as first moves:
 
 The best direction today is a thin Nextflow wrapper around NF1 Stage 3 that
 consumes a manifest of image-set work items and runs ZedProfiler in a controlled
-Python 3.11+ environment. Start with independent CPU tasks on `acpu` /
+`uv` Python 3.11+ environment. Start with independent CPU tasks on `acpu` /
 `cpu-normal` through `Persistence1`, validate every output, collect trace and
 Slurm accounting, and tune batching only after real image-set runtimes show
 whether repeated image loading or scheduler overhead dominates.