Skip to content

Propagate metadata upload failures through setup - #12805

Open
youseihuayu-wonderful wants to merge 3 commits into
IQSS:developfrom
youseihuayu-wonderful:12804-metadata-upload-http-errors
Open

youseihuayu-wonderful wants to merge 3 commits into
IQSS:developfrom
youseihuayu-wonderful:12804-metadata-upload-http-errors

Conversation

@youseihuayu-wonderful

@youseihuayu-wonderful youseihuayu-wonderful commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Closes #12804. Related to #12467.

Problem

The shared metadata uploader ignored HTTP errors: an HTTP 500 could still end with exit 0 and "Done". Bootstrap and the installer also use this script.

Fix

  • Check HTTP and curl status; report the failed block, status and response body.
  • Attempt all standard blocks and return non-zero if any failed.
  • Finish remaining setup/persona configuration and security lockdown before returning failure. Preserve dev's intentionally insecure mode and installer step 8c.
  • Keep 10156 fast redeploy maven #12467 from reaching Solr updates or "Done" after a metadata failure.

Verification

  • Shellspec: 44 examples, 0 failures, 1 existing skip; stricter POST/content-type/file checks and lockdown/status regressions.
  • Live Docker/Dataverse: valid TSV persisted a new field; HTTP 500 identified geospatial TSV line 42.
  • Failed dev/demo bootstrap still created admin/root and completed persona configuration. Demo admin APIs remained blocked; dev exported its token.
  • Actual packaged installer post-deployment stage completed lockdown and remote Solr configuration, then exited 0/1 as appropriate. Full host provisioning was not tested.
  • 10156 fast redeploy maven #12467 exited 1 before Solr updates or "Done".

Guide/release note updated. Successful block uploads are not rolled back. Implementation and verification were AI-assisted.

@ErykKul
ErykKul self-requested a review October 8, 2026 07:33
@ErykKul ErykKul self-assigned this Oct 8, 2026
@ErykKul ErykKul added the Size: 10 A percentage of a sprint. 7 hours. label Oct 8, 2026
@ErykKul ErykKul moved this from Ready for Triage to In Review 🔎 in IQSS Dataverse Project Oct 8, 2026

@ErykKul ErykKul left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up! The uploader change looks good. The main issue is the early exit: a failed block now skips the rest of setup, including the admin API lockdown, and leaves an instance that can't be re-bootstrapped. The inline comments on setup-all.sh and install.py suggest letting setup finish and exiting non-zero at the end, plus a test for that.

Comment thread scripts/api/setup-all.sh Outdated
Comment thread scripts/installer/install.py Outdated
Comment thread tests/shell/spec/setup_datasetfields_spec.sh
@ErykKul ErykKul added Size: 3 A percentage of a sprint. 2.1 hours. and removed Size: 10 A percentage of a sprint. 7 hours. labels Oct 8, 2026
@youseihuayu-wonderful

Copy link
Copy Markdown
Contributor Author

@ErykKul Thanks — addressed in 17aa7aa.

Setup now retains the metadata failure, finishes remaining configuration and security lockdown, then exits non-zero. Dev stays intentionally insecure; demo completes its lockdown; the installer still runs step 8c; bootstrap preserves env export.

Added lockdown/status regressions and stricter curl request checks: 44 examples, 0 failures, 1 existing skip. Live Docker checks confirmed admin/root creation, demo/installer API blocking, and installer exit 0/1. #12467 still stops before Solr and "Done" on HTTP 500. Guide/release note corrected.

Installer verification covers its actual packaged post-deployment stage, not full host provisioning. Ready for re-review.

@youseihuayu-wonderful

Copy link
Copy Markdown
Contributor Author

@ErykKul
Hi Eryk, I’ve updated #12805 (#12805) to address the review comments. Could you take another look? The JSF job failed while pulling the Keycloak image from quay.io with a 504 error—could you rerun that job?

@ErykKul

ErykKul commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

Looks good to me, approved. QA'd with fresh unstable images and this PR's scripts, using a TSV with an invalid displayOrder:

  • dev bootstrap, valid TSVs: exit 0, everything configured
  • dev bootstrap, broken TSV: the error shows the block, the HTTP 500 and the TSV line; the rest of setup completes; exit 1
  • demo bootstrap, broken TSV: lockdown still applied (admin API blocked, burrito revoked); exit 1

#12467's update-metadata.sh: stops before Solr and "Done" on failure, works normally otherwise
Not tested: the host installer.

@pdurbin
pdurbin force-pushed the 12804-metadata-upload-http-errors branch from 17aa7aa to 3c36d05 Compare October 9, 2026 20:43

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Size: 3 A percentage of a sprint. 2.1 hours.

Projects

Status: Ready for QA ⏩

Development

Successfully merging this pull request may close these issues.

Metadata setup reports success after HTTP upload failures

3 participants