Skip to content

MultiBackendJobManager: Dynamically add all usage metrics in job DB. … - #947

Merged
soxofaan merged 6 commits into
masterfrom
job-manager-dynamic-usage-columns
Oct 5, 2026
Merged

soxofaan merged 6 commits into
masterfrom
job-manager-dynamic-usage-columns

Conversation

@VincentVerelst

@VincentVerelst VincentVerelst commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Instead of hardcoded memory, duration and cpu metrics, the MBJM now fetches all usage metrics from the job metadata and writes them to the job DB. This behaviour is more backend agnostic and doesn't require a change in the source code, each time we want a new metric in the job DB

@soxofaan soxofaan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some quick notes from superficial review

Comment thread openeo/rest/_testing.py Outdated
Comment thread openeo/rest/_testing.py Outdated
@VictorVerhaert

Copy link
Copy Markdown
Contributor

Only thing I am a bit worried about, but I'd have to look into more thoroughly, is data type handling. If a users updates their openeo and loads in an old dataframe, will it give issues?
Is there a test that tries to load in an old database?

@VincentVerelst

Copy link
Copy Markdown
Collaborator Author

Good call, there was one edge case where the job manager would fail: for an unfinished job database with the legacy columns present. I.e. when a user would update the python client version in the middle of processing. That should now also be handled + unit test added. Let's see if they pass.

@soxofaan

soxofaan commented Oct 5, 2026

Copy link
Copy Markdown
Member

can you resolve the conflict on the changelog?

@soxofaan
soxofaan merged commit 71bf07a into master Oct 5, 2026
17 checks passed
@soxofaan
soxofaan deleted the job-manager-dynamic-usage-columns branch October 5, 2026 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants