Skip to content

Update to NodeODM 3.6.2 - #9

Open
MJohnson459 wants to merge 10 commits into
OpenDroneMap:mainfrom
MJohnson459:nodeodm-3.6.2
Open

MJohnson459 wants to merge 10 commits into
OpenDroneMap:mainfrom
MJohnson459:nodeodm-3.6.2

Conversation

@MJohnson459

Copy link
Copy Markdown
Collaborator

This was a bit more complicated than I expected but it now passes all the tests locally, and I have run it and processed the Brighton dataset.

  1. I changed the CI following a similar pattern to ODM and NodeODM.
  2. The CI tests were failing due to a missing submodule. I believe this was deleted by accident by a script.
  3. Once I had the tests running, a lot of them were failing due to changes introduced when the Alpine branch merged with a more recent version:
    1. The update to the JWT auth library changed the auth API. I tried to walk a middle ground maintaining the old look but using the new library.
    2. Many API routes that were added recently were lost which I just restored.
  4. I removed the "impersonate" action for auth. This made the auth transition a lot more complicated and I don't believe we need it. I can reinstate it if I'm wrong.

These were the main things, but there were a few more minor ones I had to fix. And obviously this updates the NodeODM pin to 3.6.2.

Once merged this will publish a docker image to opendronemap/webui (we will need to add DOCKERHUB_USERNAME and DOCKERHUB_TOKEN secrets).

Finally, I tried to do no more in this PR than necessary to get it building and passing the tests. There are certainly some oddities left like references to WebODM datasets, frozen pyodm deps, and tools pulled from Piero's repos. We should probably do a separate cleanup pass once we have the CI working.

The gitlink was dropped in 3623d56. Without it, nodeodm/setup.sh fails
in the TEST_BUILD image and the Test Docker workflow cannot run.
opendronemap/nodeodm:stable is no longer updated. Use the release tags
3.6.2 and 3.6.2-gpu.
The node info refresh caught exceptions.GenericError, which exists in
pyodx but not in the pinned pyodm 1.5.11. An unreachable node raised
AttributeError from update_node_info instead of returning False.

nodeodm/tests.py imported django.utils.six, removed in Django 3.0.
The publish workflow triggered on master, which no longer exists, and
pushed to webodm/webodm_webapp. It now uses docker/github-builder:
:edge from main, :<version> and :latest from a v* tag, amd64 and arm64
on native runners.

docker-compose.yml runs the published image; docker-compose.build.yml
still overrides it for local builds. webui.sh update pulls main.
7e6376a replaced app/api/urls.py and app/urls.py with older versions.
The views for media, panorama tiles, video flight paths, external task
import, worker cancel, users, groups, user profile, external token auth
and OIDC still existed but nothing routed to them.
/api/token-auth/ and /api/token-auth/refresh/ are simplejwt's sliding
token views. Existing clients keep working: the response carries a
token field, refresh takes a token, and the JWT header prefix is
accepted alongside Bearer. A failed login returns 401 instead of 400.
Superuser impersonation is not carried over.

Query string authentication overrides authenticate(); simplejwt never
calls get_raw_token when there is no Authorization header.

jwt.decode calls use the PyJWT 2 signature.
filter_class and filter_fields are ignored by django-filter 24, so the
id and has_available_options filters on processing nodes did nothing.
Project task ids are listed newest first instead of in table order.
django.utils.timezone.utc was removed in Django 5, so restoring a task
from a backup logged 'Cannot read backup file' and lost its fields.

Tests: assertFormError takes the form, response headers are read through
the response, assertEquals is gone in Python 3.12, orthophoto metadata
bounds are reported in EPSG:4326, the WKT import fixture is fetched from the WebODM release that hosts
it (the rebrand had rewritten the URL to a release this repository
does not have),
and the test processing node is terminated in a finally block so one
failing test no longer leaves a node running for every test after it.
app/imageutils.py calls exiftool to detect panoramas and it was not
installed.
webui.sh rebuild removed nodeodm/external/NodeODM, which is a submodule.
The next test image build then failed in nodeodm/setup.sh and git showed
the submodule as deleted. Only its node_modules is removed now.
@smathermather

Copy link
Copy Markdown
Collaborator

Tested on my end. It'd be good to get a once over from @sashinexists and/or @Saijin-Naib, if they have the chance. But non-blocking request. It looks good from my end.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants