Skip to content

refactor(struct): consolidate shared graph-core foundations - #3270

Merged
imbajin merged 11 commits into
apache:masterfrom
hugegraph:feat/struct-consolidation-1.8
Oct 7, 2026
Merged

imbajin merged 11 commits into
apache:masterfrom
hugegraph:feat/struct-consolidation-1.8

Conversation

@imbajin

@imbajin imbajin commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Purpose of the PR

Server core and Struct currently keep separate implementations of IDs, schema, queries and codecs. A fix can land in one copy while callers continue using the other. This PR gives those shared types one implementation in Struct, keeps graph execution in Core, and removes graph-engine dependencies from PD and Store.

For example, both Server and Store will use org.apache.hugegraph.id.IdGenerator; the old Core copy is removed. Core still owns transactions, traversal and backend adapters.

Before, Core and Struct maintain parallel shared implementations. After, Core and Store use Struct; Struct and PD use Common.

Shared ownership only; the diagram omits client and runtime dependencies. Core still uses PD/Store clients for HStore execution.

Related to hugegraph#257. Earlier review is in hugegraph/hugegraph#260.

Main Changes

Area What changes What to review
Shared types Struct owns IDs, schema metadata, queries, base elements, indexes, analyzers and codecs Callers use one implementation; Core wrappers retain engine callbacks and one shared element state
Service responsibilities Store owns PD-backed schema connections, watches and cache lifecycle; Common owns JWT helpers and RPC configuration contracts Resource cleanup, namespace handling and service-specific failure responses
Java API and SPI Types move packages; signatures such as sameAs and index serialization use shared types Update implementations and callers, then recompile integrations
Historical data Readers recognize supported old Java class names and enum ordinals Frozen producer fixtures and streamed Kryo decoding; canonical writer formats stay stable
Behavioral repairs String IDs use consistent UTF-16 ordering; schema key order is preserved; HStore OLAP keys include the property ID These changes deserve separate review from package moves

What developers need to do

Migration steps: back up data and assess indexes, update Java imports and SPI signatures and recompile, upgrade Server/PD/Store together with matching namespaces, then start packaged services and test reads and writes. HStore OLAP rows use property-and-vertex keys.

This prepares the planned 1.8.0 migration without changing release version numbers.

If you… Required action
Maintain a Java plugin or integration using the moved types Update imports and affected method signatures, then rebuild against matching artifacts. Old compiled plugins are not binary-compatible
Deploy Server, PD and Store Upgrade them together and match their PD metadata namespace. Mixed-version rolling upgrades are unsupported
Used Store-side rebuilding of long-text indexes Assess and rebuild affected indexes from graph data. Old shortened keys may not match full-value queries; this PR does not rebuild them automatically
Use HStore OLAP properties New rows use (property ID, vertex ID). Reads fall back only to matching legacy rows; values already overwritten by the old writer cannot be recovered

The migration guide includes the API mapping, configuration examples and compatibility details. Affected Toolchain and Computer callers require matching artifacts and their own build/classpath validation before release.

Verifying these changes

  • Trivial rework / code cleanup without test coverage.
  • Existing engine, analyzer, PD/Store and API tests, plus focused shared-type and historical-data regressions.
  • Additional validation with resolved dependency trees and actual packaged classpaths.

Rebased on current master after #3263 merged, retaining its GraphBinary Schema/Blob, GraphSON V1 Tree, GValue/ID predicate, named range/count, DATE and kubeconfig compatibility repairs. Shared-type references now use their canonical Struct owners; the current CI policy is retained.

CI builds now install concrete parent versions for Maven 3.10, and benchmark packaging preserves the flattened test POM. Cluster tests use matching Commons with the existing client. Task cancellation waits for actual script progress, and unused PowerMock dependencies and their release entries are removed.

Local Java 17 compilation, shared-type/historical-codec regressions, Server unit tests, focused Memory/RocksDB query regressions and Store ingress/schema tests are validated against this integrated tree. Fresh service distributions are checked for duplicate HugeGraph classes and their actual runtime/nested dependency inventory. Upstream CI and full TinkerPop/HStore qualification on the updated head remain required before merging.

Does this PR potentially affect the following parts?

  • Dependencies: resolved versions, packaged class ownership, inventory and license coverage.
  • Modify configurations: Store PD namespace and Gremlin class imports.
  • The public API: Java/SPI types and method descriptors.
  • Other affects: documented index and OLAP behavior.
  • Nope

Documentation Status

Coordinate code and website merges, then complete downstream delivery before release.

@codecov

codecov Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 53.67847% with 170 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.18%. Comparing base (d9abcd4) to head (704b980).

Files with missing lines Patch % Lines
...hugegraph/backend/serializer/BinarySerializer.java 47.91% 18 Missing and 7 partials ⚠️
.../java/org/apache/hugegraph/structure/HugeEdge.java 58.62% 17 Missing and 7 partials ⚠️
...ava/org/apache/hugegraph/structure/HugeVertex.java 65.21% 19 Missing and 5 partials ⚠️
...he/hugegraph/backend/tx/GraphIndexTransaction.java 44.73% 19 Missing and 2 partials ⚠️
...va/org/apache/hugegraph/structure/HugeElement.java 61.53% 17 Missing and 3 partials ⚠️
...che/hugegraph/schema/builder/EdgeLabelBuilder.java 0.00% 8 Missing ⚠️
...n/java/org/apache/hugegraph/auth/HugeResource.java 0.00% 6 Missing ⚠️
...gegraph/backend/store/memory/InMemoryDBTables.java 16.66% 5 Missing ⚠️
.../apache/hugegraph/backend/tx/GraphTransaction.java 20.00% 3 Missing and 1 partial ⚠️
...a/org/apache/hugegraph/structure/HugeProperty.java 69.23% 1 Missing and 3 partials ⚠️
... and 18 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3270      +/-   ##
============================================
+ Coverage     41.03%   41.18%   +0.15%     
+ Complexity     7239     6772     -467     
============================================
  Files           788      766      -22     
  Lines         68451    66086    -2365     
  Branches       9256     8773     -483     
============================================
- Hits          28087    27217     -870     
+ Misses        37090    35821    -1269     
+ Partials       3274     3048     -226     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@imbajin
imbajin marked this pull request as ready for review October 4, 2026 18:07
@imbajin
imbajin force-pushed the feat/struct-consolidation-1.8 branch from 8fa29fd to 40e10a7 Compare October 5, 2026 14:28
@imbajin imbajin changed the title refactor: consolidate shared graph foundations refactor(struct): consolidate shared graph-core foundations Oct 5, 2026
MrJs133
MrJs133 previously approved these changes Oct 5, 2026
imbajin added 10 commits October 6, 2026 22:34
- make struct the canonical ID, schema, query, element and codec owner
- move schema network lifecycle to Store and JWT/RPC contracts to common
- preserve historical formats and repair bounded HStore OLAP operations
- migrate callers and add compatibility fixtures and package boundary guards
- document Java migration and refresh dependency inventory and licensing
- retain the original ASF header and third-party source attribution
- run historical Store ingress tests within their node module owner
- keep the client decoder dependency limited to the test classpath
- require the relocated CI tests and aggregate their coverage session
- prevent Python caches from entering source formatting checks
- restore the canonical exception import after merging master
- compare string IDs consistently across lazy representations
- cover Unicode ordering contracts and document the correction
- consume class descriptors once across input refills
- cover frozen legacy and canonical streamed payloads
- retain Docker build output alongside image IDs
- exclude the older Gremlin configuration dependency
- retain the baseline language utilities without PD coupling
- align the generated inventory and release notices
- declare commons-text 1.10.0 directly for PD service
- avoid inheriting version 1.9 through configuration
- retain core-free service, distribution and test graphs
- Pass generated PD peers and the Store namespace to Server REST settings
- Allocate and configure a distinct Gremlin endpoint for each Server
- Detect completed startup independently of the JVM log locale
- Explain shared ownership with a before-and-after diagram
- Lead with developer impact and concrete Java migration examples
- Add an upgrade illustration and retain compatibility reference tables
- Remove repository illustrations while retaining PR description images
- Consolidate historical provenance and remove unused Store copies
- Correct serializer wording and remove fork/session records
- Reflow Markdown paragraphs and fix producer source newlines
- adapt self-loop queries to shared element state
- execute Store ingress tests with Jupiter lifecycle annotations
- preserve the Lucene 8.11.2 analyzer dependencies
- restore upgrade legal notices and exact dependency versions
- refresh the inventory from clean runtime distributions
- flatten installed parent POMs for Maven 3.10
- preserve the test POM during benchmark shading
- align Cluster Commons with the verified client API
- synchronize task cancellation with script progress
- remove unused PowerMock and refresh legal inventory
@imbajin
imbajin merged commit 7398075 into apache:master Oct 7, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants