FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

refactor(server): adapt REST API for hubble by imbajin · Pull Request #3159 · apache/hugegraph · GitHub

refactor(server): adapt REST API for hubble - #3159

Open
imbajin wants to merge 28 commits into
apache:masterfrom
hugegraph:cx/bump-server-api-version
Open

refactor(server): adapt REST API for hubble#3159
imbajin wants to merge 28 commits into
apache:masterfrom
hugegraph:cx/bump-server-api-version

Conversation

imbajin commented Aug 16, 2026
edited
Loading

Copy link
Copy Markdown
Member

Closeout status

  • Publish REST API 0.72 through source properties, the API JAR manifest
    and the runtime version endpoint.
  • Support explicit GraphSpace-wide ALL_GRAPHS roles for existing and
    future graphs.
  • Prepare the audit limiter during login so the first authenticated write
    succeeds without a warm-up request.
  • Apply concrete-graph ALL_GRAPHS checks and reject mixed-GraphSpace role
    disclosure.
  • Keep trusted metadata callbacks behind an internal elevation boundary and
    restore every auth, request GraphSpace and task context.
  • Remove the incomplete general Gremlin mutation classifier and implicit
    request-time legacy observer migration/cleanup.
  • Preserve the HStore schema-metadata clear bugfix and its required remote
    cache invalidation.
  • Restore only docker/hbase/** to the existing Docker CI trigger.
  • Rebase onto current ASF master without dropping the existing bugfixes.
  • Finish current-head CI and exact-image Kubernetes/Hubble acceptance.
  • Obtain upstream approval.

Before → After

Area Before After
API capability Current master and 1.7 both reported API 0.71 Current master reports API 0.72
GraphSpace roles Default roles were not recognized consistently for concrete graphs Explicit ALL_GRAPHS roles cover existing and future graphs
Role disclosure A manager shortcut could expose a mixed-space grant The shortcut accepts only one GraphSpace
Trusted callbacks Elevation depended on ambient/thread context A private callback boundary saves, clears and restores all contexts
Gremlin permissions An incomplete mutation classifier changed the established contract The existing GREMLIN EXECUTE contract remains
Legacy roles Request paths scanned and mutated per-graph observer bindings API 0.72 uses explicit ALL_GRAPHS; migration is a separate TODO
HStore deletion Backend clear could leave PD schema metadata behind Schema metadata and remote caches are cleared before provider data

Final-head evidence

  • Head: 6851b1cd49f1770f6f27152e468a6e58c933af79
  • Base synced to ASF master: c6853e70c6fd10e441d66a0633aca60640d0c7dd
  • Focused reactor tests: 44 passed, 0 failed
  • Standard UnitTestSuite: 661 passed, 0 failed, 1 skipped
  • Reactor build, Checkstyle/RAT and git diff --check: passed
  • Three independent re-reviews completed; the broader deletion-lifecycle race
    is explicitly tracked in the consolidated TODO rather than expanded here
  • Current-head GitHub checks and exact-image runtime acceptance: in progress

Approval and merge remain upstream review gates, not claims of this closeout.

dosubot Bot added size:XS This PR changes 0-9 lines, ignoring generated files. api Changes of API labels Aug 16, 2026
imbajin force-pushed the cx/bump-server-api-version branch from 67ff2e0 to 52035da Compare August 16, 2026 19:41
dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Aug 16, 2026

codecov Bot commented Aug 16, 2026
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 49.53271% with 54 lines in your changes missing coverage. Please review.
✅ Project coverage is 37.72%. Comparing base (c6853e7) to head (a54dc73).

Files with missing lines Patch % Lines
.../org/apache/hugegraph/auth/HugeGraphAuthProxy.java 68.96% 10 Missing and 8 partials ⚠️
...a/org/apache/hugegraph/auth/HugeAuthenticator.java 0.00% 12 Missing ⚠️
...g/apache/hugegraph/auth/StandardAuthManagerV2.java 30.76% 7 Missing and 2 partials ⚠️
.../apache/hugegraph/api/space/SchemaTemplateAPI.java 14.28% 3 Missing and 3 partials ⚠️
.../org/apache/hugegraph/api/space/GraphSpaceAPI.java 16.66% 2 Missing and 3 partials ⚠️
...java/org/apache/hugegraph/api/auth/ManagerAPI.java 0.00% 3 Missing ⚠️
...n/java/org/apache/hugegraph/StandardHugeGraph.java 75.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3159      +/-   ##
============================================
- Coverage     37.72%   37.72%   -0.01%     
- Complexity     6521     6529       +8     
============================================
  Files           800      800              
  Lines         68821    68882      +61     
  Branches       9127     9147      +20     
============================================
+ Hits          25965    25988      +23     
- Misses        39815    39844      +29     
- Partials       3041     3050       +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Aug 17, 2026

imbajin commented Aug 17, 2026
edited
Loading

Copy link
Copy Markdown
Member Author

Why this branch contains PD Raft changes

The PD change in 3d9d9544 is independent of the REST API 0.72 bump. It was identified while preparing the real 3-PD distributed matrix, where PD peers use stable hostnames but container/Pod IPs may change after restart or membership updates.

Before:

  • IpAuthHandler resolved peer hostnames only at startup or an explicit peer-list update;
  • the same hostname resolving to a new Pod IP could leave the Raft allowlist stale and reject a valid peer;
  • peer changes could update Raft membership without a reliably synchronized allowlist;
  • a fully unresolved/empty result followed the old allow-all path;
  • resolver, singleton, RPC-server and failed-start cleanup were incomplete.

After:

  • configured/runtime peer addresses are normalized and bounded before use;
  • DNS resolution has timeout, concurrency and stale-entry limits, with background refresh;
  • successful membership changes refresh the active allowlist;
  • startup/shutdown failures clean the resolver, handler and RPC resources;
  • unit, service-update and Raft integration tests cover these boundaries.

Most of the apparent size is defensive behavior and tests: this commit is +1375/-226, including about 700 added test lines and relocation/expansion of the old IpAuthHandlerTest.

Scope note: this is not required to distinguish API 0.71 from 0.72, nor by the Hubble permission capability itself. It should be reviewed as a standalone PD reliability/security change (and preferably split from the API-version PR). Passing tests do not replace the pending final 3+3+3 runtime validation.

Scope freeze

The Hubble dependency is limited to the API 0.72 contract and the GraphSpace-wide default-role behavior. The PD Raft DNS/IP work is independent and must not grow further except for a blocker reproduced by the required 3+3+3 runtime. It should remain separately reviewable/splittable from the Hubble compatibility change.

imbajin left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Blocking: yes. Summary: No actionable code defect was independently confirmed at this head. Approval remains blocked by the current codecov/project failure. Evidence: Java 11 targeted tests passed: PD auth/raft 20/20, PDService raft update 6/6, GraphSpace API 13/13; current checks show codecov/project FAILURE.

dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. size:L This PR changes 100-499 lines, ignoring generated files. labels Aug 17, 2026
bitflicker64 added a commit to hugegraph/hugegraph that referenced this pull request Aug 18, 2026
Bring in server REST API 0.72 changes (PR apache#3159) for local Hubble auth testing.

imbajin left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Blocking: yes. Summary: The new Gremlin mutation pre-check misses TinkerPop merge steps, so execute-only users can reach mutation paths without a required WRITE check. Evidence: traversalPermissions() only recognizes AddVertexStartStep, AddVertexStep, AddEdgeStartStep, AddEdgeStep, AddPropertyStep, and DropStep at HugeGraphAuthProxy.java:2533-2541; TinkerPop 3.7.6 also has MergeVertexStep and MergeEdgeStep, whose on-match path can call Vertex.property directly. Please classify both merge steps as WRITE (including nested children) and add execute-only regression tests for mergeV/mergeE create and onMatch cases.

dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. size:XXL This PR changes 1000+ lines, ignoring generated files. labels Aug 18, 2026

imbajin left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Blocking: yes. Summary: The current head has authorization correctness and isolation regressions in GraphSpace role checks, Gremlin mutation enforcement, and callback context cleanup. Evidence: exact-head static review plus six independent read-only lanes; latest checks pass for builds/Docker/CodeQL, while dependency-review is cancelled and Codecov failures are non-blocking.

dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Aug 21, 2026

Copy link
Copy Markdown

Superseded by hugegraph#198. The reviewed fixes and remaining compatibility acceptance are now tracked in the organization repository; please close this ASF PR.

imbajin and others added 20 commits August 29, 2026 04:41
- keep membership roles out of data action matching
- preserve explicit read write and delete permissions
- verify members can read without gaining mutations
- retain direct GraphSpace administrator handling
- initialize audit limiter only after successful authentication
- keep failed password and token attempts out of limiter state
- invalidate limiter entries by username when deleting users
- remove PD dynamic DNS and IP refresh from this PR
- cover password token and cleanup paths with unit tests
- run metadata callbacks with an internal admin context
- restore the previous context on success or failure
- prevent admin propagation into callback child threads
- cover task override and context restoration boundaries
- allow space managers to inspect users in their own space
- reject users without current-space grants and global admins
- cover cross-space and multi-space permission boundaries
- recognize custom global admins for user updates
- allow custom global admins to delete ordinary users
- preserve builtin admin behavior and deletion safeguards
- cover builtin and custom admin mutation paths
- align template ownership with global admin semantics
- preserve creator and GraphSpace manager access
- cover all four template management roles
- align Checkstyle and EditorConfig at 120 columns
- update contributor and module style guidance
- compact only current PR code without legacy reformatting
- defer authenticator lookup until manager access is needed
- keep anonymous creators on the owner mutation path
- cover lazy owner and manager authorization paths
- recognize mergeV and mergeE as write operations
- preserve compatibility with the current TinkerPop baseline
- reject execute-only create and onMatch traversals
- verify recursive child traversal permissions
- match merge steps by exact TinkerPop class names
- retain superclass traversal for provider implementations
- reject unrelated steps sharing merge simple names
- preserve recursive child permission coverage
- add pinned-3.5 test fixtures for TinkerPop merge steps
- route vertex and edge merge shapes through strategy checks
- retain external same-name and child traversal regressions
- move compatibility fixtures into test output only
- add test output to the unit-test classpath
- construct exact-package fixtures reflectively
- keep main artifacts free of TinkerPop shadow classes
- select fixture sources from the pinned TinkerPop version
- keep 3.5.1 compatibility classes in test output only
- stop selecting fixtures automatically after a version change
- preserve merge permission regression coverage
- prefer the official traversal and isStart constructor
- use Merge.onMatch through reflection when available
- keep an explicit 3.5.1 fixture child fallback
- preserve merge authorization coverage
- restore the TinkerPop 3.5.1 authorization scope\n- remove future-version merge detection and fixtures\n- keep the Hubble permission closeout focused on reproduced behavior
- move dependency review action from v3 to the Node 24 v5 release
- use the supported oversized-summary handling
- keep existing severity and license policy unchanged
- add a configurable PD Raft whitelist toggle
- preserve whitelist behavior by default
- skip the IP handler when disabled
- use username consistently for audit limiter lookup
- cover distinct user id and username cleanup
- verify PD IP whitelist switch pipeline behavior
- honor ALL_GRAPHS and reject mixed-space grants
- make callback elevation explicit and restore request contexts
- remove incomplete Gremlin and legacy observer compatibility paths
- preserve HStore clear safety and the HBase Docker trigger
imbajin force-pushed the cx/bump-server-api-version branch from 45628f5 to 6851b1c Compare August 28, 2026 20:55

imbajin commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

TODO follow-ups intentionally kept out of this minimal API 0.72 closeout:

  1. Design an explicit, versioned migration/cleanup path for legacy
    <graph>_observer bindings, including deleted-and-recreated graph names.
    API 0.72 now uses explicit ALL_GRAPHS; request-time graph scans and hidden
    role mutation were removed rather than extended.
    Context: refactor(server): adapt REST API for hubble #3159 (comment)
  2. Coordinate graph deletion with concurrent data/schema mutations through one
    documented graph-wide lifecycle gate, including lock ordering and a
    concurrency regression. This spans the existing storeProvider.clear()
    boundary and should not be approximated by a local HStore-only lock.
    Context: refactor(server): adapt REST API for hubble #3159 (comment)
  3. Define exact-once schema-cache notification ownership for HStore graph clear
    without losing the direct broadcast required when
    task.sync_deletion=true. The final code preserves correctness and may emit
    a bounded duplicate under the default asynchronous-deletion path.
    Context: refactor(server): adapt REST API for hubble #3159 (comment)

These are explicit follow-ups, not silently implemented compatibility behavior
in this PR. The final head keeps the HStore metadata/cache clear bugfix, restores
the required direct remote invalidation, and leaves broader lifecycle/event
redesign for a separately scoped change.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Pull request overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated 3 comments.

imbajin requested a balanced review from Copilot August 29, 2026 05:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

imbajin commented Aug 29, 2026

Copy link
Copy Markdown
Member Author

Final dependency-review rerun note: attempt 3 completed with the same stale all-dependencies-added snapshot and failed on existing vulnerable/low-score packages. The PR changes no dependency coordinates; its only pom.xml delta changes the API JAR Implementation-Version from 0.71.0.0 to 0.72.0.0. The repository-specific dependency-check job passes, all build/storage/Docker jobs pass, and Codecov patch/project pass. This dependency-review result is classified as a base-snapshot failure rather than a current-head dependency regression; no product or dependency suppression change is made.

imbajin force-pushed the cx/bump-server-api-version branch from 4034d54 to 6851b1c Compare August 29, 2026 08:52
- identify the GraphSpace default-role change as PR-3159
- keep the existing API 0.72 contract description
- leave runtime version behavior unchanged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Changes of API size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants


Back | FazBrowse Home | New Git URL