feat: add LDBC SF1 dataset test infrastructure for Issue #363 #710
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add Issue363SF1Test and Issue363SF1OptimizedOnlyTest for SF1 scale validation - Add bi_graph_schema_sf1.sql with hasCreator edge support - Fix issue_363_original.sql vertex type declaration (d:Person) - Configure 24 workers for large-scale processing - Add data conversion script for LDBC dataset Note: SF1 tests currently fail due to GeaFlow framework bug (NullPointerException at InitFetchRequest.toPhysicalSliceMeta:169)
Contributor
Author
geaflow/geaflow-dsl/geaflow-dsl-runtime/scripts/generate_ldbc_test_data.py
Show resolved
Hide resolved
Add missing Apache License 2.0 header to the Python script to ensure CI approval for Apache RAT license checks.
…opagation - Change EdgeDirection.IN to EdgeDirection.BOTH for undirected graph semantics - Replace string comparison with numeric comparison for component IDs (fixes: "10" < "8" in lexicographic order → 8 < 10 in numeric order) - Add compareComponentIds() helper method that falls back to string comparison for non-numeric IDs - Update expected test results for correct component assignments: vertices 8,9,10 should have component 8 (not "10") - Disable SF1 tests in CI (data files not in repository)
…utRef handling Fix three broken optimization rules that caused 16+ test failures: 1. IdFilterPushdownRule: Fix label validation for RexInputRef - Bug: Line 180 blindly assumed referencesTarget=true for any RexInputRef - Fix: Use FilterMatchNodeTransposeRule pattern to check if index == fieldCount - 1 2. AnchorNodePriorityRule: Preserve PathInputRef labels in swapJoinCondition - Bug: Line 244 created plain RexInputRef, losing PathInputRef.label - Fix: Check instanceof PathInputRef and use copy(newIndex) to preserve label 3. GraphJoinReorderRule: Build proper join conditions - Bug: Line 297 returned TRUE literal, losing all join constraints - Fix: Use GQLRelUtil.createPathJoinCondition() for common-label-based joins Re-enable all three rules in OptimizeRules.java after fixes.
…mpatibility Add GQLRelUtil.toRel() unwrapping across optimization and runtime components: Core HepRelVertex fixes: - GQLOptimizer.applyRulesOnChildren() for recursive optimization - StepLogicalPlanTranslator (14 visitor methods) for runtime translation - GraphMatchFieldPruneRule and ProjectFieldPruneRule for field pruning - PhysicParameterizedRelNode.isIdOnlyRequest() for parameterized queries - GraphMatch.getFilteredFields() for field extraction Match node enhancements: - EdgeMatch, VertexMatch, VirtualEdgeMatch field pruning support Additional tests: - StepLogicalPlanTranslatorHepRelVertexTest for HepRelVertex unwrapping - Issue363PushDownFilterPlanTest for filter pushdown validation These changes are essential complements to the optimization rules in commits 6399e2b..d6146ad. Without these fixes, ClassCastException occurs at runtime because Calcite's HepPlanner wraps RelNode in HepRelVertex proxies.
…n bugs - MatchIdFilterSimplifyRule: Add RexInputRef index validation to prevent misattributing ID filters from other vertices (line 123) - MatchIdFilterSimplifyRule: Extract only first ID filter from AND conditions to preserve correct semantics for contradictory conditions like a.id=1 AND a.id=2 - StepLogicalPlanTranslator: Prioritize idSet over filter parsing to ensure ID filters extracted by optimization rules are properly used at runtime - OptimizeRules: Reorder rules so MatchIdFilterSimplifyRule runs before IdFilterPushdownRule for proper ID extraction sequence - Issue363PushDownFilterPlanTest: Restore real assertions instead of silent pass - IdFilterPushdownRule: Fix import ordering for Checkstyle compliance
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
How was this PR tested?
Note: SF1 tests currently fail due to GeaFlow framework bug (NullPointerException at InitFetchRequest.toPhysicalSliceMeta:169)