Add Rust implementation of SQLite vs Doublets benchmark #104
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adding CLAUDE.md with task information for AI processing. This file will be removed when the task is complete. Issue: #103
This commit introduces a complete Rust benchmark suite comparing SQLite and Doublets performance for basic CRUD operations (Create, Read, Update, Delete). Implementation details: - SQLite: Uses rusqlite with in-memory database (:memory:) - Doublets: Uses doublets-rs with volatile (in-memory) storage - Both United and Split store variants supported - Benchmarks use Criterion for statistical analysis - All benchmarks use configurable link counts via environment variables Benchmark groups: - create: Point creation performance - delete: Link deletion performance - update: Link update (source/target modification) - query_all: Full table scan - query_by_id: Primary key lookup - query_by_source: Index-based query by source - query_by_target: Index-based query by target Note: Uses nightly-2022-08-22 toolchain for doublets-rs compatibility. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
konard
changed the title
Windows CI was failing because tempfile pulled in windows-link v0.2.1 which requires rustc 1.71+, while we're using nightly-2022-08-22 (1.65). Pinned: - tempfile = 3.3.0 (older version without windows-link dep) - fastrand = 1.8.0 (compatible with older tempfile) - getrandom = 0.2.8 (compatible with older fastrand) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The winapi-util v0.1.11 was pulling in windows-sys which requires windows-link v0.2.1 that needs rustc 1.71+. Pinned older versions that use the older winapi crate instead: - winapi-util = 0.1.5 - walkdir = 2.3.2 - same-file = 1.0.6 - regex = 1.7.1 - regex-automata = 0.1.10 - regex-syntax = 0.6.28 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The --noplot option was being passed through -- which routes to the wrong binary. Removed the option since criterion will use plotters backend anyway when gnuplot isn't available. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The default criterion settings (100 samples, 5s measurement time) were causing CI timeouts (>50 minutes). Reduced to: - 10 samples - 1 second measurement time - 500ms warmup time This provides a quick validation that benchmarks work while keeping CI runtime reasonable. For detailed performance analysis, users can run with default settings locally by setting environment variables. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Full criterion benchmarks take >30 minutes even with reduced settings. Changed CI to run `cargo bench -- --test` which: - Validates benchmarks compile and execute correctly - Runs each benchmark once without statistical analysis - Completes in seconds rather than minutes Also reduced link counts to 100 for faster CI validation. For actual performance measurements, run locally: BENCHMARK_LINK_COUNT=1000 BACKGROUND_LINK_COUNT=3000 cargo bench Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The benchmark job was taking >1.5 hours due to slow crates.io index updates on fresh CI runs without cache. The unit tests on all 3 platforms (ubuntu, macos, windows) provide sufficient validation. Benchmarks can be run locally: cd rust cargo bench For detailed benchmarks with specific link counts: BENCHMARK_LINK_COUNT=1000 BACKGROUND_LINK_COUNT=3000 cargo bench Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This reverts commit 3ad5401.
Member
Author
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a complete Rust implementation of the SQLite vs Doublets benchmark as requested in #103.
What's implemented:
rusqlitewith in-memory database (:memory:)doublets-rswith volatile (in-memory) storageBenchmark groups:
Technical notes:
BENCHMARK_LINK_COUNT(default: 1000)BACKGROUND_LINK_COUNT(default: 3000)How to run:
Test plan
Closes #103
🤖 Generated with Claude Code