-
Notifications
You must be signed in to change notification settings - Fork 597
Allow tables to be created without providing a URL #12132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR allows tables to be created without requiring a URL by making the url parameter optional in create_table_entry. When no URL is provided, the OSS server automatically creates a Lance table in a temporary directory ($TMPDIR/rerun-data-<xxx>/lance-<some-tuid>) that is cleaned up when the process exits.
Key changes:
- Made the
urlparameter optional (Option<T>) across Python and Rust APIs - Added temporary directory management in
RerunCloudHandlerSettingsfor auto-generated table storage - Updated all existing test code to explicitly wrap
provider_detailsinSome()
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| rerun_py/tests/e2e_redap_tests/conftest.py | Updated create_table signature to accept optional URL |
| rerun_py/src/catalog/connection_handle.rs | Changed url parameter from reference to Option<Url> |
| rerun_py/src/catalog/catalog_client.rs | Added optional URL parsing with map and transpose |
| rerun_py/rerun_sdk/rerun/catalog/_catalog_client.py | Updated Python type hint to accept optional URL |
| rerun_py/rerun_bindings/rerun_bindings.pyi | Updated type stub for optional URL parameter |
| crates/store/re_server/src/rerun_cloud.rs | Added temporary directory creation and auto-generation of table paths when URL is not provided |
| crates/store/re_redap_tests/src/tests/update_entry.rs | Wrapped provider_details in Some() for test compatibility |
| crates/store/re_redap_tests/src/tests/create_table.rs | Wrapped provider_details in Some() for test compatibility |
| crates/store/re_redap_tests/src/tests/create_dataset.rs | Wrapped provider_details in Some() for test compatibility |
| crates/store/re_redap_client/src/connection_client.rs | Changed to create optional provider_details based on URL presence |
| crates/store/re_protos/src/v1alpha1/rerun.cloud.v1alpha1.rs | Fixed typo in comment ("one one" to "one") |
| crates/store/re_protos/src/v1alpha1/rerun.cloud.v1alpha1.ext.rs | Made provider_details optional in request/response conversion logic |
| crates/store/re_protos/proto/rerun/v1alpha1/cloud.proto | Made provider_details field optional in protobuf definition and fixed comment typo |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Web viewer built successfully.
View image diff on kitdiff. Note: This comment is updated whenever you push a commit. |
# Conflicts: # rerun_py/rerun_bindings/rerun_bindings.pyi
1e0e93e to
5aef93f
Compare
abey79
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me.
The big missing change is that the e2e_redap_tests should be updated to not use an explicit table path for most tests. This should allow us to remove the creates_table marker. There may be just 1-2 tests to actually verify that passing something else than None works, but these can be local_only (and maybe a version of it that is cloud_only). Still ✅ since that could happen in a separate PR.
| let details = if let Some(details) = request.provider_details { | ||
| details | ||
| } else { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: this looks like a `unwrap_or_else(|| {})
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed. Now I used this approach because of the url::Url::from_directory_path(...)? in the else branch: exiting the enclosing function when an error happens in the closure would be more convoluted than this if/else…
Co-authored-by: Antoine Beyeler <49431240+abey79@users.noreply.github.com>
Co-authored-by: Antoine Beyeler <49431240+abey79@users.noreply.github.com>
Fixes RR-3012
The
urlparameter increate_table_entryis now optional. When it'sNone, the OSS server will create the Lance table in a temporary directory that will be deleted when the process exists.The path used is
$TMPDIR/rerun-data-<xxx>/lance-<some-tuid>, e.g.$TMPDIR/rerun-data-jYteWy/lance-187E63126233FA3A3f7ce35ea997f9c8