feat: 2.10 nexus support

[daquinteroflex]

Seems to be working fine.

(tidy3d-A) flexdaemon@dxps:~$ tidy3d configure --apikey xxx  --nexus-url http://localhost
Nexus configuration saved successfully to profile: profiles/nexus.toml
  API endpoint: http://localhost/tidy3d-api
  Website endpoint: http://localhost/tidy3d
  S3 endpoint: http://localhost:9000

Default profile set to 'nexus'. Tidy3D will now use these endpoints by default.
To switch back to production, use: config.set_default_profile(None) in Python
(tidy3d-A) flexdaemon@dxps:~$ cat ~/.config/tidy3d/config.toml 
default_profile = "nexus"

[web]
apikey = "xxx" # Tidy3D API key.
(tidy3d-A) flexdaemon@dxps:~$ ls ~/.config/tidy3d
config.toml  profiles
(tidy3d-A) flexdaemon@dxps:~$ ls ~/.config/tidy3d/profiles/
nexus.toml
(tidy3d-A) flexdaemon@dxps:~$ cat .config/tidy3d/profiles/nexus.toml 
[web]
website_endpoint = "http://localhost/tidy3d" # Tidy3D website URL.
api_endpoint = "http://localhost/tidy3d-api" # Tidy3D API base URL.

[web.env_vars]
AWS_ENDPOINT_URL_S3 = "http://localhost:9000"

 default_profile                                                                              │
│ │   └── 'nexus'                                                                                  │
│ ├── local_cache                                                                                  │
│ │   └── {                                                                                        │
│ │           'enabled': False,                                                                    │
│ │           'directory': '/home/flexdaemon/.cache/tidy3d/simulations',                           │
│ │           'max_size_gb': 10.0,                                                                 │
│ │           'max_entries': 0                                                                     │
│ │       }                                                                                        │
│ ├── logging                                                                                      │
│ │   └── {                                                                                        │
│ │           'level': 'WARNING',                                                                  │
│ │           'suppression': True                                                                  │
│ │       }                                                                                        │
│ ├── microwave                                                                                    │
│ │   └── {                                                                                        │
│ │           'suppress_rf_license_warning': False                                                 │
│ │       }                                                                                        │
│ ├── plugins                                                                                      │
│ │   └── {}                                                                                       │
│ ├── simulation                                                                                   │
│ │   └── {                                                                                        │
│ │           'use_local_subpixel': None                                                           │
│ │       }                                                                                        │
│ └── web                                                                                          │
│     └── {                                                                                        │
│             'apikey': 'xxx',                        │
│             'ssl_verify': False,                                                                 │
│             'enable_caching': False,                                                             │
│             'api_endpoint': 'http://localhost/tidy3d-api',                                       │
│             'website_endpoint': 'http://localhost/tidy3d',                                       │
│             's3_region': 'us-east-1',                                                            │
│             'timeout': 120,                                                                      │
│             'ssl_version': None,                                                                 │
│             'env_vars': {                                                                        │
│                 'AWS_ENDPOINT_URL_S3': 'http://localhost:9000'                                   │
│             }                                                                                    │
│         }                          

Greptile Overview

Greptile Summary

This PR adds Nexus on-premises server support for Tidy3D, enabling enterprise customers to connect to custom deployments.

Key changes:

• Extended CLI configure command with --nexus-url, --api-endpoint, --website-endpoint, --s3-region, --s3-endpoint, and SSL/caching flags
• Added default_profile persistence to config.toml so Nexus configuration persists across sessions
• Extended legacy config migration to handle Nexus-related fields (web_api_endpoint, s3_endpoint, etc.)
• Updated S3 client to use custom endpoint URL from web.env_vars for MinIO/S3-compatible storage
• Added comprehensive documentation and tests

Architecture:

• Base config (config.toml) stores API key and default_profile setting
• Profile-specific settings are stored in profiles/nexus.toml
• Profile resolution priority: explicit parameter → environment variables → default_profile → "default"

Confidence Score: 4/5

• This PR is safe to merge with one minor edge case to address in the S3 endpoint URL derivation
• Score reflects well-tested feature with comprehensive documentation, but there's a minor URL parsing edge case in the CLI that could produce invalid S3 endpoints when nexus-url contains trailing slashes or ports
• tidy3d/web/cli/app.py - S3 endpoint derivation logic needs to handle edge cases

Important Files Changed

File Analysis

Filename	Score	Overview
tidy3d/web/cli/app.py	3/5	Extended configure command for Nexus support; S3 endpoint derivation has edge case issues with URLs containing ports or trailing slashes
tidy3d/config/manager.py	5/5	Added set_default_profile/get_default_profile methods and updated profile resolution logic; well-implemented with proper validation
tidy3d/config/loader.py	5/5	Added default profile persistence and auto-migration logic; clean implementation with proper error handling
tidy3d/config/legacy.py	5/5	Extended legacy config migration to support Nexus fields; correctly maps old flat keys to new nested structure
tidy3d/web/core/s3utils.py	5/5	Updated get_client() to configure custom S3 endpoint from web.env_vars; clean and correct implementation
tests/config/test_nexus_migration.py	5/5	Comprehensive tests for Nexus config migration covering legacy config parsing, auto-migration, and profile handling
docs/configuration/nexus.rst	5/5	Comprehensive documentation for Nexus environment configuration with examples for CLI, Python, and troubleshooting

Sequence Diagram

sequenceDiagram
    participant User
    participant CLI as tidy3d CLI
    participant Config as ConfigManager
    participant Loader as ConfigLoader
    participant Profile as profiles/nexus.toml
    participant Base as config.toml
    participant S3Client as boto3 S3 Client

    User->>CLI: tidy3d configure --nexus-url
    CLI->>CLI: Derive endpoints from nexus_url
    CLI->>Config: update_section(web, apikey)
    Config->>Base: Save API key
    CLI->>Config: switch_profile(nexus)
    CLI->>Config: update_section(web, endpoints)
    Config->>Profile: Save nexus-specific settings
    CLI->>Config: set_default_profile(nexus)
    Config->>Loader: set_default_profile(nexus)
    Loader->>Base: Write default_profile
    
    Note over User,S3Client: On next session
    
    User->>Config: import tidy3d
    Config->>Loader: init
    Loader->>Base: Read default_profile
    Loader-->>Config: profile is nexus
    Config->>Profile: Load nexus settings
    
    User->>S3Client: Upload or Download
    S3Client->>Config: Read web.env_vars
    S3Client->>S3Client: Configure endpoint_url from env_vars

Loading

[greptile-apps]

_{11 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[frederikschubertflex]

To switch back to production, use: config.set_default_profile(None) in Python

Should we also provide a way to switch back with the CLI?

[frederikschubertflex]

I find it a bit strange that we configure the nexus backend and S3 storage differently (arg vs. env var). Just saw that this was only in your example.

[github-actions]

Diff Coverage

Diff: origin/develop...HEAD, staged and unstaged changes

• tidy3d/config/legacy.py (100%)
• tidy3d/config/loader.py (90.3%): Missing lines 69-70,74
• tidy3d/config/manager.py (100%)
• tidy3d/web/cli/app.py (88.3%): Missing lines 106,221,240-241,251-252,278-279,296-298
• tidy3d/web/core/s3utils.py (100%)

Summary

• Total: 165 lines
• Missing: 14 lines
• Coverage: 91%

tidy3d/config/loader.py

Lines 65-78

  65                 )
  66 
  67                 # Re-read the newly created config
  68                 return self._read_toml(config_path)
! 69             except Exception as exc:
! 70                 log.warning(
  71                     f"Failed to auto-migrate legacy configuration: {exc}. "
  72                     "Using legacy data without migration."
  73                 )
! 74                 return legacy
  75 
  76         if legacy:
  77             return legacy
  78         return {}

---

tidy3d/web/cli/app.py

Lines 102-110

  102         Whether to verify SSL certificates
  103     enable_caching : bool
  104         Whether to enable result caching
  105     """
! 106     configure_fn(
  107         apikey,
  108         nexus_url,
  109         api_endpoint,
  110         website_endpoint,

---

Lines 217-225

  217     if website_endpoint:
  218         web_updates["website_endpoint"] = website_endpoint
  219 
  220     if s3_region is not None:
! 221         web_updates["s3_region"] = s3_region
  222 
  223     if ssl_verify is not None:
  224         web_updates["ssl_verify"] = ssl_verify

---

Lines 236-245

  236     if apikey:
  237 
  238         def auth(req: requests.Request) -> requests.Request:
  239             """Enrich auth information to request."""
! 240             req.headers[HEADER_APIKEY] = apikey
! 241             return req
  242 
  243         # Determine validation endpoint
  244         validation_endpoint = api_endpoint if api_endpoint else str(config.web.api_endpoint)
  245         validation_ssl = ssl_verify if ssl_verify is not None else config.web.ssl_verify

---

Lines 247-256

  247         target_url = f"{validation_endpoint.rstrip('/')}/apikey"
  248 
  249         try:
  250             resp = requests.get(target_url, auth=auth, verify=validation_ssl)
! 251         except (requests.exceptions.SSLError, ssl.SSLError):
! 252             resp = requests.get(target_url, auth=auth, verify=False)
  253 
  254         if resp.status_code != 200:
  255             click.echo(
  256                 f"Error: API key validation failed against endpoint: {validation_endpoint}\n"

---

Lines 274-283

  274                 config.update_section("web", **nexus_updates)
  275                 config.save()
  276         else:
  277             # Non-nexus config: save everything to base config
! 278             config.update_section("web", **web_updates)
! 279             config.save()
  280 
  281         if has_nexus_config:
  282             # Set nexus as the default profile when nexus is configured
  283             config.set_default_profile("nexus")

---

Lines 292-302

  292                 "\nDefault profile set to 'nexus'. Tidy3D will now use these endpoints by default."
  293             )
  294             click.echo("To switch back to production, run: tidy3d configure --restore-defaults")
  295         else:
! 296             click.echo("Configuration saved successfully.")
! 297     elif not apikey and not has_nexus_config:
! 298         click.echo("No configuration changes to apply.")
  299 
  300 
  301 @click.command()
  302 @click.argument("lsf_file")

---

[yaugenst-flex]

Thanks @daquinteroflex overall LGTM

[yaugenst-flex]

Let's add a message in e.g. ConfigManager.__init__() after profile resolution with something like

if self._profile != "default":
  log.info(f"Using configuration profile: '{self._profile}'", log_once=True)

to make sure users are aware if they're using a non-default profile.

[daquinteroflex]

oh I forgot, thanks