Skip to content

Conversation

@JuanFKurucz
Copy link

This PR is work-in-progress and intentionally opened as a draft as an idea/necessity on Issue #250 “ONNX Runtime minimal dependencies.”


TL;DR

Regarding #250, this change lets users install ONNX-only builds of llm-guard while preserving full functionality for PyTorch users.

  1. Optional-extras matrix (onnxruntime, onnxruntime-gpu, pytorch, extended, full).
  2. Lazy imports for heavy libs (torch, transformers, …).
  3. Smarter device/provider detection that doesn’t require PyTorch.
  4. No breaking API changes; legacy imports still work.

✨ Key changes

Area Details
Dependencies pyproject.toml splits heavy libs into extras; base wheel is slim.
Scanners New get_scanner_class() + refactored get_scanner_by_name() use lazy imports.
util.device() No longer hard-imports torch; falls back to env/OS inspection when absent.
Model helpers ONNX provider chosen via onnxruntime.get_available_providers().
Back-compat Public re-exports unchanged.

🔍 Motivation

  • Smaller memory footprint
  • Faster cold-starts
  • User choice — install only what you need.

🛠️ Install matrix

Scenario Command
ONNX / CPU pip install llm_guard[onnxruntime]
ONNX / GPU pip install llm_guard[onnxruntime-gpu]
Full PyTorch pip install llm_guard[pytorch]
Everything pip install llm_guard[full]

⚠️ Breaking changes?

By default now it will install the minimal dependencies


📌 Next steps before “Ready for Review”

  • Revisit if this breaks something, I have only tested with CompetitorsCheck

References


Thanks for reviewing! Feedback is welcome before moving this draft to a full PR.

@JuanFKurucz JuanFKurucz requested a review from asofter as a code owner June 3, 2025 18:01
@JuanFKurucz JuanFKurucz marked this pull request as draft June 3, 2025 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant