Initial benchmark command

[drbh]

This is a work in progress PR to add a new benchmark command to the kernels cli tool. The idea is to enable a standard way to benchmark kernels

uvx \
--from git+https://github.com/huggingface/kernels.git@initial-benchmark-command \
--with torch \
--with numpy \
kernels benchmark kernels-community/activation # <- the expected command once merged

output

    Updated https://github.com/huggingface/kernels.git (daa75e4edfaccca487b9de9fb2b85b4cd052fd42)
      Built kernels @ git+https://github.com/huggingface/kernels.git@daa75e4edfaccca487b9de9fb2b85b4cd052fd42
Installed 45 packages in 81ms
Downloading kernels-community/activation@main...
Running benchmark.py...

┌────────────────┬──────────┬────────────┬────────────┬────────────┬────────────┬────────────┬──────────┬───────────┐
│ Benchmark      │ Workload │   Mean(ms) │    Std(ms) │    Min(ms) │    Max(ms) │    IQR(ms) │ Outliers │ Ref Match │
├────────────────┼──────────┼────────────┼────────────┼────────────┼────────────┼────────────┼──────────┼───────────┤
│ SiluWorkloads  │ medium   │     0.0109 │     0.0003 │     0.0105 │     0.0130 │     0.0002 │        3 │     ✓     │
│ SiluWorkloads  │ small    │     0.0051 │     0.0005 │     0.0049 │     0.0095 │     0.0002 │        3 │     ✓     │
│ SiluWorkloads2 │ medium   │     0.0298 │     0.0017 │     0.0282 │     0.0390 │     0.0010 │        5 │     ·     │
│ SiluWorkloads2 │ small    │     0.0061 │     0.0025 │     0.0055 │     0.0305 │     0.0001 │        7 │     ✓     │
└────────────────┴──────────┴────────────┴────────────┴────────────┴────────────┴────────────┴──────────┴───────────┘

Dry run - use --upload to submit results

benchmark file

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.