- PoC implementation of STT eval for below providers along with Word Error Rate and Lenient Word Error Rate for the below providers.
- Gemini-2.5-pro
- Gemin-2.5-flash
- Gemini-3
- Chirp V3 (Google-STT)
- OpenAI - Whisper -1
- OpenAI - OpenAI-4o-transcribe
- AI4Bharat
- Bhasini
Evaluation metrics
- WER
- Lenient WER
- Average scores
- Count of errors insertion, substitution, deletion.
- Summary scores