ONNX Runtime
Cross-platform inference accelerator for ONNX models: CPU, GPU, and mobile execution providers with graph optimizations.
Why it is included
Standard OSS runtime for shipping models across frameworks and hardware without retraining.
Best for
Edge, mobile, and server inference where a single graph bundle must run everywhere.
Strengths
- Broad EP support
- ONNX interchange
- MS + community maintenance
Limitations
- Not every op combo exports cleanly; validate graphs per target
Good alternatives
TensorRT · OpenVINO · TVM
Related tools
AI & Machine Learning
PyTorch
Deep learning framework with strong research-to-production paths.
AI & Machine Learning
TensorFlow
End-to-end platform for machine learning and deployment.
AI & Machine Learning
OpenVINO
Intel toolkit to optimize and deploy deep learning on Intel CPUs, GPUs, and NPUs with model conversion and runtime APIs.
AI & Machine Learning
faster-whisper
CTranslate2 reimplementation of Whisper for faster CPU/GPU inference with lower memory use than reference PyTorch.
AI & Machine Learning
Ollama
Local LLM runner and model library with simple CLI and API for workstation inference.
AI & Machine Learning
llama.cpp
Plain C/C++ inference for LLaMA-class models with broad community backends.