GPT-NeoX
EleutherAI framework and 20B-class models for training large autoregressive LMs with 3D parallelism—Apache-2.0 training stack.
Why it is included
Foundational open training codebase predating many commercial stacks.
Best for
Research groups customizing distributed LLM training.
Strengths
- Training stack
- Research heritage
- Eleuther ecosystem
Limitations
- Operating costs; not an app layer
Good alternatives
Megatron-LM · DeepSpeed
Related tools
AI & Machine Learning
PyTorch
Deep learning framework with strong research-to-production paths.
AI & Machine Learning
OLMo
Allen AI fully open LLM **pipeline**: weights, training code, data mixes, and evaluation—research transparency flagship.
AI & Machine Learning
Pythia (Hugging Face)
EleutherAI’s public scaling suite: matched GPT-NeoX–architecture models from 70M–12B with public datasets for interpretability research.
AI & Machine Learning
Axolotl
YAML-configured fine-tuning for LLMs: LoRA, QLoRA, FSDP, and many architectures on top of Hugging Face trainers.
AI & Machine Learning
Unsloth
Optimized fine-tuning library claiming 2× faster LoRA/QLoRA with less VRAM via custom kernels and Hugging Face compatibility.
AI & Machine Learning
BLOOM
BigScience 176B multilingual causal LM—landmark collaborative open training effort on Jean Zay (weights under BigScience Responsible AI License).