SantaCoder
1.1B parameter models for code generation
December 29th, 2024
About SantaCoder
SantaCoder is a tool that provides a series of 1.1B parameter models trained on the Python, Java, and JavaScript subset of The Stack (v1.1). These models are designed for code generation tasks, offering advanced natural language processing capabilities for code-related tasks. The main model uses Multi Query Attention with a context window of 2048 tokens and was trained using near-deduplication and comment-to-code ratio as filtering criteria. It is a powerful tool for automated programming, code ...
Key Features
- 1.1B parameter models trained on Python
- Java
- and JavaScript.
- Trained on The Stack dataset (v1.1).
- Multi Query Attention for advanced natural language processing capabilities.
- Context window of 2048 tokens.
- Near-deduplication and comment-to-code ratio filtering.
Use Cases
- Automated programming.
- Code completion.
- Code translation.
Loading reviews...
