wiola

Release 0.1.1

Wiola: a novel decoder-only Small Language Model with Spiral Rotary Positional Encoding, Gated Cross-Layer Attention, Adaptive Token Merging, Dual-Stream Feed-Forward, and WiolaRMSNorm.

Homepage PyPI

Keywords
small-language-model, transformer, language-model, spiral-rotary, attention, pytorch, huggingface
License
Apache-2.0
Install
pip install wiola==0.1.1

Stats

Dependencies
7
Dependent packages
0
Dependent repositories
0
Total releases
2
Latest release
First release
SourceRank
4

Releases

0.1.1
0.1.0

Login to resync this project