Feedback is welcome, especially on useful ablations: digit reversal, place_ids, role_ids, or whether auxiliary features should extend beyond arithmetic spans into ordinary text.
Maksymilian PRO
ucr-max
AI & ML interests
None yet
Recent Activity
liked a model about 14 hours ago
openbmb/MiniCPM5-1B new activity 4 days ago
UniversalComputingResearch/Atom2.7m:Kudos on Atom 2.7M updated a Space 5 days ago
UniversalComputingResearch/READMEOrganizations
replied to their post 5 days ago
posted an update 5 days ago
Post
97
Published under Universal Computing Research: Atom2.7m, a 2.74M-parameter causal LM experiment with arithmetic-aware numeric representation.
It keeps regular BPE-style text continuation, while adding structure for numeric spans: digit ordering, place_ids, and role_ids.
Atom2.7m reaches 69.24% accuracy on ArithMark2.0.
The question behind the project is narrow: how much of small-model arithmetic performance depends on representation, rather than scale alone?
Model: UniversalComputingResearch/Atom2.7m
Article: https://huggingface.co/blog/ucr-max/atom2-7m-arithmetic-representation
Demo: UniversalComputingResearch/Atom2.7-Demo
It keeps regular BPE-style text continuation, while adding structure for numeric spans: digit ordering, place_ids, and role_ids.
Atom2.7m reaches 69.24% accuracy on ArithMark2.0.
The question behind the project is narrow: how much of small-model arithmetic performance depends on representation, rather than scale alone?
Model: UniversalComputingResearch/Atom2.7m
Article: https://huggingface.co/blog/ucr-max/atom2-7m-arithmetic-representation
Demo: UniversalComputingResearch/Atom2.7-Demo
- 1 reply