InterwebAlchemy
/

PGNTokenizer

Model card Files Files and versions

Commit History

Revert back to NFD tokenizer

3341d84
verified

InterwebAlchemy commited on Feb 11, 2025

Remove normalizer

767e594
verified

InterwebAlchemy commited on Feb 11, 2025

Switch from NFD -> NFC

0dcea10
verified

InterwebAlchemy commited on Feb 11, 2025

Update README.md

6d5f592
verified

InterwebAlchemy commited on Jan 28, 2025

Upload tokenizer

57dcf89
verified

InterwebAlchemy commited on Jan 28, 2025

initial commit

74fae32
verified

InterwebAlchemy commited on Jan 25, 2025