4 Commits

Include renames

Author	SHA1	Message	Date
Luxferre	18ebe25d34	tok/mkf upd	2026-10-15 23:13:11 +03:00
Luxferre	561d90629f	Switch embedded tokenizer to Tencent Hy3 120k ByteLevel BPE	2026-10-14 14:15:28 +03:00
Luxferre	80ad504d16	recent changes	2026-10-14 00:00:02 +03:00
Luxferre	8a0c1d0ec0	Tokenizer implemented	2026-10-13 20:37:36 +03:00