tokenizer-eval/tokenizer-data
Viewer • Updated • 24.8k • 28
Tokenizer Eval
non-profit
AI & ML interests
Tokenizers
Recent Activity
View all activity
christopher
updated 2
datasets 27 days ago
benjamin
authored a
paper 3 months ago
benjamin
submitted a
paper to Daily Papers 3 months ago
christopher
authored a
paper 4 months ago
sanderland
authored a
paper 5 months ago
christopher
authored a
paper 5 months ago
christopher
posted an update 6 months ago
sanderland
authored a
paper 10 months ago
sanderland
authored a
paper 11 months ago
sanderland
authored a
paper 12 months ago
benjamin
authored 2
papers 12 months ago
christopher
posted an update over 1 year ago
Post
2111
The folks at Foursquare released a dataset of 104.5 million places of interest ( foursquare/fsq-os-places) and here's all of them on a plot
- 4 replies
christopher
posted an update over 1 year ago
Post
2452
The Lichess database of games, puzzles, and engine evaluations is now on the Hub: 
Lichess
Billions of chess data points to download, query, and stream and we're excited to see what you'll build with it! ♟️ 🤗
- https://huggingface.co/collections/Lichess/positions-datasets-66f50837db5cd3287d60d489
- https://huggingface.co/collections/Lichess/games-datasets-66f508df78f4b43e1bb2d353
Billions of chess data points to download, query, and stream and we're excited to see what you'll build with it! ♟️ 🤗
- https://huggingface.co/collections/Lichess/positions-datasets-66f50837db5cd3287d60d489
- https://huggingface.co/collections/Lichess/games-datasets-66f508df78f4b43e1bb2d353
christopher
posted an update over 1 year ago
Post
1347
4 million chess puzzles
benjamin
authored a
paper over 1 year ago
benjamin
authored 3
papers almost 2 years ago
Zero-Shot Tokenizer Transfer
Paper • 2405.07883 • Published • 5
Where's the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence Segmentation
Paper • 2305.18893 • Published • 2
CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models
Paper • 2305.14214 • Published