Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
I recently completed my PhD in Computer Science at Tokyo Institute of Technology (now the Institute of Science Tokyo) as an American student on the MEXT Scholarship. Simultaneously, I continued my work as a software engineer and researcher at Google.
#13 on Trending Abstract Language models process token IDs, not raw text. Converting text into those units has consequences for what models process and how much computation they use. The choice of units can also change how much processing equivalent text requires across languages. A recent survey maps how tokenizers are built, what can go wrong, and how to measure their effects in modern language models.
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
