Training a 22M parameter model from scratch to beat billion-parameter LLMs at writing FOI requests
This week I wanted to explore whether a model that is smaller than the average Android app could outperform billion-parameter LLMs on a specialised task. To test this out, I trained a 22 million pa...