Epoch AI fits a trend through the models whose context windows ranked among the ten longest on their release date, and finds that since mid-2023 the frontier context window has been getting about 30× longer every year

Epoch AI analyzed the context window sizes of large language models (LLMs), which refers to the number of tokens an LLM can ingest per query. They identified models with the ten longest context windows on their release date and fit a linear model to this data. This analysis indicates that the frontier context window has been expanding by approximately 30 times annually since mid-2023.