Recap of GPT-4 Microsoft Research claims that GPT-4 represents the first sparks of AGI and performs strikingly close to human-level performance on many tasks. GPT-4 has an 8,000 token window, which is a game-changer, and it is an early yet incomplete AGI. GPT-4 is also multimodal, supports images, and is qualitatively better at everything than GPT-3.

GPT-5 Predictions and Rumors There is an open letter calling for a six-month moratorium on building anything more powerful than GPT-4 due to safety, ethics, and regulations. GPT-5 is rumored to be already trained on 25,000 Nvidia GPUs, and Sam Altman said that GPT-4 came about from hundreds of small incremental improvements. GPT-5 predictions and rumors suggest that it could be released by the end of 2024 or early 2025, with a shorter testing and release cycle. GPT-5 could have anywhere from 64,000 to 256,000 tokens, which is roughly 42,000 words up to 170,000 words.

Window Size The window size of GPT-4 is predicted to be much larger than GPT-3, possibly taking 10 to 40 times as much compute. However, there may be diminishing returns on an algorithmic level, and it is uncertain how many tokens are actually needed for functional value.

Modality and Intelligence GPT-4 is multimodal, able to process images, and GPT-5 is predicted to be even more so, possibly including audio, video, and text. The vectors used to represent these modalities may be more abstract and human-like, unlocking new capabilities. GPT-5 is also predicted to surpass humans in most tasks, including artistic endeavors, and may lead to the replacement of human actors. The potential implications of GPT-5's intelligence and capabilities are uncertain, and it is unclear whether there will be regulation or competition in the field.