OpenAI, the company who has stirred up the world with its ChatGPT, is now training its new AI model Orion. However, the new model has failed to meet the expectations during its initial training despite being able to significantly outperform previous versions like ChatGPT.
OpenAI’s Orion Struggles with Coding
Despite being a step forward than earlier predecessor, Orion struggled, particularly with coding tasks. This issue also goes on to reflects broader challenges in the AI industry, which includes the likes of companies like Google and Anthropic, which are encountering diminishing returns on their increasingly expensive AI models.
A key obstacle is difficulty of sourcing high-quality training data, with Orion’s coding performance limited by insufficient data. In addition to this, from economic point of view, even these incremental improvements do not justify the enormous costs of building and running new models.
OpenAI has been refining Orion through a post-training process while incorporating human feedback and adjusting the model’s responses, but the release isn’t expected until early 2025.
Challenging the AI Assumption: Experts Question the Path to AGI Amid Rising Costs and Data Constraints
There is an assumption in Silicon Valley that larger models as well as more computing power would lead to inevitable breakthroughs and these issue challenge this assumption. Many experts are now questioning the feasibility of achieving artificial general intelligence (AGI) in the near future.
Dario Amodei of Anthropic acknowledged that challenges such as data limitations could derail progress, although he remains optimistic about overcoming these hurdles.
Tech companies are increasingly relying on synthetic data and specialized experts to improve training, but these efforts are slow and costly. With increasing complexity of AI models, the cost of these system increases which leads to difficult decisions about whether to continue investing in newer models or improve older ones.
OpenAI, for example, has focused on incremental updates to ChatGPT rather than introducing a major new release like GPT-5. Ultimately, these challenges raise doubts about the rapid pace of AI advancement and the sustainability of the industry’s current trajectory.
