In a big step toward building artificial intelligence that constantly learns and improves itself, Google researchers said they have developed a new machine learning model with a self-modifying architecture. The new model is called HOPE, and is said to be better at managing long context memory than current AI models.
It’s meant to serve as a proof-of-concept for a new approach known as “nested learning” pioneered by Google researchers, where a single model is treated as “a system of interconnected, multi-level learning problems that are optimized simultaneously” rather than a single continuous process, the search giant said in a blog post on Saturday, November 8.
Google said the new concept of “interleaved learning” could help solve limitations found in modern large language models (LLMs) such as continuous learning, which is a crucial stepping stone on the path to artificial general intelligence (AGI) or human-like intelligence.
Last month, Andrei Karpathy, a widely respected AI/ML research scientist who previously worked at Google DeepMind, said that AGI is still a decade away, and that’s because no one has been able to develop an AI system that learns continuously — at least not yet. “They don’t have continuous learning. You can’t just tell them something and they’ll remember it. They lack cognition and it doesn’t work. It would take about a decade to solve all these problems.” Karpathy said in an appearance on the podcast.
“We believe the interleaved learning model provides a strong foundation for bridging the gap between the limited and forgetful nature of current LLMs and the remarkable continuous learning capabilities of the human mind,” Google said. The researchers’ findings were published in a paper titled “Interleaved Learning: The Illusion of Deep Learning Architectures” at NeurIPS 2025.
What is continuous learning? Why is it a challenge?
AI-powered MBA (LLM) chatbots are currently able to write sonnets and generate code within seconds. However, they do not yet possess the primitive ability to learn from experience.
Unlike the human mind, which is constantly learning and improving, today’s LLM holders cannot acquire new knowledge or skills without forgetting what they already know. This deficit is referred to as “Catastrophic forgetting” (CF).
Story continues below this ad
For many years, researchers have been looking to address cystic fibrosis by making modifications to the model structure or coming up with better optimization techniques. However, Google researchers argue that the structure of the model and the rules used to train it (i.e. the optimization algorithm) are essentially the same concepts.
“By recognizing this inherent architecture, interleaved learning provides a new, previously unseen dimension to designing more capable AI, allowing us to build learning components with deeper computational depth, ultimately helping to solve problems like catastrophic forgetting,” the researchers wrote.
What is interleaved learning?
According to the researchers, the concept of nested learning views a complex machine learning model as “a set of coherent and interconnected optimization problems nested within each other or running in parallel.” “Each of these internal problems has its own context — a distinct set of information that you are trying to learn from,” they added.
By relying on these principles, developers will be able to build MBA learning components with deeper computational depth, Google said. “The resulting models, such as the Hope architecture, show that a principled approach to unifying these elements can lead to more expressive, capable, and efficient learning algorithms,” she added.
Story continues below this ad
The proof-of-concept model, HOPE, showed less confusion and higher accuracy compared to state-of-the-art LLMs when tested on a variety of commonly used general language paradigms and logical reasoning tasks, according to the company.
(tags for translation) Google HOPE model
