On a suite of benchmark tests that Databricks designed that it says reflects real world enterprise use cases involving instruction-following, domain-specific search, report generation, list generation, and searching PDFs with complex layouts, the company’s Instructed Retriever architecture resulted in 70% better accuracy than a simple RAG method and, when used in a multi-step agentic process, delivered a 30% improvement over the same process built on RAG, while requiring 8% fewer steps on average to get to a result.
The idea of the Instructed Retriever architecture is that it turns these implied conditions into explicit search parameters. Bendersky says the breakthrough here is that Instructed Retriever knows how to turn a natural language query into one that will leverage meta data.
Databricks tested the Instructed Retriever architecture using OpenAI’s GPT-5 Nano and GPT-5.2, as well as Anthropic’s Claude-4.5 Sonnet AI models, and then also a fine-tuned small 4 billion parameter model they created specifically to handle these kind of queries, which they call InstructedRetriever-4B. They evaluated all of these against a traditional RAG architecture. Here they scored between 35% to 50% better in terms of the accuracy of the results. And the Instructed Retriever-4B scored about on par with the larger frontier models from OpenAI and Anthropic, while being cheaper to deploy.
As always with AI, having your data in the right place and formatted in the right way is the crucial first step to success. Bendersky says that Instructed Retriever should work well as long as an enterprise’s dataset has a search index that includes metadata. (Databricks also offers products to help take completely unstructured datasets and produce this meta data.)
The company says that Instructed Retriever is available today to its beta test customers using its Knowledge Assistant product in its Agent Bricks AI agent building platform and should be in wide release soon.
This is just one example of the kinds of innovations we are almost certainly going to see more of this year from all the AI agent vendors. They might just make 2026 be the real year of AI agents.
With that, here’s more AI news.
