Databricks bolsters Mosaic AI with tools to build and evaluate compound AI systems

Databricks is raising the bar on enterprise gen AI developer tools. Today, at its annual data and AI conference, the Ali Ghodsi-led company announced several new enhancements for its Mosaic AI platform, aimed at helping enterprises deploy large language model (LLM)-powered applications.

While Databricks has been providing enterprises tooling to build AI applications for quite some time, the Mosaic AI platform, which originated from the company’s $1.3 billion acquisition of MosaicML, has accelerated the efforts on the gen AI side.

The latest capabilities bolster the offering with a focus on three key areas: 1. development of compound AI systems, 2. their evaluation across different metrics, and 3. the governance of the entire pipeline.

The move creates a robust end-to-end ecosystem to help enterprises build reliable gen AI apps from their data. It also strengthens the company’s offering against Snowflake, which has been moving in the same direction ever since Sridhar Ramaswamy took over as the CEO.

Just recently, Snowflake even released its own enterprise-grade open LLM “Arctic” to take on Databricks’ DBRX. 

What’s coming to Databricks Mosaic AI?

Organizations bullish on generative AI are racing to make the most of the novel technology by creating applications leveraging their internal data assets with powerful AI models.

The approach works, but in many cases, teams find it difficult to get the desired return-on-investment from large models. Essentially, the app fails to provide high-quality outputs while sticking to the expected budgets and privacy guardrails.

To solve this, organizations have shifted to building retrieval augmented generation (RAG)-based compound AI systems that leverage multiple components, including various small models, retrievers, vector databases and tools for evaluation, monitoring, security and governance. Databricks has been upgrading Mosaic AI to enable the creation of these systems. 

A few months ago, the company announced Vector AI search as a serverless vector database integrated into its data platform. Now, it is adding Mosaic AI Model Training and Agent Framework into the mix.

The former lets users use the Databricks API or UI to finetune small, open-source foundation models, giving them new knowledge to handle specific domains or tasks while being cost-efficient at the same time.

Meanwhile, the latter, integrated with Mosaic AI Vector Search and Model Serving, powers high-quality RAG apps using those fine-tuned models.

“First, the Agent Framework will make it easy to measure/evaluate the quality of the app through Agent Evaluation,” Joel Minnick, VP of Product Marketing at Databricks, told VentureBeat. “It will have built-in proprietary AI-assisted evaluation that can automatically determine if outputs are high quality as well as an intuitive tracing UI to get feedback from human stakeholders. Then, it will make it easy to take the feedback and rapidly iterate on changes. Developers can test every hypothesis and then re-deploy their application into production with an end-to-end LLMOps workflow.”

The platform also includes an AI Tools Catalog that lets organizations govern, share, and register tools using Databricks Unity Catalog, which the company just today made open source.

These tools aid compound AI systems as functions, equipping them with new capabilities like intelligently generating and executing code, searching the web and calling APIs. Minnick noted that any Python or SQL function registered in the Unity Catalog will be supported by the Mosaic AI Tools Catalog and become available for models to use, increasing the quality of the final response.

Stronger governance with Mosaic AI Gateway

Finally, to round things up and ensure complete trust in the developed AI apps, the company is adding what it calls “Mosaic AI Gateway.”

This offering provides teams with a unified interface to query, manage, and deploy open-source or proprietary models, enabling them to switch the LLMs, without making complicated changes to the application code. 

Most importantly, the AI Gateway comes with built-in governance and monitoring capabilities. It supports usage tracking and guardrails, letting organizations track who is calling the model, and can even set up rate limits to control spending and filters for safety and personally identifiable information.

All new Mosaic AI offerings, except the AI Tools Catalog, are in public preview and expected to become generally available over the coming months. The tools catalog is currently in private preview, although Databricks has given no word on its broader release. The company also announced other notable products at the event, including Databricks AI/BI for gen AI-powered analytics, Databricks LakeFlow for data engineering, and an enterprise-centric image generation model developed in partnership with Shutterstock.

Databricks Data and AI Summit runs from June 10 to June 13, 2024.