OpenAI has acquired technology from Rockset that will enable it to create new products, real-time data analysis, and recommendation systems, potentially signaling a new phase for OpenAI that could change the face of search marketing in the near future.
What is Rockset and why is it important?
Rockset calls its technology Hybrid Search, a multi-faceted approach to search that brings together vector search, text search, and metadata filtering to retrieve documents that can power the generative process of the RAG system. RAG is a combination of search and generative AI with the goal of creating more factual and contextually relevant results. It's the technology that plays a role in BING's AI search and Google's AI brief.
As Rockset states in their research paper on the Rockset Hybrid Search Architecture:
“Vector search is all about becoming hybrid search to deliver the most relevant real-time application experience. Hybrid search involves the integration of vector search with text search.
You can perform search and metadata filtering all in one query. Hybrid search is used in search, recommendation, and search advancement generation (RAG) applications.…Rockset is designed and optimized to ingest data in real time, index a variety of data types, and run search and ranking algorithms.”
What's important about Rockset's hybrid search is that it can index and use multiple data types (vector, text, geospatial data for objects and events), including using real-time data. This powerful flexibility allows the technology to interact with many different kinds of data that can be used in internal and consumer-facing applications related to contextually relevant product recommendations, customer segmentation and analysis for targeted marketing campaigns, personalization, personalized content aggregation, location-based recommendations (restaurants, services, etc.), and applications that increase user engagement (Rockset cites numerous case studies on how the technology is being used).
As OpenAI explains in their announcement:
“AI has the opportunity to transform how people and organizations leverage their own data, which is why we acquired Rockset, the leading real-time analytics database that offers world-class data indexing and query capabilities.”
Rockset enables users, developers and businesses to better leverage their own data and access real-time information as they use AI products to build more intelligent applications.
…Rockset's infrastructure enables companies to transform data into actionable intelligence. We are excited to bring these benefits to our customers…”
OpenAI's announcement also explained that it plans to integrate Rockset's technology into its own search infrastructure.
For now, we know the transformative power and potential of hybrid search, but OpenAI has currently only provided general ideas about how this translates into APIs and products that businesses and individuals can build and use.
The official announcement of the acquisition from Rockset, written by one of the co-founders, offered this clue:
“We are excited to join the OpenAI team and bring our technology and expertise to building safe and beneficial AGI.
“…Advanced search infrastructure like Rockset makes AI apps more powerful and useful. This acquisition will build on what we've been building over the years to make AI available to everyone in a safe and beneficial way.”
“Rockset will become part of OpenAI and power the search infrastructure that underpins OpenAI's product suite. We will help OpenAI solve the tough database problems that AI apps face at scale.”
What exactly does acquisition mean?
Duane Forrester (LinkedIn profile), formerly of Bing Search and Yext, shares his thoughts:
“Sam Altman has stated publicly on several occasions that he is not going after Google. I get the impression that he is not keen on being seen as a search engine. Rather, he seems to want to redefine what the term search engine means – reshape the category and in that way overtake Google. And Rockset could be a helpful part of that approach.”
Additionally, Apple is releasing an updated Siri this fall, which will bring ChatGPT into the mainstream among consumers, and it will be very easy to see query initiation moving away from the traditional search engine box. It started with TikTok/social and is now moving towards an AI assistant.”
Another approach that could impact SEO is that OpenAI could create products based on its APIs that companies could use to power their internal and consumer-facing applications. In this approach, OpenAI would provide the infrastructure (as it does today with ChatGPT and the foundational model) and enable the world to innovate everywhere with OpenAI at the center as the infrastructure (as it does today).
I asked Duane about that scenario, and he agreed, but remained open to a broader range of possibilities.
“Of course it's possible. In working on this subject, I had to step up a level, or change the way I think conceptually. Search teethat its core is information search. So if we go down the path of IR, how can we reinvent “search” using today's systems and structures that redefine how we search for information?
Keep in mind that this is also a description of the next generation of advanced site search. This system could literally take over site search for a wide range of medium to large businesses. It is as advanced as most advanced site search systems are today. It could become even more advanced when released. So ultimately, this could be the harbinger of a change towards consumer search (IR) and site search based systems.
It expands from there, as they allude to, to an app. So, that's where I see them heading.”
Deedy Das of Menlo Ventures (Poshmark, Roku, Uber) speculated on Twitter about what changes the acquisition could bring to OpenAI:
“This is speculation, but I believe Rockset will power any enterprise search services that compete with Glean, as well as consumer search services that compete with Perplexity/Google. Given Rockset's permission capabilities, I think the former is more likely than the latter.”
Here's what people on Twitter are saying about what this means for the future of AI:
“I don’t see OpenAI entering the enterprise search race, it’s too hard and it’s an area that Microsoft and Google are best positioned to tackle.
This is an effort to accelerate agent action and develop deep expertise within the enterprise. Some might argue it's the same as enterprise search, but taking an agent-first approach is very much aligned with OpenAI's mission.”
Important developments for OpenAI and beyond
The acquisition of Rockset could be the basis for one of the most significant changes in how companies use and deploy AI and, like many other technological developments, could impact the business of digital marketing.
See how Rockset customers are powering recommendation systems, real-time personalization, real-time analytics, and other applications.
Featured Case Studies
Read Rockset's official announcement:
OpenAI Acquires Rockset
Read the official announcement from OpenAI:
OpenAI Acquires Rockset
Strengthening search infrastructure to make AI more useful
Read Rockset's original research paper:
Rockset Hybrid Search Architecture (PDF)
Featured image: Shutterstock/Iconic Bestiary
