We say: Put your AI vectors into your RDBMS …
Relational databases are where AI data belongs. Users need their vectors along with the rest of their data, in a standard database which offers performance, scalability, and all the other traditional virtues, such as ACID compliance.
This is why we are developing MariaDB Vector. Expect to see a first preview release later this month.
… but don’t take our word for it – ask Amazon!
Now, we’re not alone in advocating the above logic. That’s probably because the logic makes sense. The best articulation of the logic of “you want your Gen AI integrated in your relational database” I’ve heard is by MariaDB Foundation Board Member Sirish Chandrasekharan, General Manager of Amazon Relational Database Services. He was interviewed by The New Stack at the Open Source Summit 2024 in Seattle.
The interview is just 18 minutes long and very much worth a watch. It’s fairly self explanatory and doesn’t require the viewer to have an overly deep understanding of AI, but let me still give a bit of an overview of the lingo and concepts used.
Two key Gen AI concepts: LLM and RAG
Generative AI is the overall concept. It usually is spoken of in the context of text, pictures, and videos, but it can refer to any data. The word “generative” would imply the generation of new data, but in the context of vector databases, it’s very much about retrieval, about finding data. The AI part here enables search on fluffier criteria, requiring characteristics that resemble intelligence.
LLM, Large-Language Models, are the place where knowledge is “intelligently” encoded. The LLMs are not themselves a form of relational databases, nor do I believe that they ever will be. This is because they form a static snapshot created (“trained”) at a certain point in time, and they aren’t updated until the next snapshot. They contain the wiring of everything about “how the world works”, and even casual users of ChatGPT, Claude or other models have a fairly good grasp of what the LLM “knows”.
RAG, Retrieval-Augmented Generation, is a less well-known term but soon to be all the rage. One could say this is the dynamic part that enables updates to the overall LLM, based on your own data. LLMs are static and so enormous that few entities in the world are large enough to create (“train”) their own ones. Using RAG, users of all sizes can create (“vectorise” or “embed”) representations of their own data, using the intelligence of the chosen LLM to interpret the data.
Side note: If you want to avoid corrections by your know-it-all AI colleagues, don’t talk about “training” models on your own data, but instead say that you “embed” or “vectorise” your own data. The word “training” is usually confined to the building of models from scratch, which is what the LLM developers do when creating the LLM.
Vectors? Vector databases? Vector indexes?
Vector, or multi-dimensional vector, is an overloaded word with a special meaning in AI. It does resemble a mathematical vector, which some of us know from physics or engineering, but those vectors are usually confined to two or three dimensions. AI vectors have lots and lots of dimensions, and constitute a numeric representation of knowledge specific to the model itself.
A vector database is a database that can store, index, and search vectors. Yes, there are databases specialised in vectors only, but that was my initial argument above – it makes much more sense to store them in a normal relational database. In MariaDB Server.
Vector indexes are, like one would expect, indexes that work on vectorised data. The key point here is that vectors cannot cleverly be indexed using traditional forms of indexing in relational databases, but need specialised methods, such as IVFFlat or HNSW. No, IVF is not In-Vitro fertilisation and NSW is not New South Wales. Ivfflat stands for inverted file flat and HNSW for Hierarchical Navigable Small World, but aside from impressing your AI know-it-all friends, you hardly need to train your memory to remember those abbreviations. The important concept is “nearest neighbour” and the idea that the vector index search is about finding similarities. Finding the most similar entry in the relational database to what you are looking for (which may be user input, or another database entry, or anything).
ACID is still a virtue
ACID, Atomicity Consistency Isolation Durability, are the four core virtues of transactions in relational databases. ACID is not in itself related to vectors. Rather, the point is that we continue to need the good old virtues of conventional IT in general and relational databases in particular, also in the brave new world of generative AI.
Interpreting Sirish
Sirish is very articulate in his interview, and hence hardly in need of my mansplaining. But let me nonetheless provide a short summary of some of his points.
- Customers bet their businesses on relational databases, which are a huge part of what drives the world and at the heart of enterprises.
- AWS customers say Open Source databases have caught up, and they can bet their businesses on Open Source relational databases.
- AWS is increasing their investment in Open Source.
- For Gen AI to be impactful, it needs to be based on existing, accumulated data. And that data is in RDBMSes.
- RAG allows you to customise an LLM to your business based on the data you have.
- We don’t need new vector databases; vectors should be brought into existing databases, which users know and have invested in.
- PostgreSQL already has a vector storage plugin as supported by AWS, but AWS is also supporting MariaDB Foundation’s vector work.
- Users have their usual expectations (scalability, performance) on databases also in the vector context.
- Amazon is the third-biggest contributor to MariaDB Server, after MariaDB plc and MariaDB Foundation.
- Beyond Amazon itself contributing to MariaDB Server, AWS is also helping MariaDB Foundation ensure that the MariaDB ecosystem is healthy and vibrant.
- Indexing accuracy and performance are important, and HNSW scales better than ivfflat.
Let me repeat that I encourage you to look at the Sirish video. No, Sirish doesn’t have a philosophy degree (in contrast to what the otherwise well prepared interviewer thought), but he has a PhD in computer science (on streaming databases). And he is philosophical enough to give a proper perspective on what relational databases are good for and whether they are Boring Old Technology or still a hotbed of innovation. Sirish in this interview is the same clear, smart, and affable Sirish that I have spoken to in my role as CEO of MariaDB Foundation.
MariaDB is soon a vector database, too …
Let me put MariaDB in perspective and set some expectations related to the above generic terminology, as well as to Sirish’s thinking.
- MariaDB Foundation and MariaDB plc have already been working for a while on MariaDB Vector.
- MariaDB will soon have vector search, too – expect the first developer preview binaries still in May. [1]
- MariaDB’s vector search is focusing on HNSW (“usually better than ivfflat”) from the start.
- Unlike PG Vector, the vector search functions in MariaDB will not be an extension (“a plugin”, like PG Vector on top of PostgreSQL), but a core part of the native syntax of all MariaDB Server binaries.
- Much like PG Vector, MariaDB Vector will keep the LLM itself separate from the binaries, and be agnostic towards the user’s choice of LLM; in other words, you can use ChatGPT (3.5 or 4 or whatever), Claude, LLaMa2 and many other LLMs with MariaDB.
- RAG (retrieval-augmented generation) is the core use case of MariaDB Vector, but that’s a statement which is still on a fairly high abstraction level; the concrete use cases are still emerging.
… and then we’d like you to test it!
[1] Vicentiu Ciorbaru has created a bb-11.4-vec-preview branch where one can try out the nightly version of MariaDB Vector; our estimate is that by end of May, the developer preview will be stable enough for it to be testable on common use cases.
All in all: Gen AI is not a mere hype, and combining it with open-source relational databases is the crucial next step. Around FOSDEM in March, we already demoed the core functionality of MariaDB Vector. Expect to soon get the first preview release of MariaDB Vector on our download pages!
