I completed agree with your point

In my company, we build the internal chatbot based on RAG through LangChain + Milvus + LLM. Since the documents are well formatted, it is easy to do the overlapping chunking, then all those chunking data are inserted into vector db Milvus. The hybrid search (combine dense search and sparse search) is native supported in the Milvus could help us to do better retrieve. Thus the better quality answers are there.

Hybrid search usually refers to traditional keyword search (BM25, TF-IDF) combined with a vector similarity search.

I would like to mention that vector databases like Milvus got lots of new features to support RAG, Agent development, features like BM25, hybrid search etc..

The test environment is deployed on us-east-1, whereas the production environment deployed on us-west-2 on our side.

https://discord.com/blog https://blog.cloudflare.com/ https://netflixtechblog.com/