I’m using n8n in conjunction with the Qdrant vector database and I’m facing an issue with duplicate documents. According to the Qdrant documentation, if I use the same ID for inserting points, the previous point should be updated instead of creating a new entry. However, I’m struggling to find a way to set a specific ID for Qdrant points within n8n. I’ve tried to include the ID in the metadata but that doesn’t correspond to the point ID, which results in duplicates being generated whenever I submit the same document. Can someone please guide me on how to successfully set the point ID in the Qdrant integration for n8n?
You need to set up your custom IDs correctly to update Qdrant points instead of duplicating them. Don’t put the ID in the metadata - create a separate ‘id’ field at the same level as your vector data. The Qdrant node expects the ID as a top-level property. Structure it with an ‘id’ field, then add your vector and metadata separately. This should fix the duplication issue and update points like you want.
I faced a similar issue with n8n and Qdrant before, and it took me quite a bit of troubleshooting to resolve it. It’s crucial to ensure that the ‘id’ property is at the top level of your input JSON, as the Qdrant node won’t recognize it if it’s nested within another structure like payload or metadata. When you define your input, directly include your custom ID; this must be explicitly done to prevent n8n from generating random UUIDs. Additionally, consider using a consistent way to create unique IDs, such as hashing or extracting them from your source data, which will help maintain unique references in Qdrant.
yeah, this got me too! use upsert instead of insert in the qdrant node settings. even with proper ids, insert mode still creates duplicates sometimes. upsert forces the update behavior you’re looking for.