TGLIST.COM.UA
The Duplicate Problem
RSS feeds, mailboxes, and other sources often return the exact same data on every run. Without filtering, you would reprocess and publish content that has already been handled.
How Filtering Works
When the "Only Unique" option is enabled in a node, the system:
- Gets data from the source
- Generates a hash (fingerprint) for each element
- Compares it with hashes already stored in the database
- Passes forward only new, previously unseen elements
Which nodes support this?
Filtering is available in source nodes:
- RSS ✅
- Prom.ua ✅
- Rozetka ✅
- Gmail Get ✅
- Google Sheet Get ✅
- Telegram Source ✅
Behavior During Testing
⚠️ Important: During a test run (manual run, single node test), filtering is automatically disabled, even if the "Only Unique" option is enabled in the node settings. This is done so that nodes always return data when testing.