I’m working on setting up automated daily reports that should refresh when new database entries are detected. Here’s my current SQL query:
SELECT COUNT(distinct te.transaction_id) AS "id"
FROM transaction_events te
WHERE te.status IN ('confirmed', 'auto_confirmed') AND
te.timestamp >= current_date AND
te.timestamp < current_date + INTERVAL '1 day';
The issue I’m facing is strange. The dashboard numbers update correctly for about the first day, but then they completely stop refreshing. When I manually run the same query in my database, the results clearly show different numbers.
I discovered something important in the documentation: it mentions that once an item has been processed by a Zap, it won’t trigger again even if the data changes later. This explains why my daily metrics only work for the initial 24-hour period. After that, since the system has already seen those particular results before, it won’t process them again.
Has anyone else run into this deduplication issue when building daily dashboards? I’m not sure how to work around this limitation.
Same issue here! Switched to webhook triggers instead of database ones and it solved everything. I just send a daily webhook request (using cron, but geckoboard’s built-in refresh works too) and it completely bypasses zapier’s dedup. Takes a bit more setup but beats dealing with zapier’s quirks.
Yes, Zapier’s deduplication can indeed hinder automated reporting. I faced a similar hurdle and found effective solutions. The best method involves replacing database change triggers with a Schedule trigger in Zapier. Set this trigger to execute daily at a designated time, allowing it to run your SQL query directly. This effectively circumvents the deduplication issue since it relies on scheduled execution rather than data changes. Alternatively, consider modifying your SQL query to generate unique results for each day, such as appending a date to your metric value. This ensures that Zapier treats the data as new each day. Both strategies can maintain the functionality of your reporting dashboard.
I hit this same issue about six months ago and just worked around it instead of fighting Zapier’s deduplication. I added a timestamp field to my query that forces uniqueness - modified my SELECT to include current_timestamp alongside the count. Now each run looks different to Zapier even when the count hasn’t changed. Geckoboard still shows the metric correctly since it only uses the count field anyway. It’s hacky but it’s been rock solid ever since.