<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Deduplication on ads' corner</title><link>https://andreas.scherbaum.la/tags/deduplication/</link><description>Recent content in Deduplication on ads' corner</description><generator>Hugo</generator><language>en-us</language><lastBuildDate>Wed, 21 Oct 2020 17:25:00 +0200</lastBuildDate><atom:link href="https://andreas.scherbaum.la/tags/deduplication/index.xml" rel="self" type="application/rss+xml"/><item><title>Remove duplicates from Huginn event stream</title><link>https://andreas.scherbaum.la/post/2020-10-21_remove-duplicates-from-huginn-event-stream/</link><pubDate>Wed, 21 Oct 2020 17:25:00 +0200</pubDate><guid>https://andreas.scherbaum.la/post/2020-10-21_remove-duplicates-from-huginn-event-stream/</guid><description>&lt;p&gt;One of the things I&amp;rsquo;m using &lt;a href="https://github.com/huginn/huginn"&gt;Huginn&lt;/a&gt; for is monitoring for &lt;em&gt;Twitter&lt;/em&gt; keywords. Some of them might appear in pairs, as example people like to tag Tweets about &lt;a href="https://www.postgresql.org/"&gt;PostgreSQL&lt;/a&gt; with both &lt;a href="https://twitter.com/search?q=%23postgresql&amp;amp;src=typed_query&amp;amp;f=live"&gt;#postgresql&lt;/a&gt; and &lt;a href="https://twitter.com/search?q=%23postgres&amp;amp;src=typed_query&amp;amp;f=live"&gt;#postgres&lt;/a&gt;. When I was using &lt;a href="https://ifttt.com/"&gt;IFTTT&lt;/a&gt; this always created two emails, one for each hashtag. With &lt;em&gt;Huginn&lt;/em&gt; I can deduplicate the events, and only notify about the first occurrence.&lt;/p&gt;</description></item></channel></rss>