Skip to content

Huginn: Monitor changes in websites

The Huginn software is not only good for monitoring Twitter feeds, it can also be used to monitor websites for changes.

That's actually easy and needs only two agents:

 

 

In my example I'm monitoring a certain Play Store entry for price changes. Because, why not. So I need to configure the Website Agent with the following details:

Website Agent

  • expected_update_period_in_days: 2 (since I run this agent every day, it should produce an update every day)
  • url: <the Play Store URL>
  • type: html
  • mode: on_change

So far, so easy. The complicated part is figuring out the CSS selector for the price. It helps to open the website in any modern browser in inspection mode, and hover over the interesting area. The CSS selector is listed there. You also need to name the selector in Huginn, in my case I name it "price".

  • extract: add "price"

Under "price" add:

  • css: #fcxH9b > div.WpDbMd > c-wiz > div > div.ZfcPIb > div > div > main > c-wiz:nth-child(1) > c-wiz:nth-child(1) > div > div.D0ZKYe > div > div.wE7q7b > div > div.hfWwZc > div > c-wiz > c-wiz > div > span > button
  • value: normalize-space(.)

If the content is parsed as "html" (or "xml"), then the selector is an XPath function. If parsing as JSON, the selector is a JSONPath function.

The complete option field, for reference:

{
  "expected_update_period_in_days": "2",
  "url": "https://play.google.com/store/apps/details?id=com.flashlight.ultra.gps.logger",
  "type": "html",
  "mode": "on_change",
  "extract": {
    "price": {
      "css": "#fcxH9b > div.WpDbMd > c-wiz > div > div.ZfcPIb > div > div > main > c-wiz:nth-child(1) > c-wiz:nth-child(1) > div > div.D0ZKYe > div > div.wE7q7b > div > div.hfWwZc > div > c-wiz > c-wiz > div > span > button",
      "value": "normalize-space(.)"
    }
  }
}

 

Email Agent

All what's left is sending a notification when something changes. I'm using the Email Agent for this, but any notification agent can be used.

  • expected_receive_period_in_days: 3650 (since I don't expect changes anytime soon, I just set this to a very high number - otherwise Huginn will disable the agent because it's not functioning)
  • subject: set the subject of the email
  • content_type: text/html
  • recipients: specify add email addresses for all recipients
  • body: text for the email body

As body text I configured this text:

Updates for the Ultra GPS Logger App!<br/><br/><a href="https://play.google.com/store/apps/details?id=com.flashlight.ultra.gps.logger">Visit App Store</a>

 

Trackbacks

No Trackbacks

Comments

Display comments as Linear | Threaded

No comments

Add Comment

Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
E-Mail addresses will not be displayed and will only be used for E-Mail notifications.
To leave a comment you must approve it via e-mail, which will be sent to your address after submission.
Form options