Non-human Click and Open Engagement Recorded

Issue

Aggressive spam filters can open messages and click links in incoming mail before delivering them to attempt to expose malicious content. Similarly, some email service providers prefetch opens to improve their user experience. Currently, Twilio-SendGrid does not filter out clicks or opens that are produced by recipient bots or security software. As a result, all activity will be included in reporting.

Product

Email 

Cause

  • High number of clicks reported at approximately the same time that a message is delivered
  • Same IP address reported for a high number of click events. The IP address reported for a click event should be unique for each recipient.

 

Resolution

Click troubleshooting

It is possible some anti-spam filters or recipient servers might employ link test software to verify that a link or visible URL, is in fact, safe to click for a human to prevent phishing scams. Although it has benefits to security, it can complicate the reliability of engagement data for senders in certain cases.
 
At this time, Twilio SendGrid's system does not distinguish between legitimate and illegitimate clicks because the signal is essentially the same for a human user and a bot. We are not able to filter clicks like this from being tracked on our side, as the tracking takes place when a link is clicked whether or not the click is initiated by a human recipient or software. However, there are tools you can use to try and isolate false positives caused by bots or filters.

 

Open troubleshooting

Another cause of non-human engagement reporting is Google and Apple pre-fetching opens. Each provider does this for different reasons: Google to improve speed for user experience and Apple to preserve user privacy.
  • Gmail signals someone was active in the inbox, resulting in Gmail wanting to render the content quicker to provide a better user experience with shorter loading times. 
  • Apple signals that the recipient has requested anonymity in engagement—they are seeking data privacy. 

Identifying prefetched opens

As of today, if you are looking to identify Gmail prefetch opens, you can find the "useragent" string below passed to you in your SendGrid-associated Event Webhook. 

  • Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246 Mozilla/5.0
  • Mozilla/5.0 (Windows NT 5.1; rv:11.0) Gecko Firefox/11.0 (via ggpht.com GoogleImageProxy)

The Event Webhook also includes an Apple Open Indicator. The Apple Open Indicator is a boolean field, “sg_machine_open,” included with all open events to identify MPP-enabled user opens. When the “sg_machine_open” boolean field is “true,” it will indicate that the open came from a user with MPP (Apple Mail's Privacy Protection) enabled.

Additional engagement troubleshooting

You can potentially use the fields: "useragent", "timestamp", and "IP" that are provided for click and open events and passed in the Event Webhook to uncover more information regarding where these events originated.

  • Useragent strings usually contain information that look like real-world combinations of devices you know. Web crawlers, mail filters or other pieces of software can be identified using the user-agent, potentially programmatically.
  • A high count of the same IP clicking links at the same time is also an indication that the events are not attributed to a person clicking the links.
  • Additionally, a large number of clicks/opens associated with a single recipient, in a short amount of time, or even just before a delivered response is recorded can be indicative of illegitimate engagement.

Additional Information

 

 

Have more questions? Submit a request