Monday 14 May 2018

System update - privacy enhancements

We've been rolling out numerous small updates over the last few months, and we've just pushed a big one. It's mainly about our data protection configuration for GDPR and ePR compliance. Smartmessages has always had a very strong policy on data protection, and this means we have not had to change anything fundamental for GDPR (that's why you're not seeing any of those silly "reconsenting" emails from us), however, we have improved some smaller things to enhance our compliance.

Support for Do Not Track

The biggest change is that we now support "Do Not Track " and anonymous tracking. If a subscriber opens a message we send, or clicks a link in a message, those requests are served by Smartmessages, and normally we record these in full, including the identity of the subscriber (something which is mentioned at the point of sign-up so subscribers are aware of this before they subscribe - see below). If a subscriber has the "Do Not Track" setting enabled in their browser, we will still record that an open or click has occurred, and which mailshot that it happened in, but we do not record their identity. This will mean that you see some new stats in your mailshot reports for anonymous opens and clicks, and subscribers making use of this feature will not appear in "Hot List" reports.

Enhanced Data Subject Access Request support

Under GDPR and earlier data protection law, anyone that you store data about can request to see, amend, and delete the data that you store about them. These are called DSARs. It's extremely rare for web apps to have any integrated support for DSARs, but we introduced built-in support in 2005. Anyone whose data is used by Smartmessages (whether as an account holder or list subscriber) can log in and see the data that is stored about them, and amend or delete it at will, as is their right.

Improved data retention implementation

We have deleted low-level data after 6 months for many years, but there were some places where user data was kept unnecessarily, particularly in archived mailshots (ones more than 6 months old). We have made some internal changes to make it easier for us to delete data held in logs and archived mailshots, either due to expiration or DSARs.

Subscription page privacy policy

We have clarified key items of our privacy policy on our standard subscribe and landing pages, right where it's needed most. You can see in in action on our own subscribe form. If you host your own subscribe forms, you need to present these same options to your potential subscribers - to skimp on that means that even double-opt-in subscriptions will be invalid since transparency of processing is a requirement under GDPR.

Landing page improvements

Our default landing/preferences page — a simple destination to manage multiple subscriptions and basic data collection — has had a cleanup, making the layout more compact and easier to use on mobile devices, and also easier to customise by providing more ID and class selectors for your custom CSS to target.

Gravatar privacy proxy

Previously we made direct use of the Gravatar service (operated by Wordpress) to provide avatars for subscribers. Doing this leaks IP addresses of the subscribers to a US-based entity without explicit permission, and we don't like that, so we implemented a proxy service that means that subscriber avatars are served via our own servers, in a way that means that Gravatar is never contacted by subscribers directly, and their IPs are never revealed. This was the only remaining external service that could handle subscriber data, so now we can be certain that data is shared with nobody except  Smartmessages account holders, who are the data controllers for subscriber data. Yes - we're now entirely free of tracking cookies and scripts.

Enhanced Content Security Policy

We have strengthened our content security policy (a technical feature in HTTP) substantially. This mainly applies to the domain that we use for open & click tracking, and for serving images. The new configuration now means browsers will reject anything served from this domain that's not an image. This helps us stay off malware scanners - if someone should ever manage to upload, for example, a malicious javascript file that ends up served from this domain, browsers will refuse to load it. Our CSP has been tightened on the rest of our sites too, and that may interfere with things that rely on privacy contraventions, such as Facebook "like" buttons. We also no longer leak data through HTTP referrer headers - some other ESPs had serious issues with this, but we were never exposed to that. This will not affect mailings as normal HTTP links continue to work just fine.

You're welcome to test our domains at any time, using tools like and Qualys SSL labs. Should you find a security issue that you would like to report to us privately, please use our standard security.txt file. Of course you should run the same tests on our competition too!

Importing mailing lists

We've always supported importing mailing lists, but one very common aspect of exported lists is a lack of information about the origin of the subscription, and precisely when it occurred. We have supported the export of this information in our list exports for many years, however, we have not supported it on import. That's now changed, and we are now able to import IP, timestamp, referrer URL, and user agent strings used by subscribers (at the point of subscription confirmation) from imported records. Imported records that contain a valid public IP and timestamp will now be marked as having completed a double-opt-in process. This is specifically compatible with the format used by Mailchimp, making migration to Smartmessages even easier. Documentation on our export and import formats has been updated to match.

Importing templates

Importing templates has been an important feature since the beginning, but to date it's been limited to copy/paste, or importing from a URL. We now support importing from local files, and specifically to import from zip files containing HTML & plain text files. This is a common format used by third-party email template creation tools, and also used in exports from various other ESPs. We automatically apply format conversions so that templates designed for other ESPs can work as expected - though of course you should always test before committing to a big send. We will be adding the ability to import images linked to these templates automatically as well.

Improved Excel report generation

You won't see much visible difference, but the system used for generating Excel-format reports has been overhauled, switching to a new PHPSpreadsheet implementation, which should be faster and more reliable.

Migration to PHP 7.1 complete

All our services are now running on at least PHP 7.1, and in some parts, 7.2, helping both security and performance. Work on migration to PHP 7.3 and MySQL 8.0 has already begun.

We also switched this status blog to use HTTPS. Woohoo!

I think that's quite enough to be getting on with, but there is more to come! As always, if you would like to ask us anything, contact us.

No comments:

Post a Comment