How we managed to create a safe and rich HTML email renderer
Everything used to be bad, and here is why
In the world of email, the quality with which HTML emails are displayed leaves much to be desired. The main problems are associated with the differences in the ways HTML and CSS are supported in different email clients and web interfaces. Often emails that look perfect in one service are displayed poorly in another service. Users understandably get upset, conversion drops, and the brand’s image suffers as a result.
In large part this is explained by the fact that there are only a handful of monopolistic services in the world that have a web interface for displaying emails, and only a few popular email clients. Gmail and Yahoo haven’t changed much in the way they display emails in years, even though HTML and CSS have improved drastically over time and now provide many new options for content display. For example, adaptive design first emerged with the implementation of media queries around 10-12 years ago, support for the dark theme in CSS emerged about 4-5 years ago, but to this day none of the large email services support these features. They simply have no reason to change anything — instead, it’s webmasters who have to adapt to the whims and undocumented limitations of these services.
For instance, some support animations and others don’t. Gmail doesn’t even support its own (invented and popularized by Google) webp image format, but iCloud does. And there are hundreds of examples like this one. Webmasters are forced to relive the early days of the Internet when the browser wars were raging and you had to jump through all kinds of hoops to make web pages look similar in different browsers.
Yes, at times, some restrictions, such as the lack of script support, are dictated by safety concerns — you wouldn’t want to get an email with an interactable banner that sends your data directly to some ad network. But more often than not the set of limitations looks absolutely inadequate — what harm have simple background images, set with background-image, done to Gmail?
What we came up with
When we made our first steps on the email market with AdGuard Temp Mail, we spent a lot of time studying the email display options. We definitely didn’t want to become another Gmail, inventing our own HTML/CSS subset and cutting down on all the components that we would deem unnecessary. But we also weren’t planning on leaving users one-on-one with web threats.
Chapter 1. Seven strings of JavaScript code that solved the problem
Fortunately, the world of web development doesn’t stay still, and browser developers have long been working on browser sandboxes, which we have taken advantage of.
What are browser sandboxes? They are essentially a set of techniques to restrict the capabilities of the web page, literally a list of dos and don’ts very similar to CSP (we use it too, but more on that later): “don’t use cookies of the parent page,” “don’t execute JavaScript,” “don’t send HTTP Referrer when someone is following a link from the frame” and a number of other directives. Sounds interesting, doesn’t it? Let’s delve deeper into the details of technical implementation.
We receive the email body from the API as a string containing HTML markup, which we then display in the browser using sandbox capabilities:
const html = '<html><body>....</body></html>';
const iframe = document.createElement('iframe');
iframe.credentialless = true;
iframe.sandbox = 'allow-popups allow-popups-to-escape-sandbox';
iframe.referrerpolicy = 'no-referrer';
iframe.srcdoc = html;
document.body.appendChild(iframe);
Add some styles, position the frame on the parent page — and you’re golden, you’ve just created a robust, safe engine to display HTML emails without unnecessary limitations!
In a perfect world that would be the end of it. After all, didn’t we stop the main threat — JavaScript? Unfortunately, modern-day emails often include a mountain of trackers of all kinds, which aim to collect as much data about you as possible. They are zero-pixels — invisible images that transmit your IP address and data about your device to their servers — and special tracking links that record every single action you perform within the email. And we had to find solutions to all of these problems.
Chapter 2. User anonymization
During the development we stumbled upon an excellent instrument to estimate the safety of our engine — emailprivacytester.com.
This service generates an HTML email that contains a number of elements specifically designed to test out certain vulnerabilities. Now we can start the diagnostics process, send ourselves an email to an AdGuard TempMail mailbox and… observe multiple IP address leaks.
Just like we dealt with JavaScript and other similar problems earlier, let’s now focus on these IP address leaks. IP address leak happens when it’s the email sender can figure out your IP address just by sending the email. Of course, if you’re using AdGuard VPN then all the sender will get is the IP address of the VPN server you’re connected to. But do this even once without the protection of a VPN, and the perpetrator will be able to learn your geolocation down to the city. Of course, we couldn’t let that happen to our users. So what to do?
Image proxy
Fortunately, there was no reason to reinvent the wheel. All of this has been done before, and all the “big” email service providers have been doing hust that for a long time: they wrap all images in their own proxy.
How? For example, if your email had this image:
<img src="http://my.website/image/cat.jpg" />
then every time you open the email, the owner of the my.website server receives an HTTP request. The metadata of this request will contain your IP address, along with some additional data about your device, including your operating system, browser version, and other details that could potentially deanonymize you.
How does the proxy work? It replaces the src attribute in all image tags with its own URL, as shown below:
<img src="https://email.provider/image?url=http://my.website/image/cat.jpg" />
The email.provider web server is designed to download every source image only once. After that, it saves the image and the subsequent requests are not seen by the owner of the my.website server. Even for the initial request, my.website only sees that it’s made by email.provider and records only its IP address, not any personal data. It’s good and all, except that in this case such data is collected by email.provider itself (and we remember that it is often a large company like Google or Yahoo who plays this role).
We developed a similar solution. However, unlike others, our image proxy does not collect or store user data and is completely anonymous.
Having set up the proxy, we now need to somehow “wrap” all images inside the email in it, and for that we need to look at our source HTML as a DOM. We can’t work with DOM inside the sandbox since it doesn’t have JavaScript. We can’t execute JavaScript there even for our own code, as it would open the door for executing it for all the other code in the incoming emails. And if we’re to work with DOM outside of the sandbox, we’d like to have some insurance that the HTML is clean.
Luckily, we have found an amazing library: DOMPurify. It can “clean” HTML content and make it safe and structured. It has been extensively tested and has been deservedly praised by security experts. Let’s start using it:
const html = '<html><body>....</body></html>';
const clean = DOMPurify.sanitize(html, {
// restore partial HTML to a full-fledged document, if needed
WHOLE_DOCUMENT: true,
// these tags make no sense in the context of an HTML email, so we’re deleting them
FORBID_TAGS: ['audio', 'video', 'button', 'input', 'form'],
});
const iframe = document.createElement('iframe');
// the set of actions with the sandbox from step one
iframe.srcdoc = clean;
document.body.appendChild(iframe);
Great, we’ve successfully removed the clutter, now we can return to image processing. DOMPurify will help us here, too! It has an excellent hooks mechanism — we can solve the next problem without changing the context:
DOMPurify.addHook('afterSanitizeAttributes', (node) => {
if (node.tagName === 'IMG' && node.hasAttribute('src')) {
// changing the original src to a link to ImageProxy
const src = wrapWithImageProxy(node.getAttribute('src'));
node.setAttribute('src', src);
}
});
Now it’s time to run the diagnostics again, and we see that the number of IP leaks has dropped… but not to zero. How come? It’s time we talked about CSS.
IP address leaks through CSS
CSS carries truly impressive content stylization capabilities. No other system even begins to approach CSS — its simplicity, ease of use, and rich functionality make CSS one of the best inventions of software engineering. However, like many powerful tools, some of its features can be misused. In the context of our task, the “harmful” part is the url() function, which downloads content via a link to style the page. The most commonly seen example is background images. The code to download them looks like this:
background-image: url(http://my.website/image/cat.jpg);
The problems here are exactly the same as the ones described in the previous chapter — IP leaks. Besides the background images, the url() function, as detailed in Mozilla’s CSS documentation, can be used in various CSS rules. There are a lot of them, and the syntax of the CSS selector allows to set relatively complex rules, such as the selector from one of the examples by the link above:
background-image: cross-fade(20% url(http://my.website/image/first.png), url(http://my.website/image/second.png));
It’s not easy to safely parse a rule like that, you’d need proper tools. And we got lucky again and found exactly the tool for the job! csstree does precisely what we need — allows to analyze CSS and change some of its selector types. In code, it looks like this:
csstree.walk(ast, (node) => {
if (node.type === 'Url') {
node.value = wrapWithImageProxy(node.value);
}
});
which gives us this output:
background-image: cross-fade(20% url(https://img.agrd.eu/image?url=http://my.website/image/first.png), url(https://img.agrd.eu/image?url=http://my.website/image/second.png));
Voila, no more IP address leaks!
CSP
Data privacy is a field where there can’t be any exceptions. Not a single one. That’s why it is imperative to come up with something for cases when a vulnerability would be discovered in one of the libraries or in our code that would incorrectly interpret data. The last line of defence, when everything else fails. In our case, we use the one and only CSP in this role. CSP is a set of directives allowing granular control of page access to certain resources. Webmasters often struggle with CSP. It’s easy to forget that some new resources needed on the web page are not allowed in CSP settings — to be completely honest, we have made this mistake in the past, too. But we managed to tame CSP with a solution like this:
// restrict image sources to only those wrapped inside our proxy
img-src data: https://img.agrd.eu/image;
// disallow connections to any sources not explicitly permitted above
script-src 'none';
style-src 'none';
font-src 'none';
connect-src 'none';
media-src 'none';
object-src 'none';
prefetch-src 'none';
child-src 'none';
frame-src 'none';
worker-src 'none';
frame-ancestors 'none';
...
// which can be simplified as
default-src 'none';
Now we are safe: even if one of the libraries goes rogue or our image address transformation function code breaks, CSP will not let the browser send a request to the perpetrator’s server and leak your IP address.
Results of our work with data privacy
Thanks to the capabilities of the modern browsers and a few excellent libraries, we’ve achieved a safe, high-quality display of HTML emails without any IP leaks, JavaScript, or any other potentially dangerous content — everything is completely under our control.
What is so great about it
We’ve covered many security-related topics, possibly shifting the focus away from a crucial achievement: we managed to create 100% HTML5/CSS3-compatible HTML emails, something literally nobody else in the world has done! And that means that you have at your disposal all the capabilities that modern-day HTML pages can offer: animation, adaptivity, modern image formats, dark themes as they were intended (don’t you also despise the ugly color shifts from primitive algorithms?). You don’t have to suffer through complex layouts using tables like in the 90s, but can instead design beautiful adaptive emails for mobile devices with ease.
It is wild that in the mid-2020s, major email providers still display emails with numerous flaws and vulnerabilities. We are extremely happy that we managed to create a safe alternative that meets all the requirements of a modern email client.
If something goes wrong
We are proud of everything we have achieved with AdGuard Temp Mail so far, but we also realize that we’re still in the beginning of our journey. When you create a new product, it’s impossible to avoid all mistakes and shortcomings. If you identify a vulnerability in email display, please report it to security@adguard.com. We will deal with it in the shortest time possible, and you will become eligible for a reward as part of our Bug Bounty program.
Our developer team takes user feedback very seriously. So if you notice that your HTML email is displayed incorrectly, please send us a message to support@adguard.com, and we will address the problem promptly.
