Removing Website Functionality to Improve Privacy

Embedded third party content usually comes with third party cookies and longer loading times. Let's change that.

Embedded Content On This Site

As with many modern Web sites, this site utilises third party content embedded on pages.

At the time of writing the complete list of third party content is (I believe):

Google Fonts (requires user action).
Twitter (no user interaction required).
Facebook (no user interaction required).
YouTube (no user interaction required).
Vimeo (no user interaction required).
Google Custom Search (automatically loaded, but at /search only).

One thing you might notice about that list is that Google Fonts requires user action before being loaded. This wasn't the case in the past, but I decided as a matter of principle I should not be automatically loading the fonts this site was designed around. A lot of people now browse from mobile devices and/or are charged for data usage and/or have data caps—loading fonts uses bandwidth.

Another reason font loading requires user action is because of loading time, and that switching fonts after loading can cause a FOUC (Flash Of Unstyled Content) and/or reflows (when the sentence you are reading annoyingly moves because something is inserted/removed or changes size further up the page).

By requiring user action, pages load faster (albeit not looking exactly how I want them to look) and the user doesn't get surprised/frustrated when the sentence they are reading suddenly moves. This site does dynamically load alerts after page load, however, so there is still the possibility of a reflow.

Clicking the load fonts button dims the text of the page (making a FOUC unlikely), switches the fonts, and then undims the page text. It also saves a cookie that is valid for 30 minutes (time refreshed on each subsequent page load) so the fonts are automatically loaded for subsequent page loads.

I decided on thirty minutes because the odds of you loading a page of the site now, and you still reading the page in 30 minutes without doing anything else in your browser, is probably unlikely.

That is also why the prompt to load fonts is worded in the way it is. It doesn't say the fonts are missing but the "fonts cookie" is missing. If the fonts are still in your browser's cache clicking the load fonts button will just load the fonts and save the cookie—it will use zero bytes of network data/bandwidth.

Third Party Content Problems

One of the problems with third party content is that it relies on those third parties' servers being up. At the time I wrote this paragraph, this page relied on my server being up, Twitter's server being up, and Facebook's server being up.

Although the page would still load if Twitter or Facebook were down, it would look like it was still loading if either were taking a long time to timeout.

Every page on this site loads the Twitter and Facebook JavaScript (used for share buttons and embedded tweets/posts), even though I have removed the share buttons.

For most people the odds of their browser needing to fetch the JavaScript files for Twitter/Facebook is low, purely because a lot of sites reference the same JavaScript files so your browser will probably have it in its cache (although with short lifetimes, it might send an If-Modified-Since GET request to Twitter and/or Facebook to see if the file/files has/have changed).

Having said that, this third party JavaScript might come bundled with Web beacons (i.e. it loads tiny images that are never cached) so that content provider knows things like how many people have visited a page that embeds their content.

This is the second problem, and comes under the banner of privacy.

If a tweet is automatically embedded on the page, Twitter will not only know you are visiting a page that embeds that tweet but potentially what page you are visiting.

In terms of privacy that isn't a good thing, and is part of the reason some people are vehemently against Google having such widespread reach. The reason I use Google Web Fonts (and Google's CDN for jQuery) despite this is because there is a greater chance your browser has those resources in its cache before you visited my site (although admittedly the fonts I'm using aren't widely used).

Google Web Fonts: What does using the Google Fonts API mean for the privacy of my users?
Google CDN (jQuery): What does using the Google Hosted Libraries mean for the privacy of my users?

Local CDN Emulation

Something I will keep my eye on is Decentraleyes [Firefox Add-ons].

Although currently only available for Firefox (and derivatives), this closed issue suggests at a later date it "will be ported to WebExtensions" making it potentially cross-browser compatible.

Although created with privacy in mind, it has the potential to make Web sites and their pages load much faster.

Basically, it is a browser extension that comes bundled with the most popular JavaScript libraries. Whenever a browser wants to load (for example) jQuery version 2.1.4 from Google's Hosted Libraries CDN, Decentraleyes goes "no you don't, use this instead" and the browser loads a local copy of the JavaScript that came bundled with the extension.

Since Chrome (and other derivatives of Chromium) is the most widely used browser, if/when this browser extension gains cross-browser support I may well move some JavaScript that is served first party (i.e. bundled together in a single JavaScript file for the whole site) and load it instead from a CDN.

Although that has the potential to increase load times on browsers without the extension and the referenced JavaScript not in the browser cache, for those with the extension it will improve loading times because the site JavaScript bundle will be smaller and the CDN JavaScript will definitely be loaded from a local source.

Again, such a move is a long way off and will rely on other metrics such as benchmarks performed by others before I consider implementing such changes.

How Other Sites Deal With Third Party Content

There are various ways of dealing with third party content so that it doesn't affect privacy or (significantly affect) load times.

Screenshots and Images

One way of doing things is how BBC News does it. Instead of embedding tweets from Twitter and posts from Facebook they use screenshots of the tweets/posts.

Although first party images (or images loaded from, for example, the BBC's CDN provider) use more bandwidth than embedding would, no requests are sent to third parties and no third party JavaScript is loaded.

Conversion To First Party Before Publication

Another method of doing things is converting the embeddable content to first party content before publishing the Web page.

As long as it is in the Terms of Use/Service and Acceptable Use Policy (and any other policies the third party content provider may have), it is possible to convert something to first party content without needing to use 3rd party JavaScript or embedding.

As an example, you can transform a tweet into text. As long as you comply with Twitter's rules (Display requirements, Terms of Service, Brand Assets and Guidelines, Developer Agreement and Developer Policy), you don't need to actually use Twitter's JavaScript and embed code.

APIs and Server-Side Caching

A lot of sites now have APIs available. YouTube, for example, have an API that could actually be used (with server-side caching) to make things more private for those that don't want to watch an embedded video.

I could, for example, cache the thumbnail image of a YouTube video and have two links underneath it: Load Embedded Video, Watch on YouTube.

Watch on YouTube would just be a link to the video on YouTube using rel="noreferrer" to improve privacy in supported browsers.

Load Embedded Video would use some JavaScript (jQuery?) to swap the image for the iframe embed code.

Changing How I Do Things

Third party embedded content and JavaScript is everywhere these days. Rolling this site back from using embedded content by default to only using it when the user wants it will be a big change and divergence from the "modern way".

A lot of sites follow the paradigm that people clicking something and leaving your site is a bad thing to do. You want people to stay on your site, so you load up external content in a new window, tab, or a modal.

You prevent people from watching that embedded video in fullscreen. You ignore the fact that if someone wanted to open a link in a new tab they would right-click and choose to do so (or use another method depending on device/browser).

Do I want to open a Twitter share page in a popup using Twitter's JavaScript? I could instead create a link that takes the visitor to Twitter. Like so: Share the About page. Not that I currently use share buttons on this site.

Twitter Embeds

For Twitter, I am likely to switch back to how I did things before I used embedding, although to do so I will need to carefully make sure it complies with Twitter's rules.

As part of bringing everything (where possible) in-house, I am looking at transitioning from twitter to GNU Social. I still have a lot more research (and installation and testing) to do before working out if such a move is feasible.

Now, for content I myself posted on Twitter I still retain the copyright, so I don't actually have to follow the display requirements unless I wish to link to the tweet or mention Twitter (or use any of their trademarked property).

In essence, Twitter's Display Requirements for Online ("Mobile, Web, and Beyond") state the following:

Full Name must be displayed.
Username must be displayed and prepended with @.
In left-to-right languages, user's avatar must be positioned left of full name and username.
Full Name, Username, and avatar must link to the Tweeter's Twitter profile.
The Tweet text must be displayed on new line below Full Name and Username, and must not be modified. Whitespace must link to the Tweet's permalink.
Tweet Entities within the Tweet text must link to where they would link on twitter, and links must link to the t.co URL with the link text specified in the display_url field obtained from the Twitter API.
The Tweet timestamp must be displayed and link to the Tweet's permalink.
Tweet Actions (Reply, ReTweet, Like) "must always be visible for the user to interact with the Tweet" and "must be implemented using Web Intents of with the authenticated Twitter API".
Tweet Actions must appear separate from any other social network actions.
If displaying a Retweet, the Retweeter's Full Name must be displayed and linked to their profile, and the Retweet icon must be displayed.
"The offical Twitter logo must always be reasonably visible and displayed on the individual Tweet or directly attached to the timeline".
All timelines must let the user view the details of the original Tweet.
Grouped Tweets in a timeline must have non-Twitter content mixed with it. This appears to be only related to automatically created timelines and user created Twitter timelines.
Not applicable for a Web site, but on native mobile apps (i.e. iOS apps, Android apps, probably not HTML5 apps) all Twitter deep links (profiles, Retweet, etc.) must direct to the Twitter app (i.e. use twitter: URIs).

That is an awful lot of rules, and essentially says that if you want to use Tweets on a Web page, they have to essentially look like an embedded Tweet would.

Perhaps the simplest way of doing things without going the API method would be to screenshot the embed code on Twitter and, using a figcaption, add the HTML code minus the JavaScript and class, add the fi-social-twitter class, and replace any link text with the value of data-expanded-url.

Screenshot of Tweet — My strawberry plants are all planted. https://web.johncook.uk/blogs/gardening/growing-strawberries-planting-part-2
— John Cook (@WatfordJC) October 31, 2015

Something I would have to do is revisit this to add the proper Microdata.

Another thing that might be worth looking at, as an alternative to using the Twitter API, is whether image maps might be suitable. I haven't used image maps on a Web site in over 15 years, but linking retweet to the Web Intents URL would add back some functionality (and improve compliance with the Display Requirements).

Of course, the question then becomes "is it going to be easier to use the Twitter API server-side than it is to image map tweets?"

As an image map:

The problem with creating image maps for each tweet I wish to embed is that I would have to manually create them. The last time I used image maps was over 15 years ago because they are just too much hassle.

Note to self: the Brand Guidelines also state I must not change the colour of the bird. The Twitter Brand Assets page says Twitter Blue is #55acee or RGB(85,172,238). I therefore need to add to the site CSS .fi-social-twitter:before { color: rgba(85,172,238,1); }. Fixed in commit ad91bf6a354053071881b100ccffd4fecced3d1a.

Facebook Embeds

Facebook don't appear to have a policy like Twitter's Display Requirements. From the look of things you either have to use the embed method or the screenshot method. Facebook do have rules on using posts elsewhere, but it isn't as clear as Twitter's rules.

For the embed method, it is possible the JavaScript isn't a requirement. Not including the JavaScript falls back to a text-only blockquote. For the screenshot method, there are many rules (the most important of which seems to be the post (and any replies) must be public and no personal information is included).

I would link to Facebook's rules, but as I have said they are not easily accessible (i.e. human comprehensible) and I would probably be linking to the wrong thing.

Given the uncertainty of the rules, and the fact Facebook use Web beacons if you use their embed JavaScript, I will remove the Facebook JavaScript from this site, and replace embeds of my own Facebook posts with screenshots of them instead,

For the posts of others I will just link to the Facebook post with rel="noreferrer nofollow" (the nofollow purely because the post I link to could be edited, and the comments could become a spam attraction).

So, as with the Twitter screenshot, I will screenshot the preview of the Facebook embed, and then use the embed code minus the JavaScript and divs. For the time being I am also removing the blockquotes until I get around to modifying the CSS for blockquotes inside figcaptions.

Screenshot of Facebook post — My strawberry plants are all planted.
Posted by John Cook on Saturday, 31 October 2015

Speaking of CSS, I need to work out how best to deal with the white space to the right of the <figure>s, if it is even possible.

YouTube Embeds

YouTube is tricky. It will involve using JavaScript/jQuery and possibly the YouTube API.

What I have decided to do is the following:

Download to /img/youtube/ https://img.youtube.com/vi/[video-id]/[thumbnail]default.jpg, where [video-id] is the value of the v attribute of a YouTube video (non-playlist) and [thumbnail] is mq for 16:9 and sd for 4:3.
Create a flex-video div with an id of youtube-[video-id] and a data-video value of [video-id].
Create a hyperlink of class button with an id of load-youtube-[video-id] and a data-id value of the id of the flex-video div (i.e. youtube-[video-id]).
If the same video is being embedded more than once on the page, append to the flex-video and button id -[x] where x is an increasing number for each copy of that video id (or anything else that would make the IDs unique on the page).
Create a hyperlink of class button that links to the YouTube video (for "Watch Video on YouTube").
Add a click event listener to all load-youtube-* IDs to call a function that swaps out the image for the embed iframe.
Add a function that does the image/iframe swap.

For example, the video of me shaving with a straight razor would have the following HTML code:

<div class="flex-video widescreen" id="youtube-z1qrn4t9pbQ" data-video="z1qrn4t9pbQ">
<img src="https://web.johncook.uk/img/youtube/z1qrn4t9pbQ.jpg" alt="Thumbnail of YouTube video: Straight Razor Shave" title="Straight Razor Shave">
</div>
<p>Video: Straight Razor Shave</p>
<a class="button" id="load-youtube-z1qrn4t9pbQ" data-id="youtube-z1qrn4t9pbQ">Play Video Embedded</a>
<a class="button" href="https://www.youtube.com/watch?v=z1qrn4t9pbQ" rel="noreferrer nofollow">Watch Video on YouTube</a>

The LoadYouTubeVideo JavaScript function looks like this:

function LoadYouTubeVideo(placeholderID,button) {
  if (window.jQuery) {
    var placeholder = $("#"+placeholderID);
    placeholder.html('<iframe src="https://www.youtube.com/embed/'+placeholder.data("video")+'?autoplay=1" frameborder="0" allowfullscreen sandbox="allow-same-origin allow-scripts allow-popups"></iframe>');
	button.addClass("disabled");
  }
}

And the JavaScript to add the event listener(s) to the button(s) looks like this:

		setTimeout(function(){$("[id^='load-youtube-']").each(function(index){
			var placeholder = $(this).data("id");
			this.addEventListener('click',function(){LoadYouTubeVideo(placeholder,$(this))},false)
		});},300);

Combined, that produces the following:

Video: Straight Razor Shave

Play Video Embedded Watch Video on YouTube Google Privacy Policy Google Cookies

The img has a width of 100% so that it uses exactly the same space as the video embed would if loaded.

It is also likely to be a lower resolution that what YouTube embedding would load for a HD video due to using mqdefault.jpg and sddefault.jpg. It is stretched to show where the embedded video will be, but it uses less bandwidth than a 1080p thumbnail would and means I can skip the process of getting the highest resolution thumbnail and downscaling it for different resolutions.

I have added a robots.txt directive to not index images in /img/youtube/ (as well as /img/twitter/ and /img/facebook/) so that search engines don't index the images.

The Watch Video on YouTube button not only prevents YouTube from knowing what Web site/page you are on, but as it is a hyperlink you can (depending on OS/browser) drag it to the tab list and open it in a new tab, drag it to another window, drag it into a word processor or text editor, or any number of things without having to open the video to get its URL.

While I could use the YouTube API to add additional functionality (such as an "Add to Watch Later List" button), that is the sort of functionality I am removing from my site.

Vimeo Embeds

This is rather easy as I have already done it for YouTube.

All I have to do for Vimeo embeds is modify my YouTube code:

<div class="flex-video vimeo widescreen" id="vimeo-4958098" data-video="4958098">
<img src="https://web.johncook.uk/img/vimeo/4958098.png" alt="Thumbnail of Vimeo video: Leonard Grigoryan reviews the Kinny Stereo Acoustic Guitar for Gizmag" title="Leonard Grigoryan reviews the Kinny Stereo Acoustic Guitar for Gizmag">
</div>
<p>Video: Leonard Grigoryan reviews the Kinny Stereo Acoustic Guitar for Gizmag</p>
<a class="button" id="load-vimeo-4958098" data-id="vimeo-4958098">Play Video Embedded</a>
<a class="button" href="https://vimeo.com/4958098" rel="noreferrer nofollow">Watch Video on Vimeo</a>

Vimeo do not have standard thumbnail URLs for videos. First you need to visit https://vimeo.com/api/v2/video/[video-id].xml to get the XML file using the API (at the moment the XML file is downloaded rather than displayed in my browser). I then download thumbnail_large (replacing .webp with .png) to [video-id].png in /img/vimeo/ on my server. Again, I have added /img/vimeo/ to robots.txt so search engines don't index it.

The LoadVimeoVideo JavaScript function looks like this:

function LoadVimeoVideo(placeholderID,button) {
  if (window.jQuery) {
    var placeholder = $("#"+placeholderID);
    placeholder.html('<iframe src="https://player.vimeo.com/video/'+placeholder.data("video")+'?autoplay=1" frameborder="0" allowfullscreen sandbox="allow-same-origin allow-scripts allow-popups"></iframe>');
	button.addClass("disabled");
  }
}

And the JavaScript to add the event listener(s) to the button(s) looks like this:

		setTimeout(function(){$("[id^='load-vimeo-']").each(function(index){
			var placeholder = $(this).data("id");
			this.addEventListener('click',function(){LoadVimeoVideo(placeholder,$(this))},false)
		});},400);

Combined, that produces the following:

Video: Leonard Grigoryan reviews the Kinny Stereo Acoustic Guitar for Gizmag

Play Video Embedded Watch Video on Vimeo Vimeo Privacy Policy Vimeo Cookie Policy

Something I could do later is merge the YouTube and Vimeo JavaScript to reduce the amount of code needed.

Google Custom Search Engine

Site Search is still something that needs work on this site, and I'm not sure which way I'm going to go with it.

For the time being I am going to leave things as they are with Google CSE.

Accessibility

By switching to images (or other methods) I also need to think about accessibility.

Images should have alt tags. For YouTube and Vimeo videos I will just use "Thumbnail of [YouTube/Vimeo] video: [video title]".

For Twitter and Facebook screenshots, I am not sure what alt text to use.

If I am linking to the screenshot, I think I will simply use "screenshot of [Tweet/Facebook post]" if I include a figcaption, and the Tweet/Post text (without URLs?) if I don't include a figcaption.

Further research is needed on my part to determine if (when using a Tweet without including Twitter Cards) if all the extra stuff is actually needed. Does the Tweet text have to come below the Username or is a variation of the blockquote displayed for an embedded Tweet when JavaScript is disabled OK? Are the avatar, follow, reply, retweet, and like buttons/links/images required?

Image maps, although potentially making images of Tweets more accessible, are a pain to work with. I believe my plain image (with alt text and/or figcaption) would be the better route to take in terms of accessibility.

Conclusion

It will take some time to update the entire site to use these new methods, so the third party JavaScript will remain for a while.

Something occurred whilst I was writing this article: connect.facebook.com went down. As a result of that this page stopped loading properly (because the Facebook stuff didn't timeout straight away) and the Play Video Embedded buttons stopped working.

Such an issue will no longer be a problem once I switch away from automatically embedded third party content, and once everything on this page has been implemented site-wide things will only stop working if something like Google's CDN goes down for jQuery.

As for optionally embedded content not working, if a YouTube embed button doesn't seem to work a visitor can click the Watch Video on YouTube button. If that doesn't work the visitor will wonder 'is YouTube down?'

I still need to work out a way to not automatically embed the JavaScript on pages using Popcorn.js.