SPA woes

For some time now I have wanted the ability to route paths for a gh-pages site to its index.html for handling as a single-page app. This ability is table stakes for single-page apps because you need all requests to be routed to one HTML file, unless you want to copy the same file across all your routes every time you make a change to your project. Currently GitHub Pages doesn’t offer a route handling solution; the Pages system is intended to be a flat, simple mechanism for serving basic project content.

If you weren’t aware, GitHub does provide one morsel of customization for your project site: the ability to add a 404.html file and have it served as your custom error page. I took a first stab at doing an SPA hack by simply copying my index.html file and renaming the copy to 404.html. Turns out many folks have experienced the same issue with GitHub Pages and liked the general idea: https://twitter.com/csuwildcat/status/730558238458937344. The issue that some folks on Twitter correctly raised was that the 404.html page is still served with a status code of 404, which is no bueno for crawlers. The gauntlet had been thrown down, but I decided to answer, and answer with vigor!

One more time, with feeling

After sleeping on it, I thought to myself: “Self, we’re deep in fuck-it territory, so why don’t I make this hack even dirtier?!” To that end, I developed an even better hack that provides the same functionality and simplicity, while also preserving your site’s crawler juice – and you don’t even need to waste time copying your index.html file to a 404.html file anymore! The following solution should work in all modern desktop and mobile browsers (Edge, Chrome, Firefox, Safari), and Internet Explorer 10+.

Template & Demo: If you want to skip the explanation and get the goods, here’s a template repo (https://github.com/csuwildcat/sghpa), and a test URL to see it in action: https://csuwildcat.github.io/sghpa/foo/bar

That’s so META

The first thing I did was investigate other options for getting the browser to redirect to the index.html page. That part was pretty straight forward, you basically have three options: server config, JavaScript location manipulation, or a meta refresh tag. The first one is obviously a no-go for GitHub pages, and JavaScript is basically the same as a refresh, but arguably worse for crawler indexing, so that leaves us with the meta tag. Setting a meta tag with a refresh of 0 appears to be treated as a 301 redirect by search engines, which works out well for this use-case.

You’ll need to start by adding a 404.html file to your gh-pages repo that contains an empty HTML document inside it – but your document must total more than 512 bytes (explained below). Next put the following markup in your 404.html page’s head element:

<script>
  sessionStorage.redirect = location.href;
</script>
<meta http-equiv="refresh" content="0;URL='/REPO_NAME_HERE'"></meta>

This code sets the attempted entrance URL to a variable on the standard sessionStorage object and immediately redirects to your project’s index.html page using a meta refresh tag. If you’re doing a Github Organization site, don’t put a repo name in the content attribute replacer text, just do this: content="0;URL='/'"

Customizing your route handling

If you want more elaborate route handling, just include some additional JavaScript logic in the script tag shown above to tweak things like: the composition of the href you pass to the index.html page, which pages should remain on the 404 page (via dynamic removal of the meta tag), and any other logic you want to put in place to dictate what content is shown based on the inbound route.

512 magical bytes:

This is hands down one of the strangest quirks I have ever encountered in web development: You must ensure the total size of your 404.html page is greater than 512 bytes, because if it isn’t IE will disregard it and show a generic browser 404 page instead. When I finally figured this out, I had to crack a beer to help cope with the amount of time it took.

Let’s make history

In order to capture and restore the URL the user initially navigated to, you’ll need to add the following script tag to the head of your index.html page before any other JavaScript acts on the page’s current state:

This bit of JavaScript retrieves the URL we cached in sessionStorage over on the 404.html page and replaces the current history entry with it. However you choose to handle things from there is up to you, but I’d use popstate and hashchange if you can.

Well folks, that’s it – now go hug it out and celebrate by writing some single-page apps on GitHub Pages!

GIF

The battle ground of app-to-app interaction history is littered with abandoned ideas, half-solutions, and unimplemented APIs. The current, consumer/provider interaction paradigm for apps and services is a mess of one-off, provider-defined systems that each use their own transaction mechanisms and custom data structures. This makes it hard to do simple things across N providers, like save something to a user’s preferred storage service without jumping through provider-specific code and UX hoops.

I’d like to restart the conversation about bringing legit, app-to-app interaction APIs to the Web. There have been past spec attempts, namely Web Activities and Web Intents, but I’ll argue that while they get a lot right, they all fail to deliver an A+ solution.

Wait, what is an “App-to-App Interaction API”?

Have you ever wanted to open a programmatic connection within your app to the user’s preferred provider of a service or activity? Maybe you’ve wanted to check a user’s identity, store data to a user’s storage provider, or post to the user’s preferred social network, etc. If you’re saying to yourself, “Self, this is already solved, there are endless provider-specific social, identity, and storage includes to do just that” <– I would submit to you, that right there, sucks. There must be some way to provide a common API layer for consumption of these services that doesn’t require developers to know of, integrate, and manage specific bits of UI and code for every provider. Not having legit APIs to do this is a real problem, with significant consequences:

• Pages are cluttered with code and UI includes for countless providers, many of which are left on pages to rot after a company discontinues support.
• Developers must know of, include, maintain, and write code to juggle how and what providers respond with; which usually doesn’t share a semantic, even across the same type of service.
• The performance cost is huge – social, login, and other provider-specific includes account for a staggering amount of memory and dramatically increase page load times.
• Because developers are required to specifically include each provider, they often display only two or three options for a given service. This prevents users from leveraging different services that aren’t among those a developer has committed cycles to integrating.

So we know this is a serious issue that impacts countless uses-cases, degrades performance, and limits service mashability, but what does a serious solution look like?

Must-haves of an A+ solution:

1. Ability for providers to be registered dynamically, or via manifest declaration, but not tied to any installable app model – the whole Web deserves this API.
2. The user’s selection and preferences for servicing activities should be managed by the User Agent (browser, OS, etc.), within its trusted, chrome UI.
3. The user should have the ability to pick a service/app to handle an activity per-use, as a default per-domain, or as a global default.
4. The developer should be able to request handling of an activity via context switch to the provider app OR by establishing a silent, programmatic connection (connecting to another app/service in the background)
5. The spec should provide both declarative and imperative entry points for UI hand-offs and programmatic connections – for instance: links for declarative hand-offs, and imperative interfaces for background connections.
6. Imperative API connections to providers should be transactional – meaning a response sent back from a provider should be tied to a specific request instance (hopefully with a Promise).

How does current tech stack up?

Here’s a rundown of how the three existing mechanisms we have for app-to-app interaction fare when assessed against the base features and capabilities an A+ solution should include:

Though Custom Protocols come closest – and could be enhanced to fill the gaps – none of these options meet the criteria of an A+ solution. To add insult to injury, no current option is supported by default mobile browsers (Android, iOS, or Windows Phone), which makes general use a non-starter for most developers. The inability of these APIs to cover significant developer use-cases is one reason implementation of the above APIs is so spotty. Neither IE nor Edge currently support these APIs, and I don’t blame them – before you commit significant time to implementing an API, you need to be sure it adequately solves the problems you face. That’s part of the reason I am starting this discussion; we want to create an open dialog to better understand your needs.

Experimenting with Protocol Handlers

I needed an app-to-app interaction API for something I’m working on at Microsoft, called Web Profile. The requirement was that any consuming app would be able to open a programmatic connection to a user’s preferred service provider and perform transactions without the consuming app having to know where to look, or include any provider specific code.

To build a working prototype, I needed an existing mechanism that worked across multiple browsers, so I took another look at something I had used in the past: Custom Protocol Handlers (MDN: navigator.registerProtocolHandler). Though they may not realize it, users and developers are already familiar with custom protocol handlers; the best example is probably mailto:. URLs prefixed with a custom protocol direct the browser to open the app or site the user selects to handle the request. This usually takes the form of a contextual switch: opening a new window or browser tab. For what I was looking to do, I needed a way to stay within my app while interacting over a programmatic connection with whatever provider the user selects. Custom protocol handlers can currently be registered in the desktop versions of these browsers:

• Firefox 3+
• Chrome 13+
• Opera 11.6+

Meet ProtocolWorker

It turns out you can use custom protocols to create a programmatic, background connection for app-to-app interactions. How? By pairing them with an old friend: the iframe. I found that loading a custom protocol (any handler registered with the web+ prefix) in an iframe will redirect to the handler’s target page. With a bit of postMessage() magic handled by a prollyfill included on both sides (consumer and provider), you can create an active, two-way connection between your app and the user’s preferred service provider. Think of it as a cross between a Web Worker and Background Page (as they’re known in browser extension land). Here’s my Github repo with the JS file you’ll need to make it all work: https://github.com/csuwildcat/ProtocolWorker

Talk codey to me:

To show how this works I put together a simple demo built on the totally-100%-made-up web+whisper protocol. The hypothetical Web Whisper case I highlight in the code below lets a user store a text string (a whisper) with their provider, or retrieve the last text string the provider stored on their behalf. Any consuming app that desires to store or retrieve the user's whisper must first be permitted to do so via the browser's protocol selection UI. The whole demo uses localStorage to save the user's text string on the provider's domain, so don't expect persistence beyond that of your local machine.

Note: though this is a super simple demo, the system would be just as good for sending/retrieving things like: social posts, user profile info, or files from places like OneDrive, Dropbox, Box, etc.

Providers register a protocol handler

The first thing a site offering a Web Whisper service needs to do is register itself as a handler for the web+whisper protocol. Here's how you do that:

navigator.registerProtocolHandler('web+whisper', HANDLER_URL + '?%s', 'Whisper Demo Provider');

Protocol handlers point to a ProtocolWorker page

At the other end of that HANDLER_URL should be an HTML page that is not intended for display - we'll be treating it as more of a DOM-enhanced, Web Worker environment. In this page you will include event listeners and whatever code you need to deal with the requests you'll receive from consuming apps. Here's what the handler page looks like for our Web Whisper provider:

<script src="js/ProtocolWorker.js"></script>

<script></p>
<p>  window.addEventListener('protocolrequest', function(event){<br />
    if (event.detail['@type'] == 'Comment') {<br />
      localStorage.whisper = event.detail.comment;<br />
      event.detail.resolve();<br />
    }<br />
    if (event.detail['@type'] == 'Search') {<br />
      event.detail.resolve(localStorage.whisper);<br />
    }<br />
  });</p>
<p></script>

Consuming apps connect to providers via ProtocolWorker

After the user adds our Web Whisper provider as a handler for web+whisper requests, consuming apps can ask to open a connection to it. Below is a demo page of what it would look like for a consuming app to interact with the user's Web Whisper provider. Notice the consuming app creates a ProtocolWorker instance and sets up events in which it will attempt to send and retrieve data from the user's Web Whisper provider.

<input type="text" placeholder="Whisper something" /><button id="send">Send</button><button id="retrieve">Retrieve</button>

<script src="js/ProtocolWorker.js"></script>
<script></p>
<p>var retrieveButton = document.querySelector('#retrieve');<br />
var sendButton = document.querySelector('#send');<br />
var input = document.querySelector('input');</p>
<p>var whisperWorker = new ProtocolWorker('web+whisper');</p>
<p>sendButton.addEventListener('click', function(){</p>
<p>  var value = input.value.trim();<br />
  if (value) {<br />
    whisperWorker.request({<br />
      '@context': 'http://schema.org',<br />
      '@type': 'Comment',<br />
      comment: value<br />
    }).then(function(){<br />
      alert('Your message has been whispered to your provider!')<br />
    }).catch(function(){<br />
      alert('You either declined to whisper something to me or don\'t have a web+whisper provider :/')<br />
    });<br />
  }<br />
  else alert('You didn\'t whisper anything.');</p>
<p>});</p>
<p>retrieveButton.addEventListener('click', function(){</p>
<p>  whisperWorker.request({<br />
    '@context': 'http://schema.org',<br />
    '@type': 'Search'<br />
  }).then(function(data){<br />
    alert('You last whispered: ' + data);<br />
  }).catch(function(){<br />
    alert('You either decided not to let this app see your last whisper, or don\'t have a web+whisper provider :/')<br />
  });</p>
<p>});</p>
<p></script>

Transactions should follow shared semantics

As you may have noticed above, each ProtocolWorker request above is formatted in accordance with a matching data description from Schema.org. Schema.org is a great resource for semantic data descriptors, which I believe are a solid Lingua-franca upon which to transact custom protocol requests. Who knows, maybe browsers could validate and enforce these semantic data structures some day?

Just show me the demo, clown.

I have the demo code above running on a couple Github domains so you can test the interplay between providing and consuming apps – note: you need to be using a desktop version of Firefox, Chrome, or Opera. Here are the steps to follow:

1. Add the demo provider as your Web Whisper handler by clicking the “Add Provider” button, and accepting the handler via the browser’s permission UI: http://web-protocols.github.io/provider-demo/
2. Visit the consuming app demo page and whisper something to your provider by entering a message in the input and clicking the send button. It will prompt you to select a provider, make sure you pick the “Whisper Demo Provider” you just added: http://csuwildcat.github.io/ProtocolWorker/
3. OPTIONAL: You’ll notice an option to make your provider choice the default when the browser selection UI is raised in the consuming app demo. If you select a provider as your default, all subsequent ProtocolWorker calls will happen instantly in the background, without bugging you to choose a handler.

Christian Heilmann was kind enough to shoot a quick screen cast of the steps above:

Where do we go from here?

As we’ve discussed, there are new and existing APIs and specs that could be modified to enable these use-cases – ServiceWorker could be a candidate, but regardless of what form the solution takes, let’s come together as a community of developers and platform implementers to move the web forward. I encourage you to submit your own requirements, technical feedback, and implementation ideas directly to this Github repo: https://github.com/csuwildcat/ProtocolWorker/issues/1