Cloud Four Blog

Technical notes, War stories and anecdotes

Service Workers at Scale, Part II: Handling Fallback Resources

Part I of this series established that some of the challenges in building service workers for complex websites stem from having to accommodate page request variations. In this iteration, we’ll touch on how to handle similar variations in the acquiring and delivering of fallback resources.

Pre-caching fallback dependencies

A familiar pattern for caching dependencies is to request them in bulk during the installation stage. We began with this approach in our own service worker, specifying path strings for all fallback resources:

addEventListener('install', event => {
  event.waitUntil(
    caches.open('dependencies')
      .then(cache => {
        return cache.addAll([
          '/offline',
          '/books-offline',
          '/assets/offline-avatar.png',
          '/assets/offline-photo.png'
        ]);
      )
      .catch(err => console.warn(err))
      .then(skipWaiting())
  );
});

Cache.addAll() converts each element of the array it receives into a new Request object, so you can conveniently pass it an array of strings. This simple approach worked well during local development, but there were some issues on our more complicated test server. We needed more control over the requests made by Cache.addAll(), so we created them explicitly:

cache.addAll([
  new Request('/offline', {...}),
  new Request('/books-offline', {...}),
  new Request('/assets/offline-avatar.png', {...}),
  new Request('/assets/offline-photo.png', {...})
]);

Constructing our own requests gave us the ability to override their default options. The necessity to supply these options became obvious once we saw 401 responses to our fallback page requests. The server had enabled HTTP auth, so requests sent from our worker in attempt to pre-cache resources were failing.

The credentials option solved this problem, allowing our requests to break through the authentication barrier:

cache.addAll([
  new Request('/offline', {credentials: 'same-origin'}),
  // ...
]);

We also decided to use the cache option for future-proofing. This will be useful for controlling how requests interact with the HTTP cache. While it currently only works in Firefox Developer Edition, we included it to make sure pre-cached responses are fresh1:

cache.addAll([
  new Request('/assets/offline-photo.png', {cache: 'reload'}),
  // ...
]);

For an overview of other cache option values, check out Jake Archibald’s article with various examples of service worker and HTTP cache interoperability.

Responding with fallback images

Our pre-cache items include generic images intended to serve as “fallbacks” for unfulfilled requests. We needed to account for two different fallback image types, each with their own visual treatment:

  1. Images embedded in articles
  2. Avatars for authors and commenters

To determine which fallback should be used for a given request, we associated each with a URL hostname:

const fallbackImages = new Map([
  [location.hostname, '/assets/offline-photo.png'],
  ['secure.gravatar.com', '/assets/offline-avatar.png']
]);

Using a map like this, we can conveniently lookup the proper fallback based on the URL of an unfulfilled request:

function matchFallback (req) {
  const {hostname} = new URL(req.url);
 
  if (isImageRequest(req)) {
    const image = fallbackImages.get(hostname);
    return caches.match(image);
  }
 
  // ...
}

“Redirecting” to offline pages

As with our fallback images, we also needed to accommodate a bit of variation in our handling of fallback pages. Some pages that we wanted to make available offline had too many images to justify pre-caching. In these cases, simplified versions of those pages (minus the images) were created to use as substitutes, as if they were redirected.

Because all of the pages with offline variations are local, they can be mapped by their URL pathname, and incorporated into our matchFallback() handler accordingly:

const fallbackPages = new Map([
  ['/books', '/books-offline'],
  ['/workshops', '/workshops-offline']
]);
function matchFallback (req) {
  const {hostname, pathname} = new URL(req.url);
 
  if (isImageRequest(req)) {
    const imagePath = fallbackImages.get(hostname);
    return caches.match(imagePath);
  }
 
  if (isPageRequest(req)) {
    const pagePath = fallbackPages.get(pathname);
    return caches.match(pagePath);
  }
 
  // Use an new response if nothing better can be found.
  return Promise.resolve(new Response(/* ... */));
}

Coming up next: Cache trimming and invalidation

In the next part of this series, we’ll cover strategies for invalidating old caches and limiting the amount of storage space they can occupy.


  1. To fill in for the sparse browser implementation, it’s recommended to use some form of cache-busting when pre-caching static resources. 

Breaking Out With Viewport Units and Calc

While iterating on a new article layout for the impending Cloud Four redesign, I encountered an old CSS layout problem.

For long-form content, it’s usually a good idea to limit line lengths for readability. The most straightforward way to do that is to wrap the post content in a containing element:

.u-containProse {
  max-width: 40em;
  margin-left: auto;
  margin-right: auto;
}
<div class="u-containProse">
  <p>...</p>
  <p>...</p>
</div>

But what if we want some content to extend beyond the boundaries of our container? Certain images might have greater impact if they fill the viewport:

Mockup of container with full-width element

In the past, I’ve solved this problem by wrapping everything but full-width imagery:

<div class="u-containProse">
  <p>...</p>
</div>
<img src="..." alt="...">
<div class="u-containProse">
  <p>...</p>
</div>

But adding those containers to every post gets tedious very quickly. It can also be difficult to enforce within a content management system.

I’ve also tried capping the width of specific descendent elements (paragraphs, lists, etc.):

.u-containProse p,
.u-containProse ul,
.u-containProse ol,
.u-containProse blockquote/*, etc. */ {
  max-width: 40em;
  margin-left: auto;
  margin-right: auto;
}

Aside from that selector giving me nightmares, this technique might also cause width, margin or even float overrides to behave unexpectedly within article content. Plus, it won’t solve the problem at all if your content management system likes to wrap lone images in paragraphs.

The problem with both solutions is that they complicate the most common elements (paragraphs and other flow content) instead of the outliers (full-width imagery). I wondered if we could change that.

Flipping the Script

To release our child element from its container, we need to know how much space there is between the container edge and the viewport edge… half the viewport width, minus half the container width. We can determine this value using the calc() function, viewport units and good ol’ percentages (for the container width):

.u-release {
  margin-left: calc(-50vw + 50%);
  margin-right: calc(-50vw + 50%);
}

Voilà! Any element with this class applied will meet the viewport edge, regardless of container size. Here it is in action:

See the Pen Full-width element in fixed-width container example by Tyler Sticka (@tylersticka) on CodePen.

Browsers like Opera Mini that don’t support calc() or viewport units will simply ignore them.

One Big, Dumb Caveat

When I found this solution, I was thrilled. It seemed so clever, straightforward, predictable and concise compared to my previous attempts. It was in the throes of patting myself on the back that I first saw it…

An unexpected scrollbar:

On any page with this utility class in use, a visible vertical scrollbar would always be accompanied by an obnoxious horizontal scrollbar. Shorter pages didn’t suffer from this problem, and browsers without visible scrollbars (iOS Safari, Android Chrome) seemed immune as well. Why??

I found my answer buried deep in the spec (emphasis mine):

The viewport-percentage lengths are relative to the size of the initial containing block. When the height or width of the initial containing block is changed, they are scaled accordingly. However, when the value of overflow on the root element is auto, any scroll bars are assumed not to exist. Note that the initial containing block’s size is affected by the presence of scrollbars on the viewport.

Translation: Viewport units don’t take scrollbar dimensions into account unless you explicitly set overflow values to scroll. But even that doesn’t work in Chrome or Safari (open bugs here and here).

I reacted to this information with characteristic poise:

Not reacting with poise at all

Luckily, a “fix” was relatively straightforward:

html,
body {
  overflow-x: hidden;
}

It’s just a shame that it’s even necessary.

Autofill: What web devs should know, but don’t

Many people know that you can scan your credit credit in Mobile Safari. But how many web developers know how to create a form that supports that feature?

Not many I’d bet.

Photo of someone scanning their credit card

Photo source: Auto-Fill Credit Card Forms Using Your iPhone’s Camera in iOS 8. Used with permission.

It doesn’t help that Apple has provided zero documentation on how the scan credit card feature works.

But there is another factor at play here. The scan credit card feature is a subset of browser functionality that web developers have long ignored: autofill.

It’s understandable why web developers haven’t paid much attention to autofill. When you’re filling out forms with test data on a regular basis, autofill tends to get in the way.

But autofill is an important feature for our users. Google has found that “users complete forms up to 30% faster” when using autofill.

So let’s learn how autofill works, how to build forms that support cross browser autofill, and take advantage of new features like scanning credit cards.

How does autofill work?

Until recently, there were no standards when it came to autofill. Each browser implemented their autofill features differently and there was little documentation on how a browser determines what content a field expects.

Despite this chaotic situation, browsers seem to have settled on two main approaches:

1. Pre-determined autofill fields

Chrome, Opera and Safari have all taken the approach of identifying high-value form fields and providing a way to manage what the browser will autofill for those fields.

Opera Autofill Manager

For example, Opera provides autofill for addresses and credit cards. These can be managed in the preferences as shown above.

Chrome, Opera and Safari all differ on which fields they provide autofill for, but the basic fields needed to complete a checkout process are well supported.

Most users never have to see or edit these preferences in order to utilize autofill. The browser watches the person filling out forms and when it recognizes an address or a credit card, it will ask if the user wants them to save that information to reuse later.

2. Autofill any field

If the previous approach is like a scalpel applied to preselected fields only, this second approach is a chainsaw cutting down every field in view.

When a form is submitted, Microsoft Edge and Firefox will store the value submitted along with the value of the name attribute. If the browser sees a field in the future with a matching name attribute, it will provide autofill options. Firefox also appears to look at the id in addition to the name attribute

Because there are security and privacy concerns with this approach, the autocomplete off value has long been supported to prevent the browser from storing and autofilling sensitive information.

Which approach is better?

While the second approach works for more fields, as a developer, I much prefer the pre-determined autofill fields approach.

It makes it much easier to figure out what information the browser has to autofill. It is also easier to set up test profiles.

Plus, with the second approach, you actually need to submit a form in order for the browser to store values to use with autofill. The browser won’t remember your answers without the form submission.

It also makes me nervous to think that the browser might store credit card information in a non-encrypted way if it can’t clearly identify the type of field.

Given how concerned Microsoft and Mozilla are about security and privacy, I’m certain they’ve put protections in place. But I personally feel more secure looking at an autofill preferences pane and seeing credit card information clearly separated and understood by the browser.

All that said, I don’t know what end users prefer. The second system works in more places, but I’ve seen quite a few support questions where people have been trying to remove their autofill options from the browser history.

It will be interesting to see how Edge and Firefox change when they began to support the new autofill standard.

One behavior to watch for

Sometimes browsers require more than one field of a certain type before they will present you with autofill options.

For example, in the following animated GIF, Safari won’t offer to autofill the single cardholder field, but it will offer to autofill it once there is a second card number field.

Safari Autofill working with two fields, but not working with one field

However, if the only field being collected is the card number, then Safari will offer the autofill option.

My experience has been that creating isolated test cases for single fields can be challenging because of this behavior. At one point in my testing, I encountered Opera requiring three fields before it would autofill, but I can no longer recreate that behavior.

This should never be an issue for a user so long as your form is written to support autofill (more on this soon), but I note it here in case you’re attempting to troubleshoot autofill and encounter this behavior.

The standards-based approach to autofill

Thankfully, there is a way forward for autofill. HTML5 recently expanded the autocomplete attribute to provide hints to the browser about what content a field expects.

The autocomplete attribute has been around for several years. It started with two values: on and off. By default, the autocomplete attribute is set to on which means that the browser is free to store values submitted and to autofill them in the form.

However, some fields are poor candidates for autofill behavior. In that case, the autcomplete attribute can be set to off to tell the browser that this field should be not autofilled.

Recently, additional values have been added as options for the autocomplete attribute. These new options are called autofill detail tokens. These tokens tell the browser what information the field is looking for.

One type of token is called the autofill field names. The autofill field names tell the browser what type of information a field expects.

For example, one of the autofill field names is organization. The HTML5 specification says that organization refers to:

Company name corresponding to the person, address, or contact information in the other fields associated with this field

If you wanted to tell the browser to autofill the organization field, your code would look something like this:

<input type="text" name="foo" id="bar" autocomplete="organization">

The HTML5 specification has a great table listing all 53 possible autofill field names, what their intended use is, and what type of input must be used with it.

That’s autocomplete at its simplest, but it gets more powerful and a bit more complex.

Shipping and billing

The value of the autocomplete attribute is actually a space-separated list of tokens. So for example, if you wanted to collect shipping information, you would prepend the autocomplete value with the “shipping” token like so:

<textarea name="shipping-address" autocomplete="shipping street-address"></textarea>
<input type="text" name="shipping-city" autocomplete="shipping address-level2">
<input type="text" name="shipping-state" autocomplete="shipping address-level1">
<input type="text" name="shipping-country" autocomplete="shipping country-name">
<input type="text" name="shipping-postal-code" autocomplete="shipping postal-code">

The billing token works exactly the same way as shipping.

Telephones, email and instant messaging

Another token option applies to telephone, emails and instant messaging. For those services, there is an optional token to indicate if the autofill field name is referring to home, work, mobile, fax or pager.

For example:

<input type="tel" name="home-phone" autocomplete="home tel">
<input type="tel" name="work-phone" autocomplete="work tel">
<input type="email" name="home-email" autocomplete="home email">
<input type="url" name="chat" autocomplete="home impp">

Broad versus narrow autofill field names

The specification provides for both broad and narrow autofill field names for many of the types of information.

For example, in addition to the tel field which would be a single input containing a full telephone number, there are also:

  • tel-country-code
  • tel-national
  • tel-area-code
  • tel-local
  • tel-local-prefix
  • tel-local-suffix
  • tel-extension

The specification authors encourage you to use the broad autofill field names as often as possible:

Generally, authors are encouraged to use the broader fields rather than the narrower fields, as the narrower fields tend to expose Western biases. For example, while it is common in some Western cultures to have a given name and a family name, in that order (and thus often referred to as a first name and a surname), many cultures put the family name first and the given name second, and many others simply have one name (a mononym). Having a single field is therefore more flexible.

I agree with this recommendation. And as a practical matter, it means that it is important to pay attention to the table of values and make sure you’re using the right one for the field you’re working with.

Sections

The final feature of the new autocomplete attribute tokens is the ability to declare an arbitrary section to group fields.

A section is defined using a token that starts with section-. What comes after the dash is up to you. The specification provides this example of sections:

<fieldset>
<legend>Ship the blue gift to...</legend>
<label> Address:
  <input name="bc" autocomplete="section-blue shipping street-address">
</label>
<label> City:
  <input name="bc" autocomplete="section-blue shipping address-level2">
</label>
<label> Postal Code:
  <input name="bp" autocomplete="section-blue shipping postal-code">
</label>
</fieldset>
 
<fieldset>
<legend>Ship the red gift to...</legend>
<label> Address:
<input name="ra" autocomplete="section-red shipping street-address">
</label>
<label> City:
<input name="rc" autocomplete="section-red shipping address-level2">
</label>
<label> Postal Code:
<input name="rp" autocomplete="section-red shipping postal-code"> </label>
</fieldset>

All the tokens

Put all together, we’ve now got a much more complex set of tokens for the autocomplete attribute. And the order of the tokens matters.

First, you’re either using on and off values OR you’re using autofill field names. You can’t use them at the same time.

If you’re using the autofill tokens, the order is:

[section-](optional) [shipping|billing](optional) [home|work|mobile|fax|pager](optional) [autofill field name]

And keep in mind that [home|work|mobile|fax|pager] only applies for the telephone, email and chat autofill field names.

The longest possible set autofill token might look something like this:

<label for="foo">Mobile phone for delivery</label>
<input type="text" name="foo" id="foo" autocomplete="section-red shipping mobile tel">

Yay for standards! We’re done, right?

I’m afraid not. My hope is that eventually all of the browsers will support the expanded autocomplete standard, but that’s not the current situation.

I tested browsers on mobile and desktop to see what attributes the autofill appeared to honor. This is what I found:

Browse Version OS ID Name Autocomplete
Chrome 50 OS X 10.11.4 No Yes Yes
Opera 35 OS X 10.11.4 No Yes Yes
Firefox 46 OS X 10.11.4 Yes Yes No
Edge 25 Windows 10 No Yes No
Safari 9.1 OS X 10.11.4 Partial Partial Partial
Safari 9 iOS 9.3.1 Partial Partial Partial

Thus far, only Chrome and Opera clearly support the new autocomplete features.

Safari appears to have partial support, but because they don’t have documentation, I can’t tell if this is intentional or simply a regular expression match that happens to be looking at the autocomplete attribute as well as name and other attributes.

Safari’s strange behavior

Since the release of iOS 8’s credit card scanning feature, web developers have been reading tea leaves trying to divine what combination of markup Safari is looking for.

Some suggest that you have to have specific values in the name field. Others found values in ids work. Even labels seem to matter:

The Cardholder’s name is quite a bit more tricky. We messed around with various IDs for a long time and almost gave up. We couldn’t figure out an ID that would cause Card Scan to populate it. After much frustration we finally discovered that the TEXT of the label associated with the field matters. Setting the label text to “Name on card” was the magic trick.

I’ve done quite a bit of testing, and I’m still not confident that I understand fully what Safari is doing. However, I’ve seen enough to be able to draw some basic conclusions:

Contact and address fields support autocomplete

Safari recognizes the form I created that only includes autocomplete attributes. The moment I start typing in the first field, it offers to fill in the form with my contact information.

Safari offering to autofill my contact information on a form that only uses autocomplete attributes.

This all works as expected with a couple of caveats.

First, it is unclear what Safari is using to make decisions about what information to autofill from my contact in the Mac’s address book. My job title is filled in, but the company I work for isn’t.

Second, it doesn’t give users the option to select which information they want to use. My contact contains both my home address and my work address. Safari only wants to autofill my home address. I’m out of luck if I want to ship something to the office.

Payment fields are completely wonky

When it comes to the payment fields, Safari’s behavior changes completely. The autocomplete attribute is ignored.

Instead, there is some sort of magical heuristic that Safari is using. And because I’m not an Apple magician, it has been difficult to discern what Safari is actually doing:

Video of me struggling to figure out why some labels for payment fields work and some fail.

In the video above, I edit the labels of two fields. Both have autocomplete set as well as name and id which should help Safari identify the field.

And yet, Safari doesn’t recognize the fields until the labels are changed to Name on Card and Credit Card Number. As mentioned earlier, Safari needs to see more than one field to trigger autofill.

Then I try changing the label to CCNumber which continues to work. However, changing it to CC Number breaks the autofill.

Basically, Safari has an unpublished list of terms that it is searching for. Fortunately, Jacques Caron was able to extract a list of strings from the iOS Simulator that Safari appears to be looking for:

  • card number
  • cardnumber
  • cardnum
  • ccnum
  • ccnumber
  • cc num
  • creditcardnumber
  • credit card number
  • newcreditcardnumber
  • new credit card
  • creditcardno
  • credit card no
  • card#
  • card #
  • cvc2
  • cvv2
  • ccv2
  • security code
  • card verification
  • name on credit card
  • name on card
  • nameoncard
  • cardholder
  • card holder
  • name des karteninhabers
  • card type
  • cardtype
  • cc type
  • cctype
  • payment type
  • expiration date
  • expirationdate
  • expdate
  • month
  • date m
  • date mo
  • year
  • date y
  • date yr

In my experiments, both:

<input type="text" name="nameoncard">
<input type="text" name="ccnumber">

and:

<label for="foo">Name on Card</label>
<input type="text" id="foo" name="foo">
<label for="bar">Credit Card Number</label>
<input type="text" id="bar" name="bar">

will trigger Safari’s autofill and the scan credit card feature on iOS. However, putting the same values into the autocomplete attribute will not work.

Building a cross browser autofill form

Given what we’ve learned, is it truly possible to build a form that supports autofill across browsers? I think it is.

Or at least, we can get very close by taking these four steps.

1. Add autocomplete attributes

This is the future of autofill. Browsers that don’t recognize the values will ignore them.

This is progressive enhancement at its best.

2. Use common values for input name attributes

To take advantage of autofill for Firefox and Edge users, you have to hope that the values you pick for the name attribute match what developers on other sites have used.

One way to do this would be to survey popular sites and see what they use and then use the same values.

Or perhaps use the same values as the autocomplete field in hopes that as more web developers become familiar with the standard that they will start using those names for their fields.

Unfortunately, there is no way to guarantee that your Firefox and Edge users will have previously visited a form that uses the same name values as your form does.

3. Add name and/or label values that match the list Safari is looking for

Using the list that Jacques Caron was able to extract, you can modify the values for the name attribute or the label to match what Safari expects.

4. Make autofill part of your test plans

Lately, I’ve been asking audiences to raise their hands if autofill is part of their test plans. No one does.

I’ve been working on the web since 1996, and I have yet to see autofill as part of the test plan.

I suspect that autofill is a blindspot for web developers and designers. Therefore, it is critical we test our products to ensure that they work well with autofill.

The final form

Here is a sample form that supports autofill on Chrome, Safari, Opera, Firefox and Edge:

See the Pen Cross Browser Autofill Form — Selected Fields by Jason Grigsby (@grigs) on CodePen.

To see it in action, you’ll need to view it on CodePen under https or the browser won’t fill in the credit card information.

I’ve also built a form with all 53 fields in the autocomplete specification. No browser currently supports all 53 fields.

The future of autofill and forms

There is a lot of movement from browser makers towards tackling the problem of web payments. The Payment Request API is being co-authored by Mozilla, Microsoft, Google and Facebook.

Apple is participating in the Web Payments Working Group which is where the Payment Request API is being hashed out. So it appears that Apple is at least nominally onboard with the Payment Request API.

Plus, rumor has it that Apple Pay will be available on mobile web for the holiday shopping season which makes it feel like this new momentum around web payments might be real this time.

This renewed focus on streamling the web checkout process has me optimistic that support for autofill detail tokens will grow in the near term. And these tokens make creating forms that work with autofill so much simpler.

And most importantly, supporting autofill will make filling out forms less tedious for our users and lead to increased sales for e-commerce sites.

One amazing video that shows the potential of the physical web

Michael Mahemoff‘s recent article on how Progressive Web Apps have Leapfrogged the Native Install Model reminded me of a video I discovered while researching my new Adapting to Input talk.

I keep revisiting the video because it shows how the physical web can provide a substantially better experience than the native app install process.

Here’s the video from Matthew Sibigtroth:

Let’s compare the install process for the physical web versus what it would likely take using native apps.

Physical web install process

  1. Look in drawer for physical web beacons.
  2. Select physical web beacon.
  3. Web page launches. Prompts for pairing with toy.
  4. Select toy to complete pairing.
  5. Control toy.

There are two challenges in this process that will be solved with time. First, people need to know about physical web beacons. Second, the physical web and web bluetooth specifications are new and cross browser support is limited.

In fact, this process will likely be streamlined further in the future when the phone automatically notifies the user about nearby beacons.

Native app install process

  1. Confirm that the toy has an app for your platform.
  2. Open app store.
  3. Search for toy. Hope the toy has a unique name so you can find the app easily.
  4. Install the app and enter app store password if necessary.
  5. Wait for app to download.
  6. Open app.
  7. Pair with toy.
  8. Control toy.

Three fewer steps may not seem like much, but the total process is much slower. Apps are usually several megabytes in size making that step alone take much longer than the comparable step for the physical web.

Put a different way, it only takes 18 seconds from when the person first touches the screen to when the toy can be controlled. And that is with the person going slowly so that the people viewing the video can see each step.

It took me nearly 8 seconds on my phone to simply find, launch, and get to the home screen of the app store.

Yes, that will be faster for other people depending on placement of the app store icon, how recently they used the store, and network speed. But it is hard to envision that process ever being faster than simply following the physical web beacon.

The easier install process of the physical web and progressive web apps makes for an exciting future.