Cloud Four Blog

Technical notes, War stories and anecdotes

We Need More Mobile First Browsers. Google Needs to Step Up.

Since the app store was released, people have been suspicious of Apple’s commitment to the open, mobile web. Why would Apple push the mobile web forward when web technology may compete with its native apps platform?

I’m not usually an Apple defender, but if you want to beat up on a company for their laggard mobile browser, look in Mountain View, not Cupertino.

Apple continues to add key functionality to Mobile Safari. The rumors are that iOS 5 will include:

  • position:fixed
  • overflow:scroll
  • date input fields
  • Nitro javascript engine in home screen apps
  • Web workers
  • Native scrolling behavior

Who knows if these will actually ship in the final version of iOS 5, but their inclusion wouldn’t be a surprise. Ever since the initial release of the iPhone, each new version of the operating system has included new browser features. Often these features, like geolocation, show up in Mobile Safari before they show up on the desktop.

This is what I call a mobile first browser. Borrowing from Luke Wroblewski’s mobile first thinking, browser makers need to develop their mobile browsers first.

We Need More Mobile First Browsers

Of the browser makers that have both mobile and desktop versions, which ones have shipped new features on mobile first?

We know Apple has. I suspect Opera has as well. The rest seem to be developing for desktop first.

Recently, Dave Winer argued that Mozilla is no longer creating new features for Firefox that people care about.

What I found most interesting about Dave’s article is that mobile specific features are not mentioned in either his post nor in the comments. The Mozilla folks who defend their browser and its features talk exclusively about the desktop version.

Microsoft’s mobile browser also lags its desktop version. The mango update will bring Internet Explorer 9 to Windows Phone 7 later this year.

Mango will be a great improvement over Internet Explorer 7 which is currently being used, but while the Windows Phone team works on integrating IE9, Microsoft has already released two preview builds of Internet Explorer 10.

Google Needs to Step Up

If there was one company that we would expect to lead the charge for the open, mobile web, it would be Google. At Mobilize 2009, I watched Andy Rubin, SVP of Mobile, say that Google saw HTML5 eventually supplanting most native apps.

Given Google’s emphasis on web technology, a leading edge mobile browser seems like a natural fit. The reality is much different. The Android browser lags behind the iPhone in many ways.

At Google I/O 2010, Vic Gundotra demonstrated browser access to the camera, accelerometer, and the microphone. When Read/Write Web asked me about the demonstration, I responded, “Hell yeah, it’s about time”.

Unfortunately, none of these features shipped with Android’s Gingerbread release. Apple shipped browser access to the accelerometer before Google did. It is over a year later, and we have no idea when we will see them.

Shi Chuan recently wrote a provocative post entitled An Inconvenient Truth – Android and the Open Mobile Web. He may have got some of the details wrong, but the overall sentiment is spot on.

The contrast at Google I/O this year between a day dedicated to Android and a day dedicated to Chrome OS made the problems crystal clear.

For some time I’ve been hearing rumors of battles between the Chrome and Android teams. Seeing the two operating systems on stage made it clear that they are competing visions of the future. Should tablets use Chrome OS or Android?

It seems that the things that people accuse Apple of—slowing the pace of the browser in order to give their app platform an advantage—are actually happening at Google. Competition is generally a good thing, but when the competition is between the Android and Chrome teams, it is the users of the Android browser who lose.

Google needs to step up. Chrome is arguably the best desktop browser. There is no excuse for Android’s browser not leading as well.

Responsive Web Design Book Review

Ethan Marcotte’s new book on Responsive Web Design is both not what I had hoped for and more than I had dared to imagine.

First, let’s get my disappointment out of the way.

As I explained in a previous post, I’m struggling with a chapter on Mobile First Responsive Web Design for our upcoming book. I had secretly hoped that Ethan’s book would contain a never-seen-before blueprint on how to tackle Mobile First Responsive Web Design.

Alas, it doesn’t contain a secret decoder ring. And if I’m honest with myself, I knew it wouldn’t.

Ethan and I have talked multiple times about how much thought and work is going into the Boston Globe project. Responsive Web Design, particularly the mobile first variety, is still very young.

So it was unfair of me to expect Ethan’s book to magically make the tough stuff we’re all working on solving suddenly go away. It’s a testament to Ethan that I even thought he might be able to pull it off.

What the book did do is something I hadn’t dared to imagine: the book transcends the topic it covers.

I don’t know how else to describe it. From the first two paragraphs, Ethan paints a picture of how the world we’re living in is changing:

As i begin writing this book, I realize I can’t guarantee you’ll read these words on a printed page, holding a tiny paperback in your hands. Maybe you’re sitting at your desk with an electronic copy of the book up on your screen. Perhaps you’re on your morning commute, tapping through pages on your phone, or swiping along on a tablet. Or maybe you don’t even see these words as I do: maybe your computer is simply reading this book aloud.

Ultimately, I know so little about you. I don’t know how you’re reading this. I can’t.

With these words he sets a tone that continues throughout the book. A tone that is both reassuring and at the same time inspiring.

Now don’t get me wrong, it isn’t all inspiration and theory. Ethan does a great job of diving into how to build Responsive Web Designs. He walks you through the process, letting you see where you will run into problems, and providing suggestions for solutions.

Images from inside Ethan’s Responsive Web Design book

He anticipates problems that I wouldn’t have thought to include. For example, the fact that flexible images develop artifacts on Internet Explorer 7 and below.

At first, this IE artifact bug seems like a divergence from the main topic, but I quickly came to appreciate the fact that when someone completes this short book, that they will have all the tools they need to start building Responsive Web Designs.

Ethan ties the two threads—the inspiring view of the where the web is headed and the technical details of Responsive Web Design—together In the final chapter entitled “Becoming Responsive” where he covers mobile context, picking breakpoints, and responsive workflows.

It is in this chapter that you realize what he’s truly accomplished. He’s written a book ostensibly about making the web respond to its environment, but what he’s really done is transform you into a more responsive being.

Ethan Marcotte’s new book, Responsive Web Design, went on sale today. Needless to say, I highly recommend it.

Where are the Mobile First Responsive Web Designs?

Lyza and I entitled the second chapter of our forthcoming book “Mobile First Responsive Web Design (RWD)”. Because this is a Head First book, we can’t simply explain the theory of Mobile First RWD. We have to teach our readers how to do it.

For those unfamiliar with the phrase, Mobile First RWD takes the concepts that Ethan Marcotte taught in his A List Apart article on Responsive Web Design and flips the process around to address some of the concerns I raised about media queries.

Instead of starting with a desktop site, you start with the mobile site and then progressively enhance to devices with larger screens. As a bonus, you benefit from the advantages of Mobile First design that Luke Wroblewski espouses.

In practical terms, I think a Mobile First RWD approach requires addressing some of the following challenges:

  • The need to start over and design for the smaller, more limited devices first
  • Ability to replace images with higher resolution images as the size or dpi of the screen increases
  • Ability to test for screen size and capabilities
  • Ability to add more javascript as capabilities and/or screen real estate increases
  • Ability to add more markup as capabilities and/or screen real estate increases
  • Ability to modify the layout as screen real estate increases
  • Decision about how to handle doctype and other markup if you’re going to support older phones that primarily support XHTML-MP

This list was just a starting point for exploration. I wanted to review how other developers are approaching Mobile First RWD in an attempt to find techniques that I might not be aware of and to see how they are thinking about these problems.

So I set out in early April to find examples of Mobile First RWD by reviewing every site listed in the Mediaqueri.es gallery which at the time was just over a hundred sites.

How do you know if something was designed mobile first?

Mobile first design is primarily about the starting point. After a site is complete, how can I tell whether or not the developer started from the mobile and built up to desktop or started from the desktop and whittled down to mobile?

I didn’t want to have to tear apart over a hundred sites in the Mediaqueri.es gallery to find the handful of mobile first sites. I needed some way to narrow the number of sites I cared about to some sort of manageable number.

I believe that mobile first RWD sites will exhibit common behavior as either a direct output of or a byproduct of their mobile first focus:

  • The total size of the markup and assets for mobile web will be significantly smaller than that delivered to desktop web.
  • The total number of http requests for mobile web will be smaller than those delivered to desktop web.

So I set up Charles proxy and loaded every site on mediaqueri.es and a few others that I knew of in both Mobile and Desktop Safari. I recorded the total size, number of http requests (as resources) and the time to download for each.

My theory was that the sites that showed the biggest jump in size when going from mobile to desktop would be the ones most likely to be using mobile first techniques.

The Results Weren’t Promising

Out of the 106 sites I reviewed, 64 of them had mobile sites that were less than ten percent smaller than the desktop version. Twenty-six of the sites had mobile web sites that were larger than the desktop equivalent.

Mobile is Larger 26, Same Size	4, Less than 10% Savings 40, 11 to 50% Savings 22, 51% to 100% Savings 4, Greater than 100% Savings 10

While the numbers were disappointing, I wasn’t terribly surprised. I sorted the results by the percentage difference in size in order to find the sites that were most likely to have been design as Mobile First RWD.

I just dug into the 22 sites that have the greatest percentage difference between their mobile sites and their desktop site. I made notes on what I found in the a spreadsheet which I’ve shared on Google Docs.

After all of that work, I only found five sites that may be Mobile First RWD:

  • Yiibu.com — Bryan and Stephanie’s site continues to be the gold standard.
  • EthanMarcotte.com — Shocking, huh? :-)
  • Elections.ChicagoTribune.com — Uses the Less framework. Mobile site is about half the size of the desktop version, but still downloads assets that aren’t needed in the mobile view.
  • Calebogden.com — I need to dig into this site more. I think the difference may be a weird issue with gzip on the desktop site.
  • Sleepstreet.be — Appears to be using the Responsive Images javascript library.

Needless to say, I was hoping to have more than five sites to review.

Why are Some Mobile Sites Bigger than their Desktop Counterparts?

The number of mobile sites that were significantly larger than their desktop siblings surprised me. The source for the big differences were:

  • Font usage — Some of the sites were using an old version of Typekit that downloaded SVG fonts on iPhone. These font files made the mobile version much bigger than desktop.
  • Retina images — Sites that were using retina images. In one case, I’m not 100% certain they are retina images because they are routed through tiny src so I can only tell that file size is much bigger for the images on mobile than desktop, but not why. In the other case, both the mobile images and the retina mobile images are getting downloaded for a double hit.
  • HTML5 Audio — HTML5 audio is embedded on the page. The browser is downloading the full mp3 file regardless of whether or not the user presses play. This balloons huffduffer.com from 122k on desktop to 2.9MB in Mobile Safari.
  • HTML5 Video — Mobile Safari starts downloading a portion of the video in order to validate the video will work and to grab a scene from the video to use in the player. Desktop Safari wasn’t doing this.

The lesson here is that if you want to really know what is getting downloaded, you need to test using something like Blaze.io’s mobile test or better yet, use a proxy like I did.

Why so Few Mobile First Responsive Web Designs?

This shouldn’t be a huge surprise. Here are some of the reasons why Mobile First RWD isn’t more widely spread:

  • Mobile First RWD may require a redesign — If you’re moving an existing site to RWD—especially if the existing site has fluid grids—you can with minimal work make it responsive. If you decide to do Mobile First RWD, you’re likely going to have to redesign the site or at minimum refactor your markup.
  • It presents larger technical challenges — What do you do about the img tag and the fact it only has one source? How do you make sure javascript only gets downloaded when needed? How do you modify your content management or processes to make sure images get resized appropriately?
  • Few frameworks support mobile first approach — Only recently have frameworks that take a mobile first approach like 320 and Up and the HTML5 Mobile Boilerplate been released.
  • Javascript libraries are just beginning to evolve — The Filament Group is leading the way here with Responsive Images and Respond.js. Different approaches are being developed like Adapt.js. As more people try to tackle mobile first designs, I expect more libraries to be published.
  • We’re still waiting for equivalent of the ESPN and Wired redesigns — The ESPN and Wired redesigns kickstarted the web standards movement. We all have high hopes that the work Ethan and The Filament Group are doing for the upcoming Boston Globe design will be a shining example for mobile first RWD.
  • The technique is still young — For as much as has been written about RWD, it is hard to remember that the ALA article was only written a year ago. People only started working on mobile first RWD last summer. There is still a lot of work to be done before the techniques are well known.

For us, the results of our research meant that we postponed the chapter on Mobile First RWD until later. Our hope is that by the time we return to the chapter, that there will be more examples to learn from.

What Sites Did I Miss?

Since I did my analysis in April, another 30 sites have been added to the Mediaqueri.es Gallery. Eventually I’ll take a closer look at them as well.

In the meantime, do you know of a site that qualifies as a Mobile First Responsive Web Design? Did you help develop it? Does it tackle some of the tough problems like img tag src and progressively enhancing based on screen size?

If so, I’d love to know about it. Sooner or later we’re going to have to get back to writing that chapter on Mobile First Responsive Web Design, and we could use as many examples to draw from as possible.

Links Don’t Open Apps

Are you having trouble convincing people that they need to develop a mobile web site as part of their overall mobile strategy?

I have a solution for you. Ask the people you need to convince if they do any of the following:

  • Send email to their customers?
  • Participate in social media?
  • Search engine optimization?
  • Advertise online?

Each one of those marketing efforts is based on links. And links don’t open apps.

It seems like a basic concept, but the fact that links can only reliably open web pages is often forgotten.

This is one reason why mobile web has to be part of every company’s strategy. When someone encounters a link via an email newsletter or shared via a social network, they should be able to view that link no matter where they are and no matter what device they are using.

Technically True vs. Practical Reality

Inevitably, when I talk about how links don’t open apps at a conference, someone who wasn’t in the audience will point out that on some platforms like iOS, you can register URL schemes to open apps.

While this is technically true, it isn’t practical for most communication because:

  • Not every platform offers equivalent functionality. On Android, the way to invoke applications is intents which works very differently.
  • Even if you could somehow create the same url scheme on every mobile platform, the url would only work if the user had your application installed which you can’t guarantee nor control.

While saying that links don’t open apps isn’t “technically” true, it is a practical reality.

Power of Hyperlinks

The realization that links don’t open apps has triggered for me a renewed appreciation of the power of hyperlinks. When people talk about the differences between native apps and mobile web, they usually talk about difference like performance, cross platform development, and other technical factors.

Rarely do we talk about hyperlinks and the power it provides the web. No native platform will be able to replicate the universal utility of links any time soon.

We should stop worrying about whether or not native apps can do certain things better than web technology, and instead talk about what makes the web unique, powerful, and universal.

Adventures on the mobile Web frontier: Wrangling Drupal, third-party APIs and server oomph into an enterprise-class experience

You’ll hear us at Cloud Four often use the metaphor that mobile Web development is still in the Wild West phase: there’s a lot of hooting and hollering and posturing, but it’s still a sparsely-settled, incompletely-explored and exhilarating place full of legends and buried treasure. Well, perhaps not that glamorous. It’s sort of the unsung, rough-and-tumble mountain man cousin compared to those dashing native apps. But there’s gold in them hills.

When Cloud Four recently helped Hautelook, a high-volume “flash sale” retail site, build their mobile Web strategy, we got a chance to see if some of our bolder theories about the mobile Web—context, content adaptation, technology stack—would hold water in a performance-critical environment.

Supporting large numbers of concurrent connections from widely disparate devices, handling the crush of shopping enthusiasm at 8AM Pacific every day (when new sales events go live), all while interfacing with a remote, third-party API meant we really had to push the envelope. And invent some neat tricks. And find a few things out the hard way.

Adapting the Experience: Server or Client?

It’s an enduring debate: should the savvy mobile Web developer adapt content for mobile devices on the server, or should most of that happen on the device (client) itself? Before everyone gets too riled, let me say this: I believe both approaches have solid merit. There.

For Hautelook, we’re letting the servers do the heavy lifting. In a nutshell, we use the Drupal Web content management system (CMS), along with the WURFL device library and some Cloud Four-brewed software to define and manage content for several categories of devices.

We’re able to put the brawn of the server hardware behind making decisions about incoming requests and are able to route the right devices, very quickly—utilizing memory-based caching and other performance tuning—to the right place, ensuring that only those bytes relevant to the user are delivered. In such a high-traffic environment, it simply won’t do to be pumping out markup, images or CSS that aren’t actively needed for the particular user.

In addition, Hautelook’s mobile Web site supports a rather dizzying array of devices. While the majority of users are on newer smartphones—iOS, Android, BlackBerry 5+ and WebOS are the major platform players here—we do see a consistent flow of folks using all sorts of older devices. For older phones and feature phones, it can be dangerous to rely on the client too much, and especially perilous to assume any sort of meaningful JavaScript or advanced CSS (ahem, media query) support. That puts the server in the best position to make certain kinds of decisions.

Another thing that sometimes precludes client-side adaptation is the need not just to provide adapted content to a given device class, but in fact to provide different content altogether. For example, in Hautelook’s case, product listings are output as HTML unordered lists on modern devices—the semantically obvious way to do it. However, due to some brain-dead-ness in certain devices (I’m looking at you, BlackBerry 4.x browsers, and you, certain weird flavors of IE Mobile), CSS for unordered lists was, well, predominantly it wasn’t behaving in any socially-acceptable way. In this case, we opted to deliver product listings as HTML tables for those devices: inelegant, smacks of 1997, but easier to rein in on ill-behaving phones. We’re able to do that server-side by leveraging Drupal’s theming system and our own content-adaptation framework.

Another situation that calls for not-just-adjusted-but-different content is in the handling of different user contexts: we know that browsing high fashion on a 128-pixel Samsung smartphone is not going to be as compelling as on a big, comfy desktop monitor. Instead we know that those users probably want to find stuff fast and buy it even faster. It’s not just about delivering mobile-shaped content, it’s about delivering mobile-relevant content, fresh and piping hot from the server.

Then there are images. Shoving a 153,600-pixel image at a phone that only has about 16,000 pixels of real estate to work with is like giving 4-by-8-foot sheets of plywood to a kid who wants to build a miniature birdhouse. Even if wood is cheap, it tastelessly wastes a lot of trees, and poor little underpowered Billy can barely lift the thing or cut it with his little toy handsaw. Not to stretch a metaphor too far or anything.

By extending our definition and handling of what a device class is and does, we can deliver the right kind of images and other media to the right kinds of devices.

That doesn’t mean that there is no room for client-side awesomeness! We’re pretty excited about stuff we can do with tools like Modernizr, especially for newer devices. The full solution certainly includes both client- and server-side elements.

Nutshell: How it works

Mobile Web service overview

  1. Akamai sits in front of both sites (desktop and mobile). It’s the first-line mobile detection workhorse. It looks at incoming traffic and determines whether it is a mobile device or not. Mobile devices are sent to the mobile site*.
  2. Sometimes you need an Abrams Tank, sometimes you want the brisk efficiency of a Hyundai (hey, newer Hyundais are pretty decent!). In our case, we’re putting the nginx web server (our high gas-mileage Hyundai) out on the front lines. nginx is dinky and quick, and can handle whole scads of simultaneous connections. Apache has big guns and horsepower on its side, but sprightly little nginx acts as gatekeeper, navigator and hand-holder, only sending for the tanks when it really needs the tanks. nginx handles Keep-Alive connections and load balancing, leaving apache to do what it does best. Even with apache, we’re being as efficient as possible, disabling Keep-Alives and using APC to keep application code in memory.
  3. We can arbitrarily scale by adding more application instances (apache/Drupal) on more servers.
  4. Now we’ve made it to the apache/Drupal piece of the puzzle. We know our requester is a mobile device, but now we want to know what kind. We’ve written a Drupal module that lets us define what characteristics constitute different devices, giving us control over what each device class “means.” This is a (very) similar approach to the available, third-party Drupal module mobile_tools. Our module communicates with the WURFL PHP API, but only when it needs to. Once a device has been identified, we cache what we need to know about it, keyed by User Agent. Any further requests from devices using that User Agent are handled super duper fast, using information cached in memory.
  5. Intelligent caching, while not strictly a mobile-specific tactic, is absolutely core to our strategy. We cache as aggressively as we can, balancing performance against the need to have very fresh data out of the API. API calls are shared across users and multiple page requests, wherever possible.
  6. While we aren’t delivering the same content to each device class, nor are we replicating markup between them. We define baseline content theming, from which each device class inherits, and has the opportunity to override. Again, we cache what we can—yes, even device-specific rendered markup in some cases—to push performance even further.

* Don’t worry! Mobile users can opt to view the desktop site and they won’t be redirected back to the mobile site.

Challenges

Here’s some stuff that was hard (warning, highly Drupal-specific/tech-y):

  • Drupal has, generally, a phenomenally-flexible hook system. I can, 98% of the time, find a pleasing way to implement something without ugliness. But this breaks down (in Drupal 6) when it comes to tight control of CSS and JS inclusion in pages. Drupal 7 introduces hook_js_alter() and hook_css_alter(), which will fix the exact problem I had to lightly hack around on the theme level for this project.
  • WURFL is awesome sauce. It contains a veritable treasure trove of device-specific data. One thing that’s challenging still, however, is the conflation of devices and browsers. I hear they’re working on this as we speak! But at present, using the PHP WURFL API, Opera Mini on an iPhone is identified as…an iPhone. With Safari, and WebKit. Eep.
  • OpenWave browsers (quite popular on older feature phones, but also still used on some recent ones) cause pain. Drupal forms with an action that is the same as the page they are on will create an infinite redirect loop, somehow. The only solution seems to be to tack on an interstitial #redirect page (via hook_form_alter()) for those forms that don’t have one. Also, I was only ever successful in getting them to correctly handle cookies if I used a TLD cookie domain. UGH.
  • Some BlackBerries don’t take it well if you destroy and write to the same cookie in a single request.
  • If you want to serve up valid XHTML Mobile Profile 1.2 content, you’ll need to override Drupal’s default theme implementation for ‘table’. It uses THEAD tags, which are a no-no in XHTML MP1.2.
  • RFC 2109 is a curse. But I digress.