Cloud Four Blog

Technical notes, War stories and anecdotes

iPhone’s Magical Volume Buttons

A recent project pushed the boundaries of what is possible in mobile browsers. We learned a lot including some surprisingly simple things that everyone takes for granted like the volume buttons on the iPhone and how they work.

One of the features was a quiz that people could take on their phone. The quiz design was handled by another agency and they provided us with some cool sound files to use when the person got an answer right or wrong.

We built the site using HTML5 and quickly found that Android didn’t support HTML5 audio. Android 2.3 now supports HTML5 audio, but it was too late for this project.

On the iPhone side, we discovered what appeared to be a strange bug related to audio. Adjusting the volume didn’t seem to have any affect on the volume of the HTML5 audio clips. We began to fear that people playing the quiz would be upset when an ear drum piercing buzz came from the quiz and they couldn’t turn it down.

After nearly two weeks of believing HTML5 audio was broken, we encountered a phone that used to be loud that was suddenly quiet. What’s going here?

It was then that we learned about iPhone volume contexts. If you own an iPhone, you’re already aware of these volume contexts, but never consciously think about them because 99.9% of the time, it just works.

What do I mean by volume contexts? There is only one set of volume buttons and depending on what your iPhone is doing, they adjust different volume settings.

For example, if no audio or video is playing, the volume buttons adjust the ringer volume. However, when audio or video is playing, the volume buttons control the media audio level.

iPhone volume indicators

The iPhone provides feedback on what volume you’re adjusting by adding “ringer” above the volume level when you’re adjusting the ring level and leaves that out when you’re adjusting the volume of other audio.

But it’s not simply ringer or media contexts. The iPhone also keeps track of the volume level separately in each of these contexts:

* Headphone
* Headphone w/ microphone
* Speakers
* Bluetooth headsets

There may be more that I’m not aware of. I also don’t own a bluetooth headset so I’m relying on what I’ve been told.

The point is that it was nearly four years after owning my first iPhone before I gave any thought to how the iPhone was doing this. This is truly a remarkable design. It handles at minimum six different volume settings without the user ever giving thought to what volume they want to control.

They simply use the two volume buttons and 99.9% of the time the phone magically knows what they want it to do. You never have to think about it.

Until you build a quiz that falls into the .1% of the time when it doesn’t work the way you expect.

Our problem? the audio files were very short. Around a second each.

And unless you’re a speedster, you can’t hit the volume buttons quickly enough during that one second of audio.

Once that second passes, you’re back to adjusting the ringer volume and mistakenly wondering why the volume buttons don’t work on HTML5 audio.

Webkit: The Dominant Smartphone Platform

Based on Q2 sales of smartphones, Webkit-based browsers may soon ship on 85% of all smartphones sold.

Pie chart showing market share for webkit based on smart phone OS

Please keep in mind, this is not the reality right now. This number assumes RIM’s purchase of Torch Mobile really means that future Blackberry Browsers will based on Webkit.

There are some other caveats as well:

  • This understates Opera’s mobile market position. Opera has a large installed base of users.
  • It assumes market percentages will stay the same. We know this won’t be true.
  • It assumes that all of the “other” smartphone OS browsers are not using Webkit currently.
  • Mobile Firefox is just getting started. It is unclear how that will change the landscape.
  • Just because it is Webkit, doesn’t mean that it is the latest version of Webkit.

Caveats aside, you would be hard pressed to find another smartphone development platform with any where close to Webkit’s market share.

More importantly, this means that HTML5 for mobile is looking great. If Blackberry joins the ranks of Webkit-based browsers, that will means Symbian, iPhone, Android, Palm and Blackberry will all be on the path to HTML5 support.

The only laggard will be Mobile Internet Explorer, and even for Windows Mobile there are options like Google Gears which adds some HTML5 support to IE or installing other browsers like Opera or Firefox.

Google Argues for Mobile Web

At the recent MobileBeat Conference, Google VP of Engineering Vic Gundotra told the audience that the Mobile Web will be the larger portion of mobile application development in the long run instead of native applications.

Mr. Gundotra argued the web was the only technology to provide the potential for cross platform development. He said not “that even Google was not rich enough to support all of the different mobile platforms from Apple’s AppStore to those of the BlackBerry, Windows Mobile, Android and the many variations of the Nokia platform.”

His comments echo those that he made earlier at Google I/O. I recommend reading Tim O’Reilly’s summary of his presentation.

This is very similar to what I’ve been saying in my presentations at Web 2.0 Expo and Web Visions.

I’m not arguing that native applications will not continue to be important. I’m simply arguing that there will be far more mobile web sites and mobile web applications than native applications over the long term.

HTML 5 Resources & Discussion

I wanted to include a series of links at the end of my previous post on HTML 5, but I was afraid they would get lost after such a long post. Here they are:

HTML 5 Developer Group in Portland

A new HTML 5 developer group has formed in the Portland area called HTML 5 PDX.

The first meeting was Monday and had a representative from Firefox speaking (Dietrich Ayala) as well as a designer (Bram Pitoyo) who spoke to Safari’s HTML 5 support. Igal Koshevoy posted detailed meeting notes.

HTML 5 Resources and Demos

HTML 5 / XHTML2 Discussion

HTML5 from a Mobile Perspective

I’ve read with interest the controversy following the W3C’s decision to not renew the XHTML 2 Working Group charter. For those unfamiliar with the jargon of standards organizations, this means that XHTML 2 is effectively dead. In its place as the future of web development stands HTML5.

The comments about HTML5, and the potential for its future have been surprising. I hadn’t realized how working on mobile has changed my perspective.

At the risk of being accused of wearing mobile-tinted glasses, HTML5 is going to big a deal and it will be relevant much sooner than people think.

Lack of Extensibility

The main complaint against HTML5 is the lack of extensibility. In particular, the ability for developers to define new semantic markup that provide meaning to a document.

The need for this meaning is what created microformats which describe standard ways of presenting information like addresses in markup so that they can be understood programmatically.

The best description of this extensibility problem can be found in John Allsop’s article from A List Apart. John points out that:

This is not simply a theoretical problem. Hundreds of thousands of developers use the class and id attributes of HTML to create more richly semantic markup… Almost invariably, those developers use ad hoc vocabularies—that is, values they have made up, rather than values taken from existing schemas. It’s pseudo semantic markup at best.

HTML5 does add new elements like header, nav, article, section, aside, and footer which expand the structural definition of a page, but do not provide the level of extensibility that people have been seeking.

Documents versus Applications

The complaints about extensibility are legitimate, but the perspective is one that still looks at HTML as a document publishing markup language. John Allsop wrote:

There is a very real problem that needs to be solved here. We need mechanisms in HTML that clearly and unambiguously enable developers to add richer, more meaningful semantics—not pseudo semantics—to their markup. This is perhaps the single most pressing goal for the HTML 5 project.

This is a very telling indication of the perspective that John is coming from. He is looking at HTML as a document publishing and noting correctly that it lacks the tools it needs to describe the documents sufficiently.

I can see these short-comings, but feel that addressing the short-comings of HTML for web applications is the more pressing need—especially when it comes to being able to build rich web applications for mobile devices.

It seems like this may be the main disconnect between the developers of the HTML5 specification and its critics. The specification that eventually became HTML5 started out as the Web Applications 1.0 specification. Its sibling document was Web Form 2.0. Both of these have been merged into HTML5.

The WHATWG, which has been leading the effort to define HTML5, has stated that it “focuses primarily on the development of HTML and APIs needed for Web applications.” With that mission in mind, HTML5 is a substantial step forward.

(As an aside, John Allsop is right that ARIA support in HTML5 would give developers much of what they need, and I have yet to read a good argument against adopting it. I just don’t find it to be as pressing as the other things HTML5 includes.)

HTML5 for Mobile

HTML5 is a critical step for mobile web application development. Some of the key elements that it provides are:

  • Offline Support — The AppCache and Database make it possible for mobile developers to store thing locally on the device and now that interruptions in connectivity will not affect the ability for someone to get their work done.
  • Canvas and Video — These two features are designed to make it easy to add graphics and video to a page without worrying about plugins. When supported by the phone’s hardware, as is the case with the iPhone, they provide a powerful ways to get media into a page.
  • GeoLocation API — This is actually not part of HTML5, but is a separate specification. That said, it is often bundled together because the mobile phones that are including HTML5 are generally supporting the GeoLocation API.
  • Advanced Forms — Even simple things like the improvements in HTML5 for forms could make life easier for mobile applications. Fields that can be validated by the browser are improvements for mobile devices. The more that can be handled by the browser means less time downloading javascript code and less round trips to the server if validation can be found before the form is posted.

Most importantly, nearly all of the hybrid applications frameworks—Phone Gap, QuickConnect, RhoMobile, Titanium Mobile, and others—rely on HTML5 features to provide a rich application experience.

For these reasons and many more, I believe the adoption of HTML5 will be driven by the needs of mobile, not the needs of desktop developers.

When Can We Use HTML5?

Many people dismiss HTML5 as something they won’t have to worry about for some time. A commenter on Zeldman’s blog wrote:

None of this really matters until:

  1. IE supports whatever new standard we decide on
  2. IE no longer has the majority of the market share

This is why mobile will drive HTML5 adoption. The iPhone, Google Android, Nokia, and the Palm Pre are all based on the open source Webkit browser engine. Those phones represent somewhere around 65% of smart phones sold.

(Note: it is too earlier to find estimates the percentage of Q2 smart phone sales. A lot changed in Q2 which is why older market share breakdowns are not as useful.)

The two major platforms not using Webkit are Windows Mobile and Blackberry. Some of the capabilities of HTML5 are available to Windows Mobile users via the Google Gears plugin.

It also remains to be seen if Microsoft will have a viable mobile strategy. At the moment, it doesn’t look good for the future of this platform as HTC, which is currently responsible for 80% of Windows Mobile sales, is rumored to expect Android to comprise 50% of the units it ships in 2010.

Therefore, the real barrier to HTML5 adoption is RIM’s Blackberry platform.

Blackberry has its own specialized browser not built on any of the major browser engines. It only recently started handling html, css and javascript reasonably well, but still is insufficient and buggy compared to other browsers.

Fortunately, for both Windows Mobile and Blackberry, Opera’s browser is both available and popular. It is consistently one of the top if not the top download on mobile applications sites. And Opera is one of the leading developers of HTML5.

Finally, if you look past mobile phones to other mobile devices like the iPod Touch, Nokia’s internet devices, and the upcoming Google Chrome, you see that Webkit is even more broadly distributed.

Webkit Does Not Equal HTML5 Support… Yet

To be clear, just because a device uses webkit does not mean that it has the latest version of Webkit and can use HTML5. Recognizing the market share of Webkit is important solely as an indicator that a significant portion of smart phones will have access to HTML5 sometime in the near future.

There is great incentive for mobile operating system vendors to upgrade to the latest versions of Webkit. They see the success that the iPhone has had and the fact that one of the main contributors to that success was the browsing experience. They understand that not many companies can afford to develop native applications for all of the various platforms which makes the features of HTML5 attractive.

Because of this, I expect browser improvements to be a high priority for mobile operating systems.

The Pressing Need for HTML5 is More Features

HTML5 needs to move past simply providing geolocation and offline storage and into more of the device characteristics. The Palm Pre’s WebOS has had to forge forward in defining its own APIs for accessing things like the address book, camera, and accelerometer.

I had an opportunity to ask Michael Abbott, Palm’s Senior Vice President responsible for WebOS, about the use of standards for mobile devices characteristics. His response was that they used standards where they could, but that there were no standards for much of what they wanted to do.

This problem is only going to continue to become more pressing. People are clamoring to develop mobile applications that take full advantage of the the capabilities of devices.

Smart Phones Do Not Equal All Phones

It is important to note that while HTML5 will be very important for smart phones, it won’t reach feature phones for sometime. Therefore, content publishers should continue to work with device databases for content adaptation.

Again, web application developers can make different choices and tradeoffs than content publishers. If you are building an application that combines cameras with geolocation information, you’ve already narrowed which handsets you support.

Features will Drive HTML5 Adoption

Remember how easy it was to convince people to move from HTML working drafts to HTML 3.2? Now compare that to the education process necessary to convince people to convert to XHTML.

Part of the difference is due to the times, but a large part of it was due to the fact that you could do things in HTML3.2 that you couldn’t do in previous versions.

That’s not to say that recent web standards like XHTML haven’t provided new features, but that none of the new features were game changers. GeoLocation is a feature that can create businesses. The same is true of access to other device characteristics.

For this reason, web developers who start looking at mobile development will not shy away from building using HTML5 even if it limits their audience.

The Mobile Perspective on HTML5

From a mobile perspective—and perhaps from the perspective of web applications generally—HTML5 cannot come quickly enough.

As Vic Gundotra, Google Engineering vice president and developer evangelist, recently pointed out, not many companies are rich enough to develop native applications for all mobile platforms.

The mobile web provides are the best hope for building a cross-device mobile ecosystem. HTML5 is a critical piece for the mobile web.