Icons are everywhere. These “little miracle workers” (as John Hicks described them) help us reinforce meaning in the interfaces we design and build. Their popularity in web design has never been greater; the conciseness and versatility of pictograms in particular make them a lovely fit for displays large and small.
But icons on the web have had their fair share of challenges. They were time-consuming to prepare for every intended display size and color. When high-resolution displays hit the market, icons looked particularly low-res and blocky compared to the text they often accompanied.
So it’s really no wonder that icon fonts became such a hit. Icons displayed via @font-face were resolution-independent and customizable in all the ways we expected text to be. Sure, delivering icons as a typeface was definitely a hack, but it was also useful, versatile, and maybe even a little fun.
Most assistive devices will read aloud text inserted via CSS, and many of the Unicode characters icon fonts depend on are no exception. Best-case scenario, your “favorite” icon gets read aloud as “black favorite star.” Worse-case scenario, it’s read as “unpronounceable” or skipped entirely.
When your icon font fails, the browser treats it like any other font and replaces it with a fallback. Best-case scenario, you’ve chosen your fallback characters carefully and something weird-looking but communicative still loads. Worse-case scenario (and far more often), the user sees something completely incongruous, usually the dreaded “missing character” glyph.
And you’re right. Those techniques are great! If you’re using an icon font, you should definitely follow their recommendations to the letter.
But you probably won’t.
What you’ll probably do is adopt whatever your framework of choice has bundled, or drop in some massive free icon font you can use right away. You won’t modify how they work out of the box because that’s really hard to prioritize, especially when they look great on your monitor with virtually no effort at all.
Or maybe you will do the work, designing and curating a custom icon font, choosing your Unicode characters carefully, documenting and evangelizing the importance of implementing your icons in an accessible way with appropriate fallbacks. Then one day, Dave forgets to add a fallback image to that iconographic button he added (which looks great, by the way), which Roberta reuses for her related Pull Request, and before you know it, your app has devolved into a fragile, hack-littered wasteland once more.
These examples are not hypothetical (though names have been changed to protect the innocent). I’ve seen them happen to multiple organizations, all of them starting with the best possible intentions.
Usually when I hear this, the team’s comparing a single binary icon font to multiple, uncompressed SVG files. The gap narrows dramatically when you optimize your SVGs, combine reusable ones into sprites, and deliver those with active Gzip compression or embedded in-page.
Occasionally I’ve heard the gap is still too wide when hundreds of icons are included. This begs the question: Why are you including hundreds of icons on every page?
“The framework we chose already has an icon font.”
If your framework told you to jump off a bridge, would you?
Don’t Be “Table Guy”
I was in school when the Web Standards movement hit critical mass. While the majority of my instructors saw the merits of semantic markup and embraced it wholeheartedly, one passionately held out. “Table Guy” argued that no layout tool could usurp <table>, that it was inherently better-suited for crafting grid-based designs. He boasted of how quickly and easily he could achieve the “Holy Grail” layout with his trusty table cells. He cited the wealth of cross-browser inconsistencies that continued to plague CSS.
Table Guy and I kept in touch. Today, he freely admits he was wrong about CSS. He feels embarrassed to have been so married to a technique that was so clearly misappropriated in hindsight.
If you won’t stop using icon fonts for people with screen readers, people with dyslexia, people with browsers that don’t support @font-face, people who randomly didn’t load the icon font once for some reason, or designers who just want their icons to look right on-screen…
Fun fact: Cloud Four’s design team really digs SVG. Our enthusiasm for the image format accumulated gradually over many months, thanks in large part to Sara Soueidan’s tireless documentation of its most mysterious features and quirks. It was during the process of designing the Responsive Field Day site that our collective interest level hit fever pitch, which caused our coworkers to wonder what all the fuss was about!
It turned out to be a difficult question to answer. Most of the resources we found online either covered the very basics of the format, or jumped right into the nitty-gritty of coordinate systems, complex animation, automated sprite-building, etc. So fellow Cloud Four designer Sara Lohr and I decided to put together an internal presentation with reveal.js to bring everyone up to speed.
In spite of those challenges, the talk was a hit. I think we introduced concepts in a way that made sense for the audience, emphasizing the sorts of things they’d find most useful day-to-day.
Then again, maybe it’s just easy to win people over with demos like “Jasonflower”:
But without webkit available for tvOS, there will be no embedded web views and no third party browsers.
We now have both Apple TV and Android TV without the web, and it’s a bloody shame.
I’m well aware of the argument that people don’t want to browse the web on TVs. I believe the jury is still out on that one, but even if I concede that point, there is still tremendous utility in using web technology for building apps for TVs.
The reason I started researching the web on TVs dates back to the original Google TV Showcase. There was a Vimeo app in that showcase that I loved.
I used that app for several months before I accidentally hit a button that converted it from the TV app that I knew into the standard Vimeo web page.
I had inadvertently discovered that the Vimeo app was just a different view of Vimeo’s normal web page. Vimeo calls this couch mode.
Nintendo, Samsung, LG, and others have all built app platforms on top of HTML. For years, Netflix built all of their TV apps on HTML5 before recently going native.
Web apps on TVs can be great experiences. Maybe it is because we don’t notice the lag as much when we’re using remote controls instead of touching the interface directly. Maybe it is because the interfaces for most non-game, TV apps are fairly simple.
Whatever the reason, building TV apps using web technology just seemed to work. Bridging the gap between native and web apps on TVs was easier than it was on mobile.
But year after year at Google I/O when I’d try to talk to people about Chrome on TVs, I wouldn’t get anywhere. Google TV shipped with Chrome, but it was a forked version that the Chrome team wasn’t responsible for and grumbled about.
I remember trying desperately to figure out who to talk to about the browser on Google TV. I was repeatedly and humorously pointed to Chris Wilson. Chris hadn’t working on Google TV in months. It became a running joke between the two of us.
A couple years later, Google announced that their TV product would be called Android TV. Google touted how the TV would finally be running the same version of Android as phones and tablets. It would be kept up to date.
Back in 2012, I was trying to muster enthusiasm in browser makers for working on the web on TVs. It seemed likely that TVs were going to be the next platform and instead of playing catch up like the web did on phones, we could be ready for the web on TVs from the beginning.
I feared playing catch up again. In retrospect, I should have feared much worse.
The two biggest mobile operating systems are now on TVs. One started with a browser, but no longer has one. The other just shipped without even an embedded web view.
From what I’ve seen, the web on TV could have been a star. What a missed opportunity.
We’re about an hour away from the Apple event where they will announce the new Apple TV. Here are the things I’m going to be watching for based on my time researching Smart TVs, game consoles, and set top boxes.
How does the remote control work?
Input remains the biggest challenge for all attempts to bring computer smarts to the screens on our walls. While the software and content options for the new Apple TV will matter, if Apple truly revolutionizes TVs, I suspect if will come from an improvement in input.
Historically, improved input has accompanied Apple innovation. The Mac’s mouse. The iPod’s scroll wheel. The iPhone’s touch screen.
The other lesson here is that none of these inputs were wholly Apple inventions. In each case, the input technology had been used by other companies in the past. The iPhone’s touch screen seemed ho-hum until people actually used it and realized how much attention to detail Apple had put into perfecting the input.
So I’ll be surprised if the remote control has some feature that we haven’t seen on remote controls in the past, but I also suspect that if Apple TV is a game changer, it will be because of the remote control.
Is there a pointer? And where is it used?
The current Apple TV is limited to d-pad interactions—up, down, left, right. There are a lot of interactions that need the ability to select an arbitrary point on the screen instead of navigating to that point by successive d-pad button presses.
The most obvious need is in games. The rumors are strong that the new Apple TV will focus on games. The remote control has been described as Wii-like in its ability to detect motion.
The question is whether or not there will be any interfaces where you may see a pointer on the screen. I highly suspect app developers will build apps that include pointers, but do any of the Apple apps themselves include a pointer. And if so, where is it used and how does it work?
Where is web technology used? Is there a browser?
I have little doubt that the new TV operating system will support embedded web views. Web views are critical for many apps.
I’ve dreaded writing this installment of the Responsive Images 101 series. Selecting image breakpoints is something everyone will face, and frankly, I have no good answers for you.
But sooner or later, we will all face the image breakpoints koan. We might as well start now.
What are responsive image breakpoints?
In our responsive layouts, breakpoints refer to the viewport sizes at which we make changes to the layout or functionality of a page. These typically map to media queries.
Responsive image breakpoints are similar, but slightly different. When I think about image breakpoints, I’m trying to answer two questions:
How many image sources do I need to provide to cover the continuum of sizes that this image will be used for?
Where and when should the various image sources be used?
The answers to these questions lead to different breakpoints than the criteria we use to select breakpoints for our responsive layouts. For our layouts, we follow Stephen Hay’s advanced methodology: We resize the browser until the page looks bad and then BOOOOM, we need a breakpoint.
With the exception of art direction, the main reason why we need multiple image sources has nothing to do with where the images look bad. We want to provide multiple image sources because of performance concerns, different screen densities, etc.
So we can’t simply reuse our responsive layout breakpoints for our images. Or I guess we can, but if we do so, we’re not really addressing the fundamental reasons why we wanted responsive images in the first place.
Image breakpoints for art direction is relatively easy
In situations where the image falls under the art direction use case, the art direction itself will often tell us how many image sources we need and when they should be used.
If you think back to the Nokia browser site example, we can tell when the image switches from landscape to portrait mode. When that switch occurs, we know we’re going to need a new source image.
However, this may only be part of the picture. What if one of the art directed images covers a large range of sizes. We may find that we still need to have multiple sources that don’t map to the art direction switch.
You can see an example of this in the Shopify homepage that we looked at in Part 8.
Despite the fact that the image only has one major art direction change—from the full image to the cropped one—Shopify still provided six image sources to account for file size and display density.
So knowing that an image falls under the art direction use case can give us some clues, but it doesn’t answer all of our questions about the necessary image breakpoints.
What about resolution switching breakpoints
This is where things really get tricky. At least art direction provides us with some hints about how many image sources might be needed.
So long as we’re downscaling flexible images, they will always look good. We can’t rely on them looking bad to tell us when we need to change image sources.
Let’s take a look at a resolution switching example:
In this example, we have a photo of Michelle Obama where the image in the page is 400 x 602 pixels for the current viewport size. The largest size that the image is ever displayed at is 2000 x 3010 pixels. That large file is 250K.
We can simply shrink that 2000-pixel image, and it will look good. But it would be unnecessarily large. It would be better if we provided a smaller version like the one 800 x 1204 resolution image that is shown in the example. That image is only 73K.
We can all agree that when the image in the page is only 400 x 602 pixels in size, that providing an image that is 800×1204 and 73K is better than having people download the largest version of the image.
But why stop at 800×1204?
If we provided another image source that was 600×903 pixels wide, it would only be 42K. That saves us 31K (42%) from the 800×1204 image.
Well shoot. A savings of 42% is a big deal. Maybe we should keep going. 500 pixels wide? 450 pixels wide?
Each smaller image source offers the potential for substantial savings over the previous size. If we keep on this track, we eventually end up with an image source that is the exact size of the image in the page.
So here’s the question that has vexed me about image breakpoints. How do I know when an image source is too big for the size that the image is being used in the page?
The answer is that unless the image source matches exactly the size that the image is being displayed in the page, it is always going to be too big. There is always going to be an opportunity to optimize it further by providing a smaller image.
Why not provide the exact size of the image?
At this point, you may be wondering why we simply don’t provide the exact size of that the image is going to be used in the page.
First, the whole point of flexible images in responsive design is to provide images that scale as the size of the viewport changes. If we provided images that were exactly the size used in the page, we’d likely need to download new images whenever the viewport size changes or the device was rotated.
Second, it is unrealistic to provide images at any size imaginable. Yes, we can dynamically resize images, but when we resize images, the server needs to do that work which slows down delivery of that image to the browser.
For this reason, most larger sites cache images on content delivery networks (CDN). Caching every image size possible on the CDN would be incredibly expensive.
As I mentioned at the beginning, I have no rock solid solutions for how to pick the number of image sources that you need. Instead, I want to describe some different ways of looking at the problem that may help inform your decisions.
Winging it (aka, matching your layout breakpoints)
Someone on your team says, “Hey, how many source images do you think we need for these product photos?”
You ponder for a moment and say, “Hmm… how about three? Small, medium and large.”
Don’t be ashamed if you’ve done this. I’m pretty sure almost every person working on responsive images has done this at some point.
Perhaps your organization still thinks about mobile, tablet and desktop which makes small, medium and large make sense.
Or maybe you take a look at the range that the image will be displayed and make your best guess. Perhaps you simply look at the number of major layout breakpoints and decide to do the same for your image breakpoints.
I completely understand. And this is better than providing one huge image for all viewports.
But it sure would be nice to have more logic behind our decisions.
Testing representative images
If guessing doesn’t seem like a sound strategy, then let’s insert a little science into the art of picking image breakpoints. We can take a look at some representative images and figure out how many breakpoints they need.
The hardest part of doing this is determining representative images, or figuring out if you have any at all.
For some sites, all the photographs may have a particular style dictated by the brand. If that is the case, finding representative images is easy. Pick a few images and then resize them and save them at sizes ranging between the largest and the smallest images until you feel like you’ve got decent coverage.
Of course, if your site has a diversity of image styles, finding representative images can be nearly impossible.
Memory usage influencing the distribution of image breakpoints
Earlier this summer, Tim Kadlec gave a presentation on Mobile Image Processing. In that presentation, he took a look at the memory usage of flexible images in responsive designs.
What Tim showed is that as an image gets bigger, the impact of resizing an image gets larger.
In the example above, reducing a 600×600 pixel image by 50 pixels in each direction results in 230,000 wasted bytes versus the 70,000 wasted bytes caused by reducing a 200×200 image by 50 pixels in the exact same way.
Knowing this tells us a bit about how we should pick our breakpoints. Instead of spacing out breakpoints evenly, we should have more breakpoints as the image gets larger.
Unfortunately, while this tells us that we should have more breakpoints at larger sizes, it doesn’t tell us where those breakpoints should be.
Setting image breakpoints based on a performance budget
What if we applied the idea of a performance budget to responsive images? What would that look like?
We’d start by defining a budget for the amount of wasted bytes that the browser would be allowed to download above what is needed to fit the size of the image in the page.
So say that we decided that we had a performance budget of 20K for each responsive image. That would mean that we would need to make sure that the various sources that we’ve defined for the image are never more than 20K apart.
When we do this, we find that the number of image breakpoints change wildly based on the visual diversity of the image and the compression being used.
Let’s take a look at three sample images.
Time Square — 8 Image Breakpoints
This image has a lot of visual diversity. The variations in colors and textures means that JPEG’s lossy compression cannot do as much without damaging the image quality.
Because of this, there are eight image breakpoints—set at 20k intervals—between the smallest size of the image (320×213) and the largest size of the image (990×660).
Take a look at the other images on a sample page we created. See how the number of breakpoints vary even through all the images start with the same resolution end points.
Now, I’m not suggesting that you manually decide on image breakpoints for every single image. But I can envision a future where you might be able to declare to your server that you have a performance budget of 20K for responsive images and then have the server calculate the number of image sources on a per image basis.
For both Yoav, who works at Akamai, and Ilya, who works at Google, one of the problems they see with multiple image sources is storing all of those sources on edge servers where storage is usually more limited and costs are higher.
Not only do companies like Akamai and Google want to reduce the number of images stored at the edge, but the whole purpose of their content delivery networks is to reduce the amount of time it takes for people to render a web page.
Therefore, if they can cache the most commonly requested image sizes at the edge, they will deliver the fastest experience for the majority of their users.
For these organizations, they can tie their image processing and breakpoints logic to their analytics and change the size of the images over time if they find that new image sizes are getting requested more frequently.
When combined with the new HTTP Client Hints feature that Ilya has championed, servers could get incredibly smart about how to store images in their CDNs and do so in a way that requires little to no decision-making by designers and developers.
Humans shouldn’t be doing this
I believe that in a few years time, no one will be talking about how to pick responsive image breakpoints because no one will be doing it manually.
Sure, we may still make decisions for images that fall into the art direction use case, but even then, we’re probably not going to make finite decisions about every image source. We’ll handle the places that require our intervention and let our image processing services handle the rest.
There is a ton of benefit to either picking image sources based on a performance budget or based on the frequency with which different sizes of the image are requested. But either of these solutions are untenable as part of a manual workflow.
In the future, our typical workflow will be that we upload the highest quality image source into our content management system or image processing system and never have to think about it again.
Join us for a spirited, knowledge-packed day scouting the frontiers of responsive web design and development in lovely Portland, Oregon. Responsive Field Day is a welcoming and affordable gathering for web designers and developers.