Cloud Four Blog

Technical notes, War stories and anecdotes

JavaScript GZIP Compression in WordPress: What’s Possible and what Hurts

WordPress 2.8 introduced some constants and bits and bots that nominally make certain kinds of compression of JavaScript and CSS possible. This post looks at JavaScript compression and explains what is possible, and what is very, very hard.

In a nutshell, with a bit of tweaking, you can GZIP and concatenate the scripts that come packaged with WordPress. GZipping and concatenating anything else–plugin scripts or scripts in your own theme–is not currently supported in any obviously sane way (using WP’s WP_Scripts management class anyway).

I’m going to give away the ending to save you some time:

add_action('wp_enqueue_scripts', 'lyza_force_compress');
function lyza_force_compress()
{
    global $compress_scripts, $concatenate_scripts;
    $compress_scripts = 1;
    $concatenate_scripts = 1;
    define('ENFORCE_GZIP', true);
 
    /* enqueue your scripts here */
}

Putting the above in your functions.php file may result in insanely good compression on your subsequently-enqueued JavaScript files. I say may because GZIP is one of those fidgety server-side things that varies from hosting provider to hosting provider. I tested this approach on two vastly different environments and had good success, but as always with these things, YMMV.

While this may look rather silly and simple, figuring out exactly which constants and globals to set and when to set them took me just about forever.

The most important thing to note is that this only works on scripts that come pre-packaged with WordPress and are listed in the wp_default_scripts() function in wp-includes/script-loader.php. This is slightly irritating.

When it Does Work

The above will concatenate and GZIP all of the applicable enqueued JavaScript into a single request, served out through the wp-admin/load-scripts.php script.

  • By “applicable”, I mean scripts listed in wp-includes/script-loader.php in the wp_default_scripts() function.
  • If you have applicable JavaScript in both the head and the foot, you will end up with two script src tags. If everything is in the foot (or the head), you’ll get one tag. You not only get GZipping, you get fewer HTTP requests! Yay!
  • JavaScript in plugins and your own theme won’t get concatenated and GZipped in this approach, but it certainly doesn’t break it.

In testing, the following:

    wp_enqueue_script('jquery');
    wp_enqueue_script('thickbox');
    wp_enqueue_script('scriptaculous');
    wp_enqueue_script('editor');

Went from:

10 requests, 310kB

to

2 requests, 90kB

with the activation of the above function. Nice! I got two requests because the above enqueueing puts stuff in both the head and the footer.

That’s a 340% improvement on filesize alone.

Why Won’t It Work for All JavaScript?

Because there are no, nada, zilch, none hooks for making it do so without doing some deep surgery and writing a plugin. It boils down to a couple of places in code:

From wp-includes/class.wp-scripts.php in the do_item() function:

100
101
102
103
104
105
106
107
108
	if ( $this->in_default_dir($srce) ) {
		$this->print_code .= $this->print_scripts_l10n( $handle, false );
		$this->concat .= "$handle,";
		$this->concat_version .= "$handle$ver";
		return true;
	} else {
		$this->ext_handles .= "$handle,";
		$this->ext_version .= "$handle$ver";
	}

To get the built-in concatenation and GZIPped delivery, it is necessary that the if statement here evaluate true. I won’t explain the nitty gritty unless you really want me to.

So, my first approach was to add paths to the $wp_scripts->default_dirs Array to add entries for my theme’s scripts and my plugin dir’s scripts. Big fat fail.

The reason is that the actual file — wp-admin/load-scripts.php — that serves up the concatenated, zipped JavaScript is a standalone script. I imagine that this is both for security and for performance. It does not have the complement of WordPress constants and hooks that we are accustomed to. In fact, it is completely and absolutely unpluggable (unless I missed something). It only includes the files that define the WP_Scripts and WP_Dependencies classes, and utility functions for those. It does not include any theme- or plugin-level items.

And it does something that completely shuts down the game. It instantiates a new, clean WP_Scripts object and then calls wp_default_scripts().

If you’ll recall, the WP_Scripts object in a normal WordPress request is a global singleton that manages all of the enqueued scripts. By creating a new instantiation, the load-scripts.php script has obliterated any notion of any enqueued scripts. Further, the wp_default_scripts() function is a hard-coded list of the JavaScript files that WordPress comes packaged with. While this function nominally triggers the ‘wp_default_scripts’ action, this is not usable here because no files are getting included in which I could take advantage of that hook. Further, it redefines do_action() and other API functions as blank functions that do nothing.

The load-scripts.php file then iterates through the requested JavaScript handles (comma-delimited GET) and sees if it recognizes them from the wp_default_scripts() list. As anything that’s in your own theme or in a plugin won’t be in this list, FAIL.

Though I see some possible options for me in the writing-a-plugin realm, I also sadly see that it would replicate functionality and am highly curious as to what the WordPress developers had in mind here.

Getting all JavaScript into the Footer in WordPress? Not so fast, Buster!

[toc title="JavaScript in Footer"]
Warning: Technical WordPress post ahead!

Overview: Really Getting JavaScript Into the Footer

Quoth the WordPress Version 2.8 feature list:

– Improvements to the script loader: allows plugins to queue scripts for the front end head and footer, adds hooks for server side caching of compressed scripts, adds support for ENFORCE_GZIP constant (deflate is used by default since it’s faster)

At the time, I thought Wow, cool. When I have time, I’ll investigate that and then immediately forgot about it for a few months. During RSS-coffee-breaks I read Lester Chan’s post about how to put JavaScript in the footer (sounds easy enough!) and Andrew Ozz’s post, which left me coated with an intense and foolish optimism about compression.

What I aim to do in this post — part one of a series of three if I find time to investigate the second and third pieces — is explain why it’s not as easy as you’d expect to get some scripts (specifically scripts that come with WordPress by default) into the footer, and how you can make it happen.

If you’re in a hurry, you can skip to the Summary section at the end of the post.

WP JavaScript Inclusion: Header vs. Footer

What I read had me believe that it is falling-off-log easy to put all WordPress JavaScript in the footer. If you are trying to include a piece of JavaScript, say, in a plugin, that WP has not previously known about, it is that easy. But woe if you try this on certain scripts that come packaged with WordPress.

Let’s take a look at the functions that are involved in letting WordPress know that you want to use a given piece of JavaScript.

These are taken from wp-includes/functions.wp-scripts.php.

wp_register_script() and wp_enqueue_script()

37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
/**
 * Register new JavaScript file.
 *
 * @since r16
 * @see WP_Dependencies::add() For parameter information.
 */
function wp_register_script( $handle, $src, $deps = array(), $ver = false, $in_footer = false ) {
	global $wp_scripts;
	if ( !is_a($wp_scripts, 'WP_Scripts') )
		$wp_scripts = new WP_Scripts();
 
	$wp_scripts->add( $handle, $src, $deps, $ver );
	if ( $in_footer )
		$wp_scripts->add_data( $handle, 'group', 1 );
}
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
/**
 * Enqueues script.
 *
 * Registers the script if src provided (does NOT overwrite) and enqueues.
 *
 * @since r16
 * @see WP_Script::add(), WP_Script::enqueue()
*/
function wp_enqueue_script( $handle, $src = false, $deps = array(), $ver = false, $in_footer = false ) {
	global $wp_scripts;
	if ( !is_a($wp_scripts, 'WP_Scripts') )
		$wp_scripts = new WP_Scripts();
 
	if ( $src ) {
		$_handle = explode('?', $handle);
		$wp_scripts->add( $_handle[0], $src, $deps, $ver );
		if ( $in_footer )
			$wp_scripts->add_data( $_handle[0], 'group', 1 );
	}
	$wp_scripts->enqueue( $handle );
}

As a theme developer, you may only ever have encounters with wp_enqueue_script(). But let’s look at what each of these functions does so I can then explain the issue.

wp_register_script() tells WordPress about a script, but does not actually cause it to be included in a given page request. By default, WordPress registers a gripload of scripts that are then available to you, the theme hacker, when or if you should need them. A list of these can be found in wp-includes/script-loader.php in the wp_defalut_scripts() function. Highlights include jQuery, prototype, scriptaculous, etc., as well as extensions to those frameworks (e.g. jQuery UI). WordPress (via the WP_Scripts class, itself extended from WP_Dependencies) handles dependencies and makes sure that jQuery UI doesn’t get included without its necessary jQuery, if you should forget to enqueue jQuery itself or it gets enqueued after jQuery UI.

wp_enqueue_script() tells WordPress, hey, I actually need this script, in this request. You may have seen or done something like this:

wp_enqueue_script('jquery');

This spools up jQuery and spits out a script tag to include the requested script.

In a lot of cases, it seems easy and straightforward to do this simple enqueue request. jQuery comes packaged with WP and is automatically registered for you, so you just have to hand the wp_enqueue_script() function one argument: $handle.

As a clever theme hacker, you may have noticed that the signature for wp_enqueue_script() and wp_register_script() changed in version 2.8 and this seems exciting. An $in_footer parameter was added:

wp_enqueue_script( $handle, $src = false, $deps = array(), $ver = false, $in_footer = false );

Totally psyched, you update your theme and use:

wp_enqueue_script('jquery','','','',true);

And wait for the magic. RELOAD! Wait, maybe something’s cached. RELOAD! Wait, what? RELOADRELOADRELOAD.

jQuery is still in the head.

So you get irritable and you quit trying, or like me you spend a few hours deactivating all of your plugins and trying to figure out where the problem is. You notice that the same thing happens with other scripts that WP already knows about (that is, the scripts in wp_default_scripts()). Stubbornly, they won’t get out of your page’s head element.

Here’s why.

Why WordPress Default JavaScripts Won’t Move to the Footer

The crux is: if the script you are enqueueing has anything defined as a dependency in WordPress’ wp_default_scripts() function, it will ignore your request to put it in the footer (I’ll provide a workaround shortly).

The reason for this is twofold.

Dependency Handling and “Groups”

Part of WordPress’ dependency handling for scripts involves “groups”, the ins and outs of which I’ll leave as an exercise for an intrigued reader. In an oversimplification, it put scripts with dependencies (e.g. jQuery, Scriptaculous) in groups[0]. It explicitly puts scripts that depend on other scripts in groups[1]. The default behavior of both wp_register_script() is to put a script in groups[0] if the $in_footer argument != true.

How Groups Handling does Something Possibly Confusing

OK, now check out this snippet from class.wp-scripts.php (do_item() method):

85
86
87
88
if ( 0 === $group && $this->groups[$handle] > 0 ) {
	$this->in_footer[] = $handle;
	return false;
}

This piece of logic puts groups > 0 into the footer. So that’s how WP decides to move things into the footer–it’s based on where the script is in the $wp_scripts->groups Array.

But here’s a potential pitfall in the wp_enqueue_script() function:

	if ( $src ) {
		$_handle = explode('?', $handle);
		$wp_scripts->add( $_handle[0], $src, $deps, $ver );
		if ( $in_footer )
			$wp_scripts->add_data( $_handle[0], 'group', 1 );
	}
	$wp_scripts->enqueue( $handle );

If you don’t give wp_enqueue_script() a $src (which you don’t have to provide do if the script has already been registered) for your $handle, it is going to enqueue it in whatever group it’s already registered in. So, when you do your little simple enqueue:

wp_enqueue_script('jquery','','','',true);

The entire part of the function where it would put it in the footer doesn’t execute because the if($src) test fails. And jQuery has already been registered—in groups[0].

Possible Workaround

I don’t like this but it works:

wp_enqueue_script('jquery','/wp-includes/js/jquery/jquery.js','','',true);

jQuery is now in the footer.

This does still appear to handle dependencies correctly, but I haven’t deeply tested. Also, there are some valid reasons to have some scripts, and possibly jQuery, in the head. This was just a demo.

Getting JS into the Footer: Quick Summary

// This will NOT put jquery in the footer
wp_enqueue_script('jquery','','','',true);
 
// But this will, if inelegant and circumventing abstraction
wp_enqueue_script('jquery','/wp-includes/js/jquery/jquery.js','','',true);

For My Next Trick…Compression

Really, this post was a red herring. I tripped into this oddity while investigating the second part of the WordPress 2.8 feature claim: adds hooks for server side caching of compressed scripts, adds support for ENFORCE_GZIP constant (deflate is used by default since it’s faster). We’ll talk about that next time.

PHP Speedy Update Fixes CSS Compression Bug

Hi there! Just a quick note for people following our ongoing WordPress performance discussion. Leon Chevalier has released a new version of his PHP Speedy WordPress plugin. This update fixes the CSS compression bug we mentioned in our previous post about PHP Speedy for WordPress 2.7. If you’re using an older version of the plugin I recommend that you upgrade to this latest version, 0.5.2. And if you’re not using the PHP Speedy plugin, I recommend that you give it a try.

Happy performance hunting!

Easy Steps To Speedup Your WordPress Blog

We recently finished optimizing several WordPress blogs for improved performance and I thought “wouldn’t it be great to share our experiences with everyone?” After all, when I look around at all the blogs I know and use, I see that very few are taking advantage of even the simplest performance improvement techniques — surely it’s just a matter of getting the information out there for everyone to use?

A quick search for “wordpress speedup” or “wordpress optimization“, however, reveals that we aren’t suffering from a dearth of information. On the contrary, many, many people have posted articles about their specific techniques and experiences (I’ve listed the best of these in the Related Articles section; you really should read them). This leads me to believe that most bloggers are either apathetic about their site performance or intimidated by the prospect of diving into something they know little about. I tend to think it’s probably the latter, so let’s see if we can make it a little less scary, eh?

Here at Cloud Four, we are big fans of the work done by the Yahoo! Exceptional Performance team to research and publish best practices for improving web performance. For us, the big finding was that most of the end-user response time — as much as 80% of the total page delivery time — occurs after the server has finished all its back-end work to create the page. Why such a long delay? Well, we’ve all been steadily increasing the number of Javascript, stylesheet, and image references on our fancy blog pages, and that means the browser has a lot of work to do before it can completely render the page for the reader. This finding is good news for most of us, since improving the page layout and rendering behavior are well within the skillset of almost anyone who can setup WordPress and install plugins.

What You Should Do Right Now

Here’s the short answer: install a couple plugins, add a few lines to your .htaccess file, and reap the rewards. It is almost certainly guaranteed that you will see an immediate improvement in your site performance, shaving seconds off each page load and impressing your readers. If you truly want to know how and why all this stuff works, you’ll enjoy reading the related articles for more information and insight.

1. Use Firefox, Firebug, and YSlow to measure yourself


The Yahoo! team has published their 34 rules for speeding up your web site and a tool for evaluating web sites against those rules. Strictly speaking, you do not need to install these tools, but I think you will enjoy seeing your YSlow score move from where it is today (probably an “F”) to where it should be (at least a “B”). And joy of joys, your actual site performance will improve accordingly.

2. Enable GZIP compression


Add the following lines to your .htaccess file (the AddOutputFilterByType should all be on one line). This action tells the web server to compress files before sending them to the browser, resulting in reduced bandwidth and faster delivery to the browser. Adding compression is a very safe option.


<IfModule mod_deflate.c>
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css application/javascript application/x-javascript application/x-httpd-php application/rss+xml application/atom_xml
</IfModule>

3. Add an Expires Header


Add the following lines to your .htaccess file. This action tells the web server to add an Expires header for files that are unlikely to change in the near future. The reader’s browser will then cache the files until the expiration date, not even bothering to check for a new, updated version in the meantime. Note that if you DO change any files with a far future expiration date you will need to change the file’s name, otherwise any browser with a cached version will fail to fetch your new version. You are most likely to experience this issue with CSS or Javascript files, so don’t say I didn’t warn you.

    <IfModule mod_expires.c>
      ExpiresActive on
      ExpiresByType image/gif "access plus 1 month"
      ExpiresByType image/jpeg "access plus 1 month"
      ExpiresByType image/png "access plus 1 month"
      ExpiresByType text/css "access plus 1 month"
      ExpiresByType application/javascript "access plus 1 month"
      ExpiresByType application/x-javascript "access plus 1 month"
    </IfModule>
  

If mod_expires is not available on your system, you can try this instead:

    <FilesMatch ".(ico|jpg|jpeg|png|gif|js|css)$">
      Header set Expires "Sun, 22 Apr 2018 01:10:54 GMT"
      Header set Cache-Control "max-age=315360000"
      Header unset Pragma
    </FilesMatch>
  
4. Disable ETags


Add the following line to your .htaccess file. This action tells the web server to remove ETags from the HTTP response. For most websites, removing ETags will not affect your performance at all, but it will increase your YSlow score. Removing ETags is a very safe option.

    FileETag none
  
5. Install the PHP Speedy WordPress Plugin


The PHP Speedy WordPress Plugin is a real gem, implementing several of the Yahoo! best practices: combining multiple files to reduce HTTP requests, minifying Javascript and CSS, compressing the files via GZIP compression, and adding far future Expires headers. You should configure PHP Speedy to minify everything, compress Javascript and CSS, and add far future expire headers for both Javascript and CSS. There is no reason to enable GZIP compression for the web page itself, since we are doing that via apache.

Since this plugin modifies the way your page references Javascript and CSS files, take some care to test your site after activating your configuration. PHP Speedy appears to preserve the load order of the files, which is the main thing I care about. And if necessary you can exclude specific files from this optimization step. Still, I recommend that you test like crazy on this one.

6. Install the WP Super Cache Plugin


The WP Super Cache Plugin helps performance by dramatically reducing the time it takes the server to deliver the initial HTML file for the page. There are other steps you can take to optimize your server-side performance, such as eliminating unnecessary plugins and optimizing your page templates, but while you’re thinking about all that, this one little action will make a world of difference. My one configuration note here is to go ahead and enable “Super Cache Compression.” This option is disabled by default, but I recommend that you enable it and just run a quick test to ensure the page is being delivered correctly.

You can verify that the cache is configured and working correctly by visiting the site as an anonymous user and viewing the page source. Compressed pages being served by the cache should by quite fast and the last three lines of the page file will look something like the following.

      <!-- Dynamic Page Served (once) in 1.422 seconds -->
      <!-- Cached page served by WP-Super-Cache -->
      <!-- Compression = gzip -->
  

Results!

By using these techniques, and by adding a bit of image optimization and consolidation, we have seen dramatic performance improvements for the sites we’ve optimized. Typical page loading times have dropped from 5-10 seconds to one second (or less for pages cached by WP Super Cache).

Without sacrificing any page functionality or quality, we have been able to:

  • reduce the empty cache total size by 50% or more
  • reduce the empty cache HTTP requests by 50% or more
  • reduce the primed cache total size by 90% (!)
  • reduce the primed cache HTTP requests to as few as 1 or 2 requests (!)

Where Next?

Well this first part was certainly easy: edit one file (.htaccess) and install two WordPress plugins. These simple steps have probably provided some immediate relief for your readers, and that’s a good thing. From here, I would strongly suggest you review the related articles for other front-end improvements, such as optimizing images and using sprites to reduce the number CSS images on your site. There’s quite a bit more you can do without becoming a programmer or system administrator.

If you do have administrative access to your server, however, you can also begin experimenting with ideas to improve the back-end server performance. You might consider installing a PHP accelerator, such as eAccelerator, to speedup PHP script execution. Or you might want to expend some effort tuning MySQL for your site. For most people, however, these advanced topics are beyond their skill and training, or are simply unavailable in their current hosting environment.

If you decide try any of the techniques in this article, I’d like to hear about your experience, and about what you think you’ll try next.

Good luck!

WordPress Plugins You Need

Resources