WpW: Managing Multiple Cache Crawlers

December 5th, 2018 by LSCache 0 Comments

Welcome to another installment of WordPress Wednesday!
Today’s topic is: Managing Multiple Cache Crawlers

LiteSpeed’s WordPress Cache Crawler travels your sitemap and re-caches any pages that have expired. By default, the crawler is run from the point of view of a non-logged-in guest. This is what that might look like on the LiteSpeed Cache > Crawler page:

This is sufficient for a simple site, but some sites display different content in particular situations (for example, special content for logged-in users, specifically mobile content, or content that relies on the value of a cookie). What if you are in that situation? How do you configure the crawler to keep multiple views warm?

The LSCache crawler can be configured to simulate traffic based on three different criteria:

  1. User roles
  2. Cookies (new!)
  3. Mobile view (new!)

The first two options are controlled by settings on the LiteSpeed Cache > Settings > Crawler tab: Role Simulation and Cookie Simulation.

Mobile view is controlled via the Cache Mobile setting on the LiteSpeed Cache > Settings > Cache tab. Let’s look at each of these configurations.

User Role Simulation

The main advantage of a crawler is that it minimizes the chances that your visitors will encounter uncached content. This is a nice perk for your guests. But what if you have the kind of site where many of your visitors log in? Maybe you have a particular user role that you would like to see warmed up with a crawler?

Navigate to LiteSpeed Cache > Settings > Crawler and add a user ID to the Role Simulation box.

In this example, we entered the ID 2. User number 2 has the role of Editor, so now we have two crawlers:

Now, in addition to crawling the sitemap from the point of view of a non-logged-in user, LSCache will crawl the sitemap from the point of view of a user with the Editor role.

Of course, with more crawlers comes more server load. So you should consider the benefits of a warm cache in the context of how many resources you have available.

Cookie Simulation

What if you have implemented a cache vary based on a cookie value. Wouldn’t it be nice to crawl that view, too? There aren’t many situations where you would need to simulate a cookie crawler, but it can be useful for sites that use a cookie to control multiple languages or currencies.

To create a new crawler for a particular cookie, press the + button. Enter the cookie name, and the values you wish to crawl for. Values should be one per line, and can include a blank line.

There will be one crawler created per cookie value, per simulated role. In other words, this:

Press the + button to add additional cookies, but be aware the number of crawlers grows quickly with each new cookie, and can be a drain on system resources. Let’s add cookie2 with the values C, D, and .

One cookie, eight additional crawlers!

This is a cool feature, but be careful with it, or your server will never get a break from all of the crawling!

Mobile View

One last type of crawler is the Mobile Crawler. This one is automatically created if you have LiteSpeed Cache > Settings > Cache > Cache Mobile set to ON.

Just for fun, let’s see what happens to your crawler list when you crawl a mobile view:

As you have probably guessed, it doubles the number of crawlers you will run.

In Conclusion

You can create crawlers for many different cache vary situations. It’s a handy tool, just remember to crawl responsibly.

Disclaimer: The information contained in this post is accurate for LSCWP v2.8 [release log]. If you are using a newer version of the plugin, some details may have changed. Please refer to our wiki for the latest!

Have some of your own ideas for future WordPress Wednesday topics? Leave us a comment!

We’ll be back soon with another installment. In the meantime, here are a few other things you can do:


Related Posts