Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
By jeremy at 2008-03-14 11:18
Get Glossy with Varnish
Jeremy Garcia
Linux Magazine
In many of the recent" Tech Support" columns, I've shown you how to optimize and scale your Web infrastructure. This has ranged from fine tuning PHP and MySQL, to load balancing with Perlbal or Pound. To properly scale your infrastructure, you need to evaluate your applications and meet their specific needs. You have a variety of fantastic open source Web infrastructure components to choose from, and picking the correct components is as difficult as ever. To help sort through the cruft, I want to introduce you to a particularly useful app called Varnish, a high-performance HTTP accelerator.
What type of applications would benefits from Varnish? The primary target is dynamic applications that are modified relatively infrequently when compared to reads. Apps such as blogs and online news publications are great examples, but you'll need to look at your exact usage patterns to be sure. Keep in mind that HTTP accelerator is really just a buzzword for a caching reverse proxy-- which is to say, an application that sits in front of a server and routes requests to the server or handles the request itself.
The reverse proxies I've covered previously, Pound and Perlbal, are meant more for load balancing and do not perform any kind of content caching. Unlike Squid, which added Web acceleration as an additional feature to a product meant primarily as a forward proxy, Varnish is designed from the ground up to be an HTTP accelerator. It's available from http://varnish.projects.linpro.no/ under a BSD license.
Getting Varnish
Binary packages for varnish are available for most major distributions. If binaries aren't available for your distribution, or you'd like to compile from source, installation is the standard ./configure && make && make install. Varnish uses the Varnish Configuration Language (VCL), for all config directives. VCL is the heart of varnish, so you'll need to get to know it very well to get the most out of your cache. The VCL syntax is fairly simple, and deliberately similar to C and Perl. Blocks are delimited by curly braces, statements end with semicolons, and comments may be written as in C, C++ or Perl according to your own preferences. Regular expressions are used heavily.
The first item you'll need to add to your Varnish configuration is which server (s) you'd like to process backend requests. You can have any number of backend content servers, for simplicity the example below has only one.
Code:
backend default {
set backend.host = "192.168.1.1";
set backend.port = "80";
}
With the backend server defined, we can now move on to access control lists (ACLs). You can set up ACLs to bypass the cache for certain machines in an office, test the cache from only your workstation or even for more advanced traffic control. The following ACL will allow 192.168.1.0/24 and deny 192.168.2.0/24.
The remaining VCL directives allow you to define request handling and document caching policies. You have the following blocks available to you: vcl_recv, vcl_pipe, vcl_pass, vcl_hash, vcl_hit, vcl_miss, vcl_fetch, vcl_deliver, vcl_timeout and vcl_discard. Covering each directive in detail is beyond the scope of this article, but the VCL man page has an explanation of each along with some example configuration code.
I will cover a couple brief examples to get you started, but would like to reiterate that for you to get the most out of Varnish you'll need to have both a good understanding of your application behavior and VCL.
vcl_recv is one of the main VCL directives. Its purpose is to decide whether or not to serve the request, how to do it, and, if applicable, which backend to use. To cache all images and CSS, use a block similar to the following.
Code:
sub vcl_recv {
if (req.url ~ "\.(css|gif|png|jpg)$") {
lookup;
}
}
You can use multiple directives and set a default actions as follows.
Code:
sub vcl_recv {
if (req.request != "GET" && req.request != "HEAD") {
pipe;
}
if (req.http.Authenticate || req.http.Cookie) {
pass;
}
lookup;
}
Once you have a Varnish policy you're happy with, it's time to start varnishd.
The above will start varnish as the nobody user on 192.168.3.1 port 80 with the management interface running on port 2222 of localhost. It'll use a 1GB cache file named storage.bin in /var/varnish using the policy you've defined in /path/to/VCL.config. For additional startup parameters, see man varnishd.
The management interface is a command line interface that can be accessed using telnet. It allows you to get statistics, dynamically reload VCL policies, stop/start the varnishd process and much more. The full details are covered in the varnishd man page.
Properly setting up and configuring Varnish is far from trivial. You'll need to know a fair amount about your application usage and get familiar with VCL. If you're a high traffic Web site, however, the learning curve will be well worth it.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.