pro_apache_third_edition..pdf

Contents About the Author...............................................................................................xix About the Technical Reviewer and Contributing Author.................xxi Chapter1 Apache and the Internet..............................................1 Apache: The Anatomy of a Web Server.........................................................1 The Apache Source .............................................................................................1 The Apache License............................................................................................1 Support for Apache.............................................................................................2 How Apache Works..............................................................................................3 The Hypertext Transfer Protocol..................................................................7 HTTP Requests and Responses..........................................................................7 HTTP Headers...................................................................................................12 Networking and TCP/IP......................................................................................13 Definitions.........................................................................................................13 Packets and Encapsulation...............................................................................14 ACKs, NAKs, and Other Messages....................................................................15 The TCP/IP Network Model.............................................................................16 Non-IP Protocols...............................................................................................19 IP Addresses and Network Classes...................................................................19 Special IP Addresses..........................................................................................20 Netmasks and Routing......................................................................................21 Web Services: Well-Known Ports......................................................................23 Internet Daemon: The Networking Super Server...........................................24 The Future: IPv6................................................................................................25 Networking Tools...............................................................................................26 Server Hardware...................................................................................................29 Supported Platforms.........................................................................................29 Basic Server Requirements...............................................................................30 Memory..............................................................................................................31 Network Interface..............................................................................................32 Internet Connection.........................................................................................32 Hard Disk and Controller.................................................................................33 Operating System Checklist.............................................................................33 Redundancy and Backup..................................................................................34 Specific Hardware Solutions............................................................................35 Get Someone Else to Do It.............................................................................36 Summary....................................................................................................................36 v 3006_Ch00_CMP2 12/14/03 8:56 AM Page v Chapter 2 Getting Started with Apache.................................37 Installing Apache..............................................................................................38 Getting Apache..................................................................................................38 Installing Apache from Binary Distribution....................................................39 Installing Apache from Source.........................................................................41 Installing Apache from Prebuilt Packages.......................................................41 Installing Apache by Hand...............................................................................45 Upgrading Apache.............................................................................................47 Other Issues.......................................................................................................49 Basic Configuration..........................................................................................50 Decisions............................................................................................................50 Introducing the Master Configuration File.....................................................55 Other Basic Configuration Directives..............................................................56 Starting, Stopping, and Restarting the Server.................................57 Starting Apache on Unix...................................................................................58 Starting Apache on Windows...........................................................................59 Invocation Options...........................................................................................60 Restarting the Server.........................................................................................73 Stopping the Server...........................................................................................75 Starting the Server Automatically....................................................................76 Testing the Server............................................................................................81 Testing with a Browser......................................................................................82 Testing from the Command Line or a Terminal Program..............................82 Testing the Server Configuration Without Starting It.....................................85 Getting the Server Status from the Command Line.......................................86 Using Graphical Configuration Tools.......................................................86 Comanche..........................................................................................................87 TkApache...........................................................................................................91 LinuxConf..........................................................................................................91 Webmin..............................................................................................................91 ApacheConf.......................................................................................................97 Other Configuration Tools................................................................................99 Summary..................................................................................................................100 Chapter 3 Building Apache the Way You Want It...........101 Why Build Apache Yourself?.........................................................................101 Verifying the Apache Source Archive.............................................................103 Building Apache from Source......................................................................105 Configuring and Building Apache.................................................................106 Determining Which Modules to Include.......................................................111 Building Apache As a Dynamic Server..........................................................116 Contents vi 3006_Ch00_CMP2 12/14/03 8:56 AM Page vi Changing the Module Order (Apache 1.3)....................................................118 Checking the Generated Configuration........................................................120 Building Apache from Source As an RPM (Apache 2)..................................122 Advanced Configuration.................................................................................124 Configuring Apache’s Layout..........................................................................124 Choosing a Layout Scheme............................................................................124 Choosing a Multiprocessing Module (Apache 2)..........................................132 Rules (Apache 1.3)...........................................................................................135 Building Apache with suExec support...........................................................137 Configuring Apache’s Supporting Files and Scripts.....................................139 Configuring Apache 2 for Cross-Platform Builds.........................................140 Configuring Apache for Production or Debug Builds..................................142 Configuring Apache for Binary Distribution.................................................143 Configuring Apache’s Library and Include Paths..........................................143 Configuring the Build Environment.........................................................144 Building Modules with configure and apxs..........................................146 Adding Third-Party Modules with configure................................................146 Building Modules with apxs...........................................................................148 Installing Modules with apxs..........................................................................150 Generating Module Templates with apxs......................................................151 Overriding apxs Defaults and Using apxs in makefiles................................152 Summary..................................................................................................................153 Chapter 4 Configuring Apache the Way You Want It...155 Where Apache Looks for Its Configuration..........................................155 Configuration File Syntax...............................................................................156 Configuration for Virtual Hosts......................................................................156 Including Multiple Configuration Files.........................................................157 Per-Directory Configuration..........................................................................159 Conditional Configuration.............................................................................160 How Apache Structures Its Configuration............................................163 Apache’s Container Directives........................................................................164 Directive Types and Locations.......................................................................168 Where Directives Can Go................................................................................171 Container Scope and Nesting.........................................................................172 How Apache Combines Containers and Their Contents.............................174 Legality of Directives in Containers...............................................................175 Options and Overrides....................................................................................176 Enabling and Disabling Features with Options............................................176 Overriding Directives with Per-Directory Configuration.............................179 Contents vii 3006_Ch00_CMP2 12/14/03 8:56 AM Page vii Restricting Access with allow and deny..............................................182 Controlling Access by Name...........................................................................183 Controlling Access by IP Address...................................................................184 Controlling Subnet Access by Network and Netmask..................................185 Controlling Access by HTTP Header.............................................................186 Combining Host-Based Access with User Authentication...........................187 Overriding Host-Based Access.......................................................................188 Directory Listings..........................................................................................188 Enabling and Disabling Directory Indices....................................................189 How mod_autoindex Generates the HTML Page.........................................190 Controlling Which Files Are Seen with IndexIgnore.....................................196 Controlling the Sort Order..............................................................................197 Assigning Icons................................................................................................199 Assigning Descriptions...................................................................................202 Apache’s Environment......................................................................................203 Setting, Unsetting, and Passing Variables from the Shell.............................204 Setting Variables Conditionally......................................................................205 Special Browser Variables...............................................................................207 Detecting Robots with BrowserMatch...........................................................209 Passing Variables to CGI.................................................................................209 Conditional Access Control............................................................................210 Caveats with SetEnvIf vs. SetEnv....................................................................210 Setting Variables with mod_rewrite...............................................................211 Controlling Request and Response Headers..........................................211 Setting Custom Response Headers................................................................213 Setting Custom Request Headers...................................................................215 Inserting Dynamic Values into Headers........................................................216 Setting Custom Headers Conditionally.........................................................217 Retrieving Response Headers from Metadata Files......................................217 Setting Expiry Times.......................................................................................219 Sending Content As-Is....................................................................................222 Controlling the Server Identification Header.................................223 Sending a Content Digest.............................................................................224 Handling the Neighbors.................................................................................225 Controlling Robots with robots.txt................................................................226 Controlling Robots in HTML..........................................................................227 Controlling Robots with Access Control........................................................227 Attracting Robots.............................................................................................228 Making Sure Robots Index the Right Information........................................228 Known Robots, Bad Robots, and Further Reading.......................................229 Summary..................................................................................................................229 Contents viii 3006_Ch00_CMP2 12/14/03 8:56 AM Page viii Chapter 5 Deciding What the Client Needs........................231 Content Handling and Negotiation...........................................................231 File Types.........................................................................................................232 File Encoding...................................................................................................236 File Languages.................................................................................................243 File Character Sets...........................................................................................245 Handling URLs with Extra Path Information................................................247 Content Negotiation.......................................................................................248 Content Negotiation with MultiViews...........................................................250 File Permutations and Valid URLs with MultiViews.....................................256 Magic MIME Types..........................................................................................260 Error and Response Handling......................................................................264 How Apache Handles Errors...........................................................................265 Error and Response Codes.............................................................................265 The ErrorDocument Directive.......................................................................266 Limitations of ErrorDocument......................................................................270 Aliases and Redirection...............................................................................271 Aliases and Script Aliases................................................................................271 Redirections.....................................................................................................273 Rewriting URLs with mod_rewrite.................................................................277 Server-Side Image Maps.................................................................................300 Matching Misspelled URLS............................................................................305 Summary..................................................................................................................306 Chapter 6 Delivering Dynamic Content..................................307 Server-Side Includes......................................................................................308 Enabling SSI.....................................................................................................309 Format of SSI Commands...............................................................................311 The SSI Command Set....................................................................................312 SSI Variables.....................................................................................................312 Passing Trailing Path Information to SSIs (and Other Dynamic Documents).................................................................315 Setting the Date and Error Format.................................................................316 Templating with SSIs.......................................................................................317 Caching Server-Parsed Documents...............................................................319 Identifying Server-Parsed Documents by Execute Permission...................320 CGI: The Common Gateway Interface.........................................................321 CGI and the Environment..............................................................................321 Configuring Apache to Recognize CGI Scripts.............................................323 Setting Up a CGI Directory with ExecCGI: A Simple Way............................327 Triggering CGI Scripts on Events...................................................................330 Contents ix 3006_Ch00_CMP2 12/14/03 8:56 AM Page ix ISINDEX-Style CGI Scripts and Command Line Arguments................332 Writing and Debugging CGI Scripts.........................................................333 A Minimal CGI Script......................................................................................333 Interactive Scripts: A Simple Form................................................................337 Adding Headers...............................................................................................338 Debugging CGI Scripts....................................................................................339 Setting the CGI Daemon Socket.....................................................................345 Limiting CGI Resource Usage.........................................................................346 Actions, Handlers, and Filters................................................................347 Handlers...........................................................................................................348 Filters................................................................................................................354 Dynamic Content and Security....................................................................363 CGI Security Issues..........................................................................................363 Security Advice on the Web............................................................................364 Security Issues with Apache CGI Configuration...........................................364 An Example of an Insecure CGI Script..........................................................365 Known Insecure CGI Scripts...........................................................................370 CGI Wrappers...................................................................................................370 Security Checklist............................................................................................380 Inventing a Better CGI Script with FastCGI......................................381 Summary..................................................................................................................403 Chapter 7 Hosting More Than One Web Site........................405 Implementing User Directories with UserDir......................................406 Enabling and Disabling Specific Users..........................................................407 Redirecting Users to Other Servers................................................................408 Alternative Ways to Implement User Directories.........................................409 Separate Servers..............................................................................................410 Restricting Apache’s Field of View..................................................................411 Specifying Different Configurations and Server Roots................................412 Starting Separate Servers from the Same Configuration.............................412 Sharing External Configuration Files.............................................................413 IP-Based Virtual Hosting.............................................................................414 Multiple IPs, Separate Networks, and Virtual Interfaces..............................415 Configuring What Apache Listens To.............................................................416 Defining IP-Based Virtual Hosts.....................................................................418 Virtual Hosts and the Server-Level Configuration........................................421 Specifying Virtual Host User Privileges..........................................................422 Excluded Directives.........................................................................................426 Default Virtual Hosts....................................................................................427 Contents x 3006_Ch00_CMP2 12/14/03 8:56 AM Page x Name-Based Virtual Hosting.........................................................................428 Defining Named Virtual Hosts.......................................................................428 Server Names and Aliases...............................................................................430 Defining a Default Host for Name-Based Virtual Hosting...........................430 Mixing IP-Based and Name-Based Hosting..................................................431 Issues Affecting Virtual Hosting...........................................................434 Log Files and File Handles..............................................................................434 Virtual Hosts and Server Security..................................................................436 Secure HTTP and Virtual Hosts......................................................................437 Handling HTTP/1.0 Clients with Name-Based Virtual Hosts......................439 Dynamic Virtual Hosting...............................................................................441 Mass Hosting with Virtual-Host Aliases........................................................441 Mapping Hostnames Dynamically with mod_rewrite.................................448 Generating On the Fly and Included Configuration Files with mod_perl..449 Summary..................................................................................................................455 Chapter 8 Improving Apache’s Performance........................457 Apache’s Performance Directives..............................................................458 Configuring MPMs: Processes and Threads..................................................459 Network and IP-Related Performance Directives.........................................470 HTTP-Related Performance Directives.........................................................472 HTTP Limit Directives....................................................................................475 Configuring Apache for Better Performance........................................477 Directives That Affect Performance...............................................................477 Additional Directives for Tuning Performance.............................................482 Benchmarking Apache’s Performance.........................................................490 Benchmarking Apache with ab......................................................................490 Benchmarking Apache with gprof.................................................................495 External Benchmarking Tools........................................................................496 Benchmarking Strategy and Pitfalls...............................................................496 A Performance Checklist...............................................................................497 Proxying................................................................................................................498 Installing and Enabling Proxy Services..........................................................498 Normal Proxy Operation.................................................................................499 Configuring Apache As a Proxy......................................................................500 URL Matching with Directory Containers....................................................502 Blocking Sites via the Proxy............................................................................504 Localizing Remote URLs and Hiding Servers from View.............................504 Relaying Requests to Remote Proxies............................................................508 Proxy Chains and the Via Header...................................................................509 Proxies and Intranets......................................................................................512 Handling Errors...............................................................................................512 Contents xi 3006_Ch00_CMP2 12/14/03 8:56 AM Page xi Timing Out Proxy Requests............................................................................514 Tunneling Other Protocols.............................................................................514 Tuning Proxy Operations................................................................................515 Squid: A High-Performance Proxy Alternative.............................................516 Caching..................................................................................................................516 Enabling Caching............................................................................................516 File-Based Caching.........................................................................................517 In-Memory Caching (Apache 2 Only)............................................................520 Coordinating Memory-Based and Disk-Based Caches................................522 General Cache Configuration.........................................................................522 Maintaining Good Relations with External Caches......................................527 Fault Tolerance and Clustering................................................................529 Backup Server via Redirected Secondary DNS.............................................530 Load Sharing with Round-Robin DNS...........................................................531 Backup Server via Floating IP Address..........................................................531 Hardware Load Balancing..............................................................................532 Clustering with Apache...................................................................................533 Other Clustering Solutions.............................................................................536 Summary..................................................................................................................537 Chapter 9 Monitoring Apache.........................................................539 Logs and Logging..............................................................................................539 Log Files and Security.....................................................................................540 The Error Log...................................................................................................540 Setting the Log Level.......................................................................................541 Logging Errors to the System Log..................................................................542 Transfer Logs...................................................................................................544 Driving Applications Through Logs...............................................................554 Log Rotation....................................................................................................556 Lies, Logs, and Statistics.........................................................................560 What You Can’t Find Out from Logs...............................................................560 Analog: A Log Analyzer...................................................................................561 Server Information..........................................................................................577 Server Status....................................................................................................578 Server Info........................................................................................................581 Securing Access to Server Information.........................................................582 User Tracking.....................................................................................................583 Alternatives to User Tracking.........................................................................584 Cookie Tracking with mod_usertrack............................................................584 URL Tracking with mod_session....................................................................589 Other Session Tracking Options.....................................................................594 Summary..................................................................................................................595 Contents xii 3006_Ch00_CMP2 12/14/03 8:56 AM Page xii Chapter 10Securing Apache..............................................................597 User Authentication........................................................................................597 Apache Authentication Modules...................................................................598 Authentication Configuration Requirements...............................................599 Using Authentication Directives in .htaccess...............................................601 Basic Authentication.......................................................................................601 Digest Authentication.....................................................................................603 Anonymous Authentication...........................................................................606 Setting Up User Information..........................................................................606 Specifying User Requirements.......................................................................614 LDAP Authentication......................................................................................617 Using Multiple Authentication Schemes.......................................................624 Combining User- and Host-Based Authentication......................................626 Securing Basic Authentication with SSL.......................................................627 SSL and Apache...................................................................................................627 Downloading OpenSSL and ModSSL............................................................628 Building and Installing the OpenSSL Library...............................................629 Building and Installing mod_ssl for Apache 2..............................................633 Building and Installing mod_ssl for Apache 1.3...........................................633 Basic SSL Configuration.................................................................................637 Installing a Private Key....................................................................................639 Creating a Certificate Signing Request and Temporary Certificate.............640 Getting a Signed Certificate............................................................................642 Advanced SSL Configuration.........................................................................644 Server-Level Configuration............................................................................644 Client Certification..........................................................................................657 Using Client Certification with User Authentication..................659 SSL and Logging..............................................................................................660 SSL Environment Variables and CGI.............................................................662 SSL and Virtual Hosts......................................................................................666 Advanced Features..........................................................................................668 Summary..................................................................................................................671 Chapter 11Improving Web Server Security..........................673 Apache Features.................................................................................................673 Unwanted Files................................................................................................674 Automatic Directory Indices..........................................................................674 Symbolic Links................................................................................................675 Server-Side Includes.......................................................................................676 ISINDEX-Style CGI Scripts.............................................................................677 Server Tokens...................................................................................................677 Contents xiii 3006_Ch00_CMP2 12/14/03 8:56 AM Page xiii File Permissions..............................................................................................678 Viewing Server Information with mod_info..........................................679 Restricting Server Privileges..................................................................679 Restricting Access by Hostname and IP Address...............................680 Other Server Security Measures................................................................682 Dedicated Server..............................................................................................682 File Integrity...................................................................................................683 md5sum...........................................................................................................684 Tripwire............................................................................................................685 Hardening the Server......................................................................................686 Minimizing Services........................................................................................686 Port Scanning with nmap ..............................................................................688 Probing with Nessus.......................................................................................689 Hardening Windows 2000 and XP..................................................................689 Disabling Network Services.........................................................................690 File Transfer Protocol (FTP)............................................................................690 telnet................................................................................................................690 rlogin, rsh, rexec, rcp.......................................................................................690 Network Filesystem (NFS)..............................................................................690 sendmail/Other Mail Transport Agents (MTAs)...........................................691 Restricting Services with TCP Wrappers........................................................691 Security Fixes, Alerts, and Online Resources.................................693 The WWW Security FAQ..................................................................................693 The BugTraQ Mailing List and Archive..........................................................693 Operating System Newsletters.......................................................................693 Package and Module Notification..................................................................694 Removing Important Data from the Server............................................694 Enabling Secure Logins with SSH..............................................................694 Building and Installing OpenSSH..................................................................695 Authentication Strategies...............................................................................698 Configuring SSH..............................................................................................699 Testing SSH......................................................................................................702 Expanding SSH to Authenticate Users..........................................................703 Secure Server Backups with Rsync and SSH.................................................704 Forwarding Client Connections to Server Applications...............................705 Firewalls and Multifacing Servers.........................................................706 Types of Firewall..............................................................................................706 Designing the Network Topology...................................................................707 Running Apache Under a Virtual chroot Root Directory................709 What chroot Is.................................................................................................709 What chroot Isn’t.............................................................................................710 Setting Up Apache for chroot Operation.......................................................711 Contents xiv 3006_Ch00_CMP2 12/14/03 8:56 AM Page xiv Server Security Checklist...........................................................................723 Avoid Root Services.........................................................................................723 Maintain Logs Properly..................................................................................723 Keep It Simple..................................................................................................724 Block Abusive Clients......................................................................................724 Have an Effective Backup and Restore Process............................................725 Plan for High Availability, Capacity, and Disaster Recovery........................725 Monitor the Server..........................................................................................725 Take Care with Information Flow...................................................................726 Choose an Effective robots.txt Policy............................................................726 Summary..................................................................................................................726 Chapter 12Extending Apache............................................................727 WebDAV....................................................................................................................727 Adding WebDAV to Apache.............................................................................728 The WebDAV Protocol.....................................................................................729 Configuring Apache for WebDAV...................................................................731 Restricting Options and Disabling Overrides...............................................734 WebDAV and Virtual Hosts.............................................................................735 Configuring the DAV Lock Time.....................................................................735 Limitations of File-Based Repositories..........................................................736 Protecting WebDAV Servers............................................................................737 More Advanced Configurations.....................................................................737 Cooperating with CGI and Other Content Handlers....................................740 ISAPI......................................................................................................................741 Supported ISAPI Support Functions.............................................................742 Configuring ISAPI Extensions........................................................................743 Setting the Maximum Initial Request Data Size...........................................744 Logging ISAPI Extensions...............................................................................745 Preloading and Caching ISAPI Extensions....................................................746 Handling Asynchronous ISAPI Extensions...................................................746 Perl.........................................................................................................................746 Building and Installing mod_perl..................................................................748 Migrating mod_perl from Apache 1.3 to Apache 2.......................................755 Configuring and Implementing Perl Handlers.............................................758 Configuring and Implementing Perl Filters..................................................771 Warnings, Taint Mode, and Debugging.........................................................772 Managing Perl Threads in mod_perl 2...........................................................774 Initializing Modules at Startup.......................................................................779 Restarting mod_perl and Auto-Reloading Modules.....................................780 Creating a mod_perl Status Page...................................................................782 Running CGI Scripts Under mod_perl..........................................................782 Contents xv 3006_Ch00_CMP2 12/14/03 8:56 AM Page xv CGI Caveats......................................................................................................785 Passing Variables to Perl Handlers.................................................................787 Using mod_perl with Server-Side Includes...................................................788 Embedding Perl in HTML...............................................................................789 Embedding Perl in Apache’s Configuration..................................................794 PHP...........................................................................................................................795 Installing PHP..................................................................................................796 Getting the PHP source...................................................................................796 Configuring Apache to Work with PHP..........................................................802 Configuring PHP.............................................................................................803 Testing PHP with Apache................................................................................807 Tomcat/Java.........................................................................................................807 So What Is Tomcat?..........................................................................................807 Installation.......................................................................................................808 Tomcat Configuration.....................................................................................813 mod_jk.............................................................................................................818 Mod_python....................................................................................................829 mod_ruby.........................................................................................................835 Summary..................................................................................................................839 Index....................................................................................................................843