HttpClient Tutorial
Oleg Kalnichevski
Jonathan Moore
Jilles van Gurp
ii
Preface ................................................................................................................................... iv
1. HttpClient scope ......................................................................................................... iv
2. What HttpClient is NOT ............................................................................................. iv
1. Fundamentals ....................................................................................................................... 1
1.1. Request execution ..................................................................................................... 1
1.1.1. HTTP request ................................................................................................. 1
1.1.2. HTTP response ............................................................................................... 2
1.1.3. Working with message headers ........................................................................ 2
1.1.4. HTTP entity ................................................................................................... 3
1.1.5. Ensuring release of low level resources ............................................................ 5
1.1.6. Consuming entity content ................................................................................ 6
1.1.7. Producing entity content .................................................................................. 6
1.1.8. Response handlers .......................................................................................... 7
1.2. HttpClient interface ................................................................................................... 8
1.2.1. HttpClient thread safety .................................................................................. 8
1.2.2. HttpClient resource deallocation ...................................................................... 9
1.3. HTTP execution context ............................................................................................ 9
1.4. HTTP protocol interceptors ...................................................................................... 10
1.5. Exception handling .................................................................................................. 11
1.5.1. HTTP transport safety ................................................................................... 11
1.5.2. Idempotent methods ...................................................................................... 12
1.5.3. Automatic exception recovery ........................................................................ 12
1.5.4. Request retry handler .................................................................................... 12
1.6. Aborting requests .................................................................................................... 13
1.7. Redirect handling .................................................................................................... 13
2. Connection management ..................................................................................................... 15
2.1. Connection persistence ............................................................................................. 15
2.2. HTTP connection routing ......................................................................................... 15
2.2.1. Route computation ........................................................................................ 15
2.2.2. Secure HTTP connections ............................................................................. 15
2.3. HTTP connection managers ..................................................................................... 16
2.3.1. Managed connections and connection managers .............................................. 16
2.3.2. Simple connection manager ........................................................................... 16
2.3.3. Pooling connection manager .......................................................................... 17
2.3.4. Connection manager shutdown ...................................................................... 17
2.4. Multithreaded request execution ............................................................................... 17
2.5. Connection eviction policy ....................................................................................... 19
2.6. Connection keep alive strategy ................................................................................. 20
2.7. Connection socket factories ...................................................................................... 20
2.7.1. Secure socket layering ................................................................................... 21
2.7.2. Integration with connection manager .............................................................. 21
2.7.3. SSL/TLS customization ................................................................................. 21
2.7.4. Hostname verification ................................................................................... 21
2.8. HttpClient proxy configuration ................................................................................. 22
3. HTTP state management ..................................................................................................... 24
3.1. HTTP cookies ......................................................................................................... 24
3.2. Cookie specifications ............................................................................................... 24
3.3. Choosing cookie policy ............................................................................................ 25
3.4. Custom cookie policy .............................................................................................. 26
HttpClient Tutorial
iii
3.5. Cookie persistence ................................................................................................... 26
3.6. HTTP state management and execution context ......................................................... 26
4. HTTP authentication .......................................................................................................... 28
4.1. User credentials ....................................................................................................... 28
4.2. Authentication schemes ............................................................................................ 28
4.3. Credentials provider ................................................................................................. 29
4.4. HTTP authentication and execution context ............................................................... 30
4.5. Caching of authentication data ................................................................................. 31
4.6. Preemptive authentication ........................................................................................ 31
4.7. NTLM Authentication .............................................................................................. 32
4.7.1. NTLM connection persistence ....................................................................... 32
4.8. SPNEGO/Kerberos Authentication ............................................................................... 33
4.8.1. SPNEGO support in HttpClient ......................................................................... 33
4.8.2. GSS/Java Kerberos Setup .............................................................................. 33
4.8.3. login.conf file ............................................................................................ 34
4.8.4. krb5.conf / krb5.ini file ............................................................................. 34
4.8.5. Windows Specific configuration .................................................................... 34
5. Fluent API ......................................................................................................................... 36
5.1. Easy to use facade API ............................................................................................ 36
5.1.1. Response handling ........................................................................................ 37
6. HTTP Caching ................................................................................................................... 38
6.1. General Concepts .................................................................................................... 38
6.2. RFC-2616 Compliance ............................................................................................. 39
6.3. Example Usage ....................................................................................................... 39
6.4. Configuration .......................................................................................................... 39
6.5. Storage Backends .................................................................................................... 40
7. Advanced topics ................................................................................................................. 41
7.1. Custom client connections ........................................................................................ 41
7.2. Stateful HTTP connections ....................................................................................... 41
7.2.1. User token handler ........................................................................................ 42
7.2.2. Persistent stateful connections ........................................................................ 42
7.3. Using the FutureRequestExecutionService ................................................................. 43
7.3.1. Creating the FutureRequestExecutionService .................................................. 43
7.3.2. Scheduling requests ....................................................................................... 43
7.3.3. Canceling tasks ............................................................................................. 44
7.3.4. Callbacks ...................................................................................................... 44
7.3.5. Metrics ......................................................................................................... 44
iv
Preface
The Hyper-Text Transfer Protocol (HTTP) is perhaps the most significant protocol used on the Internet
today. Web services, network-enabled appliances and the growth of network computing continue to
expand the role of the HTTP protocol beyond user-driven web browsers, while increasing the number
of applications that require HTTP support.
Although the java.net package provides basic functionality for accessing resources via HTTP, it doesn't
provide the full flexibility or functionality needed by many applications. HttpClient seeks to fill this
void by providing an efficient, up-to-date, and feature-rich package implementing the client side of
the most recent HTTP standards and recommendations.
Designed for extension while providing robust support for the base HTTP protocol, HttpClient may
be of interest to anyone building HTTP-aware client applications such as web browsers, web service
clients, or systems that leverage or extend the HTTP protocol for distributed communication.
1. HttpClient scope
• Client-side HTTP transport library based on HttpCore [http://hc.apache.org/httpcomponents-core/
index.html]
• Based on classic (blocking) I/O
• Content agnostic
2. What HttpClient is NOT
• HttpClient is NOT a browser. It is a client side HTTP transport library. HttpClient's purpose is
to transmit and receive HTTP messages. HttpClient will not attempt to process content, execute
javascript embedded in HTML pages, try to guess content type, if not explicitly set, or reformat
request / rewrite location URIs, or other functionality unrelated to the HTTP transport.
1
Chapter 1. Fundamentals
1.1. Request execution
The most essential function of HttpClient is to execute HTTP methods. Execution of an HTTP method
involves one or several HTTP request / HTTP response exchanges, usually handled internally by
HttpClient. The user is expected to provide a request object to execute and HttpClient is expected to
transmit the request to the target server return a corresponding response object, or throw an exception
if execution was unsuccessful.
Quite naturally, the main entry point of the HttpClient API is the HttpClient interface that defines the
contract described above.
Here is an example of request execution process in its simplest form:
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpget = new HttpGet("http://localhost/");
CloseableHttpResponse response = httpclient.execute(httpget);
try {
<...>
} finally {
response.close();
}
1.1.1. HTTP request
All HTTP requests have a request line consisting a method name, a request URI and an HTTP protocol
version.
HttpClient supports out of the box all HTTP methods defined in the HTTP/1.1 specification: GET,
HEAD, POST, PUT, DELETE, TRACE and OPTIONS. There is a specific class for each method type.: HttpGet,
HttpHead, HttpPost, HttpPut, HttpDelete, HttpTrace, and HttpOptions.
The Request-URI is a Uniform Resource Identifier that identifies the resource upon which to apply
the request. HTTP request URIs consist of a protocol scheme, host name, optional port, resource path,
optional query, and optional fragment.
HttpGet httpget = new HttpGet(
"http://www.google.com/search?hl=en&q=httpclient&btnG=Google+Search&aq=f&oq=");
HttpClient provides URIBuilder utility class to simplify creation and modification of request URIs.
URI uri = new URIBuilder()
.setScheme("http")
.setHost("www.google.com")
.setPath("/search")
.setParameter("q", "httpclient")
.setParameter("btnG", "Google Search")
.setParameter("aq", "f")
.setParameter("oq", "")
.build();
HttpGet httpget = new HttpGet(uri);
