PDF Explained
John Whitington
Editor
Simon St. Laurent
Copyright © 2011 John Whitington
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles ( http://my.safaribooksonline.com ). For more information,
contact our corporate/institutional sales department: (800) 998-9938 or corporate@oreilly.com .
Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. PDF Explained , the image of a lesser anteater, and related trade dress
are trademarks of O’Reilly Media, Inc.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was
aware of a trademark claim, the designations have been printed in caps or initial caps.
While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the
information contained herein.
O'Reilly Media
Preface
The Portable Document Format (PDF) is the world’s leading page description language, and the first format equally useful for print and online use.
PDF documents are now almost ubiquitous in the printing industry, in document interchange, and in the online distribution of paginated content. They are, however, widely
viewed as opaque and delicate and are poorly understood, even by those of a technical disposition.
This is partly due to a perplexing lack of documentation; the file format reference is freely available, but is of a size and complexity which requires a time investment unlikely
to be plausible for the majority of those working with PDF.
This book aims to be an approachable introduction. It is suitable both for the technically-minded, and for those who just want to understand a little of the PDF format to
give context to their work with tools which produce or process PDF documents.
Who Should Read This Book
We’ve tried to write a book which serves as a general introduction, with some optional technical interludes, giving you the chance to type in example PDF files and see how
they display.
This book is suitable for:
Adobe Acrobat users who want to understand the reasons behind the facilities it provides, rather than just how to use them. For example:
encryption options, trim and crop boxes, and page labels.
Power users who want to use command-line software to process PDF documents in batches by merging, splitting, and optimizing them.
Programmers writing code to read, edit, or create PDF files.
Industry professionals in search, electronic publishing, and printing who want to understand how to use PDF’s metadata and workflow
features to build coherent systems.
Organization of Contents
Chapter 1, Introduction
In this chapter, we give a history of the PDF format and put it into context. We look at the advantages PDF has over similar technologies,
introduce specialized kinds of PDF files such as PDF/X and PDF/A, and take a brief tour of the elements which comprise a typical PDF
document. We conclude by looking at how PDF is used in industry.
Chapter 2, Building a Simple PDF
We begin in earnest, building a simple PDF file from scratch in a text editor. We show how to process this into a fully valid PDF and open it in a
PDF viewer. We explain each component of the file, taking our first look at various parts of the PDF syntax.
Chapter 3, File Structure
In this chapter, we describe the layout and content of a PDF file, and the syntax of the objects from which it is built. We describe how a PDF
document is read from a flat file into a structured format and, conversely, written from that structured format to a flat file.
Chapter 4, Document Structure
In this chapter, we leave behind the bits and bytes of the PDF file, and consider the logical structure of its objects, describing how pages and their
resources are arranged into a document.
Chapter 5, Graphics
We describe how to create vector graphics and raster images in PDF, and how to deal with transparency, color spaces, and patterns. We
illustrate with examples, showing the code and the result in a PDF viewer.
Chapter 6, Text and Fonts
In this chapter, we look at the PDF operators for building and showing text strings using different fonts and sizes, and how to build lines and
paragraphs. We describe the different types of fonts and encodings in PDF documents, and how they are defined and used. We look at the
process of text extraction from a PDF document.
Chapter 7, Document Metadata and Navigation
Here, we discuss topics not directly related to the visual appearance of the document, but to ancillary data: bookmarks, metadata, hyperlinks,
annotations, and file attachments. For each, we describe how they are defined in PDF and give examples.
Chapter 8, Encrypted Documents
We look at how encryption and document permissions work in PDF, and see how to inspect encryption information in Adobe Reader. We
describe how programs which process PDF files read, write, and edit encrypted documents.
Chapter 9, Working with Pdftk
In this chapter, we show how to use the popular pdftk program for the command-line processing of PDF files, looking at common usage
scenarios. We describe what a program such as pdftk has to do internally to achieve certain tasks (for example, merging or splitting documents).
Chapter 10, PDF Software and Documentation
Here, we describe both Adobe and open-source software for viewing, converting, editing, and programming with PDF files. We give sources of
further documentation and other resources such as support and discussion forums.
Acknowledgments
I should like to thank my editor, Simon St.Laurent, who was enthusiastic about this project from the beginning.
Conventions Used in This Book
The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width
Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types,
environment variables, statements, and keywords.
Constant width bold
Shows commands or other text that should be typed literally by the user.
Constant width italic
Shows text that should be replaced with user-supplied values or by values determined by context.
CAUTION
This icon indicates a warning or caution.
Obtaining Code Examples
All the PDF code examples in this book are available for download in a zip archive from the O’Reilly website . The text of the book contains enough information to
reconstruct these examples (with the exception of encrypted documents, which are not suitable for typing in manually).
The examples include the PDF source for the figures in this book.
Using Code Examples
This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for
permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require
permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example
code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “PDF Explained by John Whitington
(O’Reilly). Copyright 2012 John Whitington, 978-1-449-31002-8.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com .
Safari® Books Online
NOTE
Safari Books Online is an on-demand digital library that lets you easily search over 7,500 technology and creative reference books and videos to find the answers you need
quickly.
With a subscription, you can read any page and watch any video from our library online. Read books on your cell phone and mobile devices. Access new titles before they
are available for print, and get exclusive access to manuscripts in development and post feedback for the authors. Copy and paste code samples, organize your favorites,
download chapters, bookmark key sections, create notes, print out pages, and benefit from tons of other time-saving features.
O’Reilly Media has uploaded this book to the Safari Books Online service. To have full digital access to this book and others on similar topics from O’Reilly and other
publishers, sign up for free at http://my.safaribooksonline.com .
How to Contact Us
Please address comments and questions concerning this book to the publisher:
O’Reilly Media, Inc.
1005 Gravenstein Highway North
Sebastopol, CA 95472
800-998-9938 (in the United States or Canada)
707-829-0515 (international or local)
707-829-0104 (fax)
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at:
http://oreilly.com/catalog/0636920021483
To comment or ask technical questions about this book, send email to:
bookquestions@oreilly.com
For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com .
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://www.youtube.com/oreillymedia
Chapter 1. Introduction
The Portable Document Format (PDF) is the world’s leading language for describing the printed page, and the first one equally suitable for paper and online use. In this
chapter, we take a tour of its uses, features, and history. We look at some useful free software and resources, some of which we’ll use later in this book.
A Little History
Today we take the high fidelity exchange of documents for granted, knowing that a document sent here will appear the same there and vice versa, and that it may be
displayed equally on screen and on paper. This was not always so.
Page Description Languages
We could pass documents between users, and from user to printer, as a series of bitmap pictures (e.g., TIFF or PNG), one for each page. However, this doesn’t allow for
any structure to be retained, precludes scaling to different paper sizes or resolutions without loss of quality, involves huge file sizes, and so on.
A page description language like PDF is way of describing the contents (text and graphics) of a printed or onscreen page using highly structured data, often with extra
metadata describing various aspects of the document (such as printing information or textual annotations or how it is to be viewed or printed). This way, decisions about
how the document is rasterized (converted to pixels by a printer or on screen) can be left until the end of the production process. A PDF file can contain text and associated
font definitions, vector and bitmap graphics, navigation (such as hyperlinks and bookmarks), and interactive forms.
PDF is used wherever the exact presentation of the content is important (for example for a print advertisement or book). It isn’t normally suitable when the content is to be
layed out or reflowed at the last moment, such as in a variable width web page—languages like HTML and CSS which separate content from presentation are more
suitable in those circumstances.
Other page description languages
Many page description languages were created when the printing of lines of text in fixed fonts began to be replaced by digital graphics printing. The printer would then
process the language to generate a bitmap at the appropriate resolution. For example, PostScript (Adobe), PCL (Hewlett Packard), and KPDL (Kyocera). Simpler
languages were used for vector plotters (for example, HPGL from Hewlett Packard).
These languages varied in complexity and functionality. PostScript files, for example, are full programs—the result of executing the program is the document’s visual
representation. These languages often contain extra instructions to control aspects of the document other than the page content, for example which tray paper is drawn from
or whether the output is to be duplexed.
Development of PDF
PDF began as an internal project at Adobe to create a platform-neutral method for document interchange. PostScript was already popular in the print community, but
wasn’t practical for on screen use with the computers of the day—especially for random access (to render page 50 of a PostScript document, one must process pages 1–
49 first). The idea was to use a subset of the PostScript graphics language together with ancillary data to create a structured language for standalone documents to be
viewed on (or printed from) any computer.
PDF 1.0 was announced in 1993, with Acrobat Distiller (for creating and editing PDF files) and Acrobat Reader (for viewing only), both as paid-for programs. The US Tax
Authorities started to ship tax forms as PDFs, purchasing a license to allow their users to download Acrobat Reader for free. Later on, Acrobat Reader was made available
to everybody at no cost, leading to the widespread use of PDF for the exchange of documents online.
Over the next 10 years, after a slow start as prepress features were added, PDF overtook PostScript as the language of choice in the printing industry. Today, it is the only
general page description language of note.
Some Advantages of PDF
When a number of formats compete to be the industry standard, the best contender is not always the victor—luck can intervene. In this case, though, PDF had a number of
singular advantages. We look at some of them here.
Random access and linearization
Unlike PostScript, any object (page, graphic etc.) in a PDF document can be accessed at will, in constant time. This means it’s no harder to read page 150 than page 1.
Linearization is the process of arranging the objects in the file such that all those needed for a given page are located in adjacent positions. This explains why you can
quickly jump to any page in a PDF being viewed in Acrobat Reader in a web browser window—the viewer doesn’t need to load the whole file to begin with, it fetches from
the server just the sections needed to display each new page.
Stream creation and incremental update
Stream creation is the ability inherent in the PDF format to allow files to be created in order, from beginning to end, even if the eventual file is larger than the memory
available.
Incremental update means that, when editing a file, it’s possible to write the changes to the end of the file without modifying any existing part—this makes saving changed
versions very fast, and can be used to provide an undo mechanism (since the previous version is still intact).
Embedded fonts
Fonts used in a PDF are embedded along with the document. This means that it should always be rendered correctly, regardless of which fonts are installed on a given
computer. The program creating the PDF document will remove unnecessary data from the font (such as metrics and unused characters), so the file does not become unduly
large. PDF supports all common font formats, such as TrueType and Type 1.
Searchable text
Most PDF files maintain the information to map the character shapes making up the text to Unicode character codes. This means that you can copy and paste text from a
document, or search the text easily. More recent developments in PDF allow the logical order of the text in the document to be stored separately from the layout of the text
on the page, preserving yet more structured information.
ISO Standardization
PDF was released as an open standard by the International Organization for Standardization (ISO) in 2008. The ISO-32000-1:2008 document is largely the same as the
PDF file format document previously released by Adobe.
This independence lends legitimacy and oversight to the PDF standard, which should encourage its further adoption. However, with no real tools for detecting whether a file
meets the standard (Adobe Reader will happily load malformed files, so many tools create them), genuine rigor is some time away.
THE PDF FILE FORMAT DOCUMENT
The PDF File Format Version 1.7 is documented in ISO 32000-1:2008, which is available on CD or as a PDF for 380 Swiss Francs at the International Organization for Standardization
.
The almost identical Adobe Document “Adobe PDF Reference, Sixth Edition, version 1.7” is available in PDF format at the Adobe PDF Technology Center . Adobe extensions,
which do not yet form part of the ISO standard, are published at the same location.
Unfortunately, the PDF File Format is no longer available in print.
Specialized Kinds of PDF
There are several specialized variations on the PDF format—both standardized, and in development. These are subsets of the PDF format. Each file is a valid PDF
document, but with restrictions on the facilities used or the content itself. Two of these, PDF/A and PDF/X, are now ISO standards.
PDF/A
The PDF/A Standard (ISO 19005-1:2005) defines a set of rules for documents intended for long-term archiving in libraries, national archives and bureaucracies. It also
requires a “conforming reader” to act in certain ways, using the embedded fonts, using color management, and so forth. Briefly, the restrictions on PDF/A are:
