Chapter 8. PNG Basics

Chunks were designed to be easily tested and manipulated by computer programs, easily detected by human eyes, and reasonably self-contained. Every chunk has the same structure: a 4-byte length (in ``big-endian'' format, as with all integer values in PNG streams), a 4-byte chunk type, between 0 and 2,147,483,647 bytes of chunk data, and a 4-byte cyclic redundancy check value (CRC). This is diagrammed in Figure 8-1.

Figure 8-1: PNG chunk structure.

The data field is straightforward; that's where the interesting bits (if any) go; specific content will be discussed later, as each chunk is described. The length field refers to the length of the data field alone, not the chunk type or CRC. The CRC, on the other hand, covers both the chunk-type field and the chunk data and is always present, even when there is no chunk data. Note that the combination of length fields and CRC values is already sufficient to check the basic integrity of a PNG file! The only missing information--not including the contents of the first 8 bytes in the file--is the exact algorithm (or ``polynomial'') used for the CRC. That turns out to be identical to the CRC used by gzip and many popular archiving programs; it is described in detail in Section 3.4 of the PNG Specification, Version 1.1, available from http://www.libpng.org/pub/png/pngdocs.html.

The chunk type is possibly the most unusual feature. It is specified as a sequence of binary values, which just happen to correspond to the upper- and lowercase ASCII letters used on virtually every computer in the Western, non-mainframe world. Since it is far more convenient (and readable) to speak in terms of text characters than numerical sequences, the remainder of this book will adopt the convention of referring to chunks by their ASCII names. Programmers of EBCDIC-based computers should take note of this and remember to use only the numerical values corresponding to the ASCII characters.

Chunk types (or names) are usually mnemonic, as in the case of the IHDR or image header chunk. In addition, however, each character in the name encodes a single bit of information that shows up in the capitalization of the character.[56] Thus IHDR and iHDR are two completely different chunk types, and a decoder that encounters an unrecognized chunk can nevertheless infer useful things about it. From left to right, the four extra bits are interpreted as follows:

[56] The ASCII character set was conveniently designed so that the case of a letter is always determined by bit 5. To put it another way, adding 32 to an uppercase character code gives you the code for its lowercase version.

• The first character's case bit indicates whether the chunk is critical (uppercase) or ancillary; a decoder that doesn't recognize the chunk type can ignore it if it is ancillary, but it must warn the user that it cannot correctly display the image if it encounters an unknown critical chunk. The tEXt chunk, covered in Chapter 11, "PNG Options and Extensions", is an example of an ancillary chunk.

• The second character indicates whether the chunk is public (uppercase) or private. Public chunks are those defined in the specification or registered as official, special-purpose types. But a company may wish to encode its own, application-specific information in a PNG file, and private chunks are one way to do that.

• The case bit of the third character is reserved for use by future versions of the PNG specification. It must be uppercase for PNG 1.0 and 1.1 files, but a decoder encountering an unknown chunk with a lowercase third character should deal with it as with any other unknown chunk.

• The last character's case bit is intended for image editors rather than simple viewers or other decoders. It indicates whether an editing program encountering an unknown ancillary chunk[57] can safely copy it into the new file (lowercase) or not (uppercase). If an unknown chunk is marked unsafe to copy, then it depends on the image data in some way. It must be omitted from the new image if any critical chunks have been modified in any way, including the addition of new ones or the reordering or deletion of existing ones. Note that if the program recognizes the chunk, it may choose to modify it appropriately and then copy it to the new file. Also note that unsafe-to-copy chunks may be copied to the new file if only ancillary chunks have been modified--again, including addition, deletion, and reordering--which implies that ancillary chunks cannot depend on other ancillary chunks.

  [57] Since any decoder encountering an unknown critical chunk has no idea how the chunk modifies the image--only that it does so in a critical way--an editor cannot safely copy or omit the chunk in the new image.

So chunk names encode additional information that is primarily useful if the chunk is not recognized. The remainder of this book will be concerned with known chunks, but before we turn to those, there is one more component of PNG files that has to do with the unknown: the PNG file signature. As noted earlier, the first 8 bytes of the file are not, strictly speaking, a chunk.[58] They are a critical component of a PNG file, however, since they allow it to be identified as such regardless of filename. But the PNG signature bytes are more than a simple identifier code: they were cleverly designed to allow the most common types of file-transfer corruption to be detected. Web protocols these days typically ensure the correct transfer of binary files such as PNG images, but older transfer programs like the venerable command-line FTP (File Transfer Protocol) often default to text-mode or ``ASCII'' transfers. The unsuspecting user who transfers a PNG image or other binary file as text is practically guaranteed of destroying it. The same is true of the user who extracts a PNG file from a compressed archive in text mode or who emails it without some form of ``ASCII armor'' (such as MIME Base64 encoding or Unix uuencoding).

[58] They can be thought of as such, however, since their length is known (8 bytes), their position and purpose are known (beginning of the file; signature), and their CRC is implied (the 8 bytes are constant, so effectively they are their own CRC).

The 8-byte PNG file signature can detect this sort of problem because it simulates a text file in some respects. The 8 bytes are given in Table 8-1.

Note that the 9th, 10th, and 11th bytes are guaranteed to be 0 (that is, the ASCII NUL character) by the fact that the first chunk is required to be IHDR, whose first 4 bytes are its length--a value that is currently 13 and, according to the spec, will never change. (Instead, ``new chunk types will be added to carry new information.'') The fact that the 0 bytes in the length come first is another benefit of the big-endian integer format, which stores the high-order bytes first. Since NUL bytes are also often stripped out by text-mode transfer protocols, the detection of damaged PNG files is even more robust than the signature alone would suggest.

Before we start putting chunks together, however, a brief interlude on the representation and terminology of color is useful. Color fundamentally refers to a property of light--namely, its wavelength. Each color in the rainbow, from red to purple, is a relatively pure strain of wavelengths of light, and none of these colors can be generated by adding together any of the others.[59] Furthermore, despite what our eyeballs would have us think, the spectrum does not end at deep purple; beyond that are the ultraviolet, X-ray, and gamma-ray domains. Nor does it end at dull red--smoke on the water glows in the infrared, if only we could see it, and still further down the spectrum are radio waves.[60] Each of these wavelength regions, from radio on up to gamma, is a color.

[59] Mathematically, this is known as orthogonality and is the basis for Fourier decomposition, among other things.

[60] It is probably not coincidence that the range of light visible to our water-filled orbs just happens to be the precise range of wavelengths that is not strongly absorbed by water.

So when someone refers to an RGB image--that is, containing only red, green, and blue values--as ``truecolor,'' what twisted logic lies behind such a claim? The answer lies not in physics but in physiology. Human eyes contain only three classes of color sensors, which trigger color sensations in the brain in ways that are not yet fully understood. One might guess that these sensors (the cones) are tuned to red, green, and blue light, but that turns out not to be the case, at least not directly. Instead, signals from the three types of cones are added and subtracted in various ways, apparently in more than one stage. The details are not especially important; what matters is that the end result is a set of only three signals going into the brain, corresponding to luminosity (or brightness), a red-versus-green intensity level, and a yellow-versus-blue level. In addition, the cones are not narrow-band sensors, but instead each responds to a broad range of wavelengths. The upshot is that the human visual system is relatively poor at analyzing colors, so feeding it different combinations of red, green, and blue light suffices to fool it into thinking it is seeing an entire spectrum. Keep in mind, however, that while true yellow and a combination of red and green may look identical to us, to spectrometers (or nonhuman eyes) they are quite different.

In fact, even printers ``see'' color differently. Since they employ pigments, which absorb light rather than emit it, the RGB color space that works so well for computer monitors is inappropriate. Instead, use a ``dual'' color space based on cyan, magenta, and yellow, or CMYK for short.[61] And in video processing, television, and the JPEG image format, yet another set of color spaces is popular: YUV, YIQ, and YC_bC_r, all of which represent light as an intensity value (Y) and a pair of orthogonal color vectors (U and V, or I and Q, or C_b and C_r). All of these color spaces are beyond the scope of this book, but note that every single one of them has its basis in human physiology. Indeed, if YUV and its brethren sound quite a lot like the set of three signals going into the brain that I just discussed, rest assured that it's not coincidence. Not a single color space in common use today truly represents the full continuum of physical color.

[61] The K is for black. Since black is the preferred color for a huge class of printed material, including text, it is more efficient and considerably cheaper to use a single pigment for it than always to be mixing the other three. Some printing systems actually use five, six, or even seven distinct pigments.

Finally, note that image files may represent the appearance of a scene not only as a self-contained item, but also in reference to a background or to other images or text. In particular, transparency information is often desirable. The simplest approach to transparency in computer graphics is to mark a particular color as transparent, but more complex applications will generally require a completely separate channel of information. This is known as an alpha channel (or sometimes an alpha mask) and enables the use of partial transparency, such as is often used in television overlays. In the text that follows, I will refer to an RGB image with an alpha channel as an RGBA image. PNG adheres to the usual convention that alpha represents opacity; that is, an alpha value of 0 is fully transparent, and the maximum value for the pixel depth is completely opaque. PNG also uses only unassociated alpha, wherein the actual gray or color values are stored unchanged and are only affected by the alpha channel at display time. The alternative is associated or premultiplied alpha, in which the pixel values are effectively precomposited against a black background; although this allows slightly faster software compositing, it amounts to a lossy transformation of the image data and was therefore rejected in the design of PNG.

We've looked at the fine details of a PNG file--the subatomic structure, if you will--so let us turn now to a few of the basic atoms (chunks) that will allow us to create a complete ``molecule,'' or valid Portable Network Graphics file. The simplest possible PNG file, diagrammed in Figure 8-2, is composed of the PNG signature and only three chunk types: the image header chunk, IHDR; the image data chunk, IDAT; and the end-of-image chunk, IEND. IHDR must be the first chunk in a PNG image, and it includes all of the details about the type of the image: its height and width, pixel depth, compression and filtering methods, interlacing method, whether it has an alpha (transparency) channel, and whether it's a truecolor, grayscale, or colormapped (palette) image. Not all combinations of image types are valid, however, and much of the remainder of this chapter will be devoted to a discussion of what is allowed.

Figure 8-2: Layout of the simplest PNG.

IEND is the simplest chunk of all; it contains no data, just indicates that there are no more chunks in the image. IEND is primarily useful when the PNG image is being transferred over the network as a stream, especially when it is part of a larger MNG stream (Chapter 12, "Multiple-Image Network Graphics"). And it serves as one more check that the PNG file is complete and internally self-consistent.

These three chunk types are sufficient to build truecolor and grayscale PNG files, with or without an alpha channel, but palette-based images require one more: PLTE, the palette chunk. PLTE simply contains a sequence of red, green, and blue values, where a value of 0 is black and 255 is full intensity; anywhere from 1 to 256 RGB triplets are allowed, depending on the pixel depth of the image. (That is, for a 4-bit image, no more than 16 palette entries are allowed.) The PLTE chunk must come before the first IDAT chunk; the structure of a colormapped PNG is shown in Figure 8-3.

Figure 8-3: Layout of the second-simplest PNG.

I noted earlier that not all possible combinations of PNG image types and features are allowed by the specification. Let's take a closer look at the basic types and their features.

We'll wrap up our look at the basic elements of Portable Network Graphics images with a quick consideration of progressive rendering and interlacing. Most computer users these days are familiar with the World Wide Web and the method by which modern browsers present pages. As a rule, the textual part of a web page is displayed first, since it is transmitted as part of the page; then images are displayed, with each one rendered as it comes across the network. Ordinary images are simply painted from the top down, a few lines at a time; this is the most basic form of progressive display.

PNG's interlacing method, on the other hand, is a two-dimensional scheme with seven passes, known as the Adam7 method (after its inventor, Adam Costello). If one imagines the image being broken up into 8 × 8-pixel tiles, then the first pass consists of the upper left pixel in each tile--that is, every eighth pixel, both vertically and horizontally. The second pass also consists of every eighth pixel, but offset four pixels to the right.

Figure 8-4: Schematic of an 8 × 8 tile (a) after the third pass and (b) after the fifth pass.

The third pass consists of two pixels per tile, offset by four rows from the first two pixels (see Figure 8-4a). The fourth pass contains four pixels in each tile, offset two columns to the right of each of the first four pixels, and the fifth pass contains eight pixels, offset two rows downward (see Figure 8-4b). The sixth pass fills in the remaining pixels on the odd rows (if the image is numbered starting with row one), and the seventh pass contains all of the pixels for the even rows. Note that, although I've described the method in terms of 8 × 8 tiles, pixels for any given pass are stored as complete rows, not as tiled groups. For example, the fifth pass consists of every other pixel in the entire third row of the image, followed by every other pixel in the seventh row, and so on.

The primary benefit of PNG's two-dimensional interlacing over GIF's one-dimensional scheme is that one can view a crude approximation of the entire image roughly eight times as fast.[66] That is, PNG's first pass consists of one sixty-fourth of the image pixels, whereas GIF's first pass consists of one-eighth of the data. Suppose one were to save a palette image as both an interlaced GIF and an interlaced PNG. Assuming the compression ratio and download speeds were identical for the two files, the PNG image would have completed its fourth pass as the GIF image completed its first. But most browsers that support progressive display do so by replicating pixels to fill in the areas that haven't arrived yet. For the PNG image, that means each pixel at this stage represents a 2 × 4 block, whereas each GIF pixel represents a 1 × 8 strip. In other words, GIF pixels have an 8-to-1 aspect ratio, whereas PNG pixels are 2-to-1. At the end of the next pass for each format (GIF's second pass, PNG's fifth; one-quarter of the image in both cases), the PNG pixels are square 2 × 2 blocks, while the GIF pixels are still stretched, now as 1 × 4 strips. In practical terms, features in the PNG image--particularly embedded text--are much more recognizable than in the GIF image. In fact, readability testing suggests that text of any given size is legible roughly twice as fast with PNG's interlacing method.

[66] As I (foot)noted in Chapter 1, "An Introduction to PNG", this implicitly assumes that one-eighth of the compressed data corresponds to one-eighth of the uncompressed (image) data, which is not quite accurate. The difference is likely to be small in most cases, however. I'll discuss this further in Chapter 9, "Compression and Filtering".

JPEG also supports a form of progressive display, but it is not interlacing in the usual sense of reordering the pixels spatially. Rather, it involves reordering the frequency components that make up a JPEG image, first displaying the low-frequency ones and working up to the highest frequency band; this is known as spectral selection. In addition, progressive JPEG can transmit the most significant bits of each frequency component earlier than the less significant ones, a feature known as successive approximation that is very nearly the same as turning up the JPEG quality setting with each scan. The two approaches can be used separately, but in practice they are almost always used in combination. Because JPEG operates on 8 × 8 blocks of pixels, progressive JPEG bears a strong resemblance to interlaced PNG during the early stages of display, though it tends to have a softer, fuzzier look due to the initial lack of high-frequency components (which is often deliberately enhanced by smoothing in the decoder). This is visible in Figures C-4a and C-4b in the color insert, which represent the second pass of a progressive JPEG image (26% of the compressed data), both unsmoothed and smoothed. Note in particular the blockiness in the shadowed interior of the box and the ``colored outside the lines'' appearance around the child's arms and hands; the first effect is completely eliminated in the smoothed version, and the second is greatly reduced. JPEG's first pass is actually more accurate than PNG's, however, since the low-frequency band for each 8 × 8 pixel block represents an average for all 64 pixels, whereas each 8 × 8 block in PNG's first pass is represented by a single pixel, usually in the upper left corner of the displayed block. By its fifth pass, which represents only 40% of the compressed data, the progressive JPEG version of this image (Figure C-4c) is noticeably sharper and more accurate than all but the final pass of the PNG version. Keep in mind also that, since the PNG is lossless and therefore 11 times as large as the JPEG, 40% of the compressed JPEG data is equivalent to only 3.5% of the PNG data, which corresponds to the beginning of PNG's third pass. This only emphasizes the point made previously: for non-transparent, photographic images on the Web, use JPEG.

Note that smoothing could be applied to the early passes of interlaced PNGs and GIFs, as well; tests suggest that this looks better for photographic images but maybe not as good for simple graphics. (On the other hand, recall that smoothing did seem to enhance the readability of early interlace passes in Figure 1-4.) As for representing blocks by the pixel in the upper left corner, it would be possible to replicate each pixel so that the original would lie roughly at the center of its clones, as long as some care were taken near the edges of the image. This would prevent the apparent shift in some features as later passes are displayed. But neither smoothing nor centered pixel replication is currently supported by the PNG reference library, libpng, as of version 1.0.3.

It is worth noting that TIFF can also support a kind of interlacing, although like everything about TIFF, it is much more arbitrary than either GIF's or PNG's method. Baseline TIFF includes the concept of strips, each of which may include one or more rows of image data though the number of rows per strip is constant. A list of offsets to each strip is embedded within the image, so in principle one could make each strip a row and do GIF-style line interlacing with any ordering one chose. But since TIFF's structure is fundamentally random access in nature, this approach would only work if one imposed certain restrictions on the locations of its internal directory, list of strip offsets, and actual strip data--that is, one would need to define a particular subformat of TIFF.

In addition, libtiff supports a TIFF extension called tiles, in which the image data is organized into rectangular regions instead of strips. Since the tile size can be arbitrary, one could define it to be 1 × 1 and then duplicate PNG's Adam7 interlacing scheme manually--or even extend it to 9, 11, or more passes. However, since every tile must have a corresponding offset in the TIFF image directory, doing something like this would at least double or triple the image size. Also, TIFF's compression methods apply only to individual strips or tiles, so there would be no real possibility of compression aside from reusing tiles in more than one location (that is, by having multiple tile offsets point at the same data). And, as with the strip approach, this would require restrictions on the internal layout of the file. Nevertheless, the capability does exist, at least theoretically.