14.1. Preliminaries
14.2. readpng2_init()
14.3. readpng2_decode_data()
14.4. readpng2_info_callback()
14.5. readpng2_row_callback()
14.6. Compositing and Displaying the Image
14.7. readpng2_end_callback()
14.8. readpng2_cleanup()
14.9. Getting the Source Code
As I noted in Chapter 13, "Reading PNG Images", the basic style of PNG viewer that reads each image from a file in a single gulp is appropriate to some applications, but not all. In particular, web browsers and the like tend to read images from a network, and they often download more than one image at the same time. It is usually desirable for them to display whatever is available at regular intervals so the user can get some idea of the contents of the page as quickly as possible. The alternative--waiting the minute or more that some web pages take to download--went out of style almost as soon as Netscape Navigator became available late in 1994.
This style of display is known as progressive, and as one might imagine, it places strong constraints on the structure of the program. In fact, in many ways a progressive reader is completely inverted from the basic design showed in the last chapter: instead of giving the image library control for the duration of the decoding process, in a progressive reader, the main program retains control, effectively throttling the library by restricting the amount of encoded image data it makes available per call. This will become much clearer with a concrete example, so let us jump right in.
As in the first demo program, I have divided this program into a PNG-specific file (readpng2.c this time) and a platform-dependent file whose filename, logically enough, depends on the platform. I refer to these two parts as the ``back end'' and ``front end,'' respectively; I'll once again concentrate on the libpng-specific back end. This time through, I'll skim over many of the most basic libpng concepts, however. Indeed, most of the individual blocks of PNG code are virtually identical to their counterparts in the basic reader. What has changed is their overall order in the grand scheme of things.
I'll first note some of the things that haven't changed. As before, our overall design choices include a desire to deal only with 24-bit RGB or 32-bit RGBA data; I will instruct libpng to transform the PNG image data exactly as before. I will also make a game attempt at doing proper gamma correction; the main program not only calculates reasonable defaults based on the platform but also gives the user a chance to specify things precisely. The code for this is unchanged and will not be presented again. Likewise, I will continue to use the abbreviated typedefs uch, ush, and ulg in place of the more unwieldy unsigned char, unsigned short, and unsigned long, respectively.
Within the PNG-specific module, I will once again begin with the inclusion of the libpng header file, png.h, which in turn includes the zlib.h header file. (The latest releases at the time of this writing are libpng 1.0.3 and zlib 1.1.3, which are the versions used by the demo programs.) The four-line readpng2_version_info() routine is no different from that in the first demo program.
Because this style of PNG reader is intended for the kind of application that decodes multiple images simultaneously (read: browsers), one difference from the first program is the lack of global or static variables in the PNG code. Instead, all image-specific variables are embedded in a structure, which could be allocated repeatedly for as many images as desired. Although some globals are still used in the front-end code, they are all either truly global (that is, they could be used in a multi-image program without problems), or else they could be moved into the per-image struct, too.
The serious PNG code once again begins with the main program opening the PNG file, and I emphasize that it is opened in binary mode--hence the ``b'' flag in the second argument to fopen() ("rb"). A real browser would open an HTTP connection to a remote server and request the image instead of opening it as a local file. Rather than immediately jumping into our PNG initialization routine, readpng2_init(), as was the case in the first demo, this version first reads a block of data from the file and checks the first eight bytes for the PNG signature:
if (!(infile = fopen(filename, "rb")))
/* report an error and exit */
} else {
incount = fread(inbuf, 1, INBUFSIZE, infile);
if (incount < 8 || !readpng2_check_sig(inbuf, 8)) {
/* report an error and exit */
} else {
rc = readpng2_init(&rpng2_info);
[etc.]
}
}
The readpng2_check_sig() function is nothing more than a wrapper to call png_check_sig(). It would also have been possible to call the libpng routine directly; libpng is unique in that it does not require any special setup or datatypes, and it returns an integer value, which is the default for C functions. But that would violate our separation of libpng and non-libpng code, if only in a tiny way, and it would prevent the compiler from checking the argument and return types against a prototype, in case the libpng function should ever change.
Sharp-eyed readers will have noticed that I call readpng2_init() with a different argument than last time:
int readpng2_init(mainprog_info *mainprog_ptr)
The difference from the first version is that the function now has only one argument, a pointer to an object type called mainprog_info. This is just the per-image struct mentioned earlier. It is defined as follows:
typedef struct _mainprog_info {
double display_exponent;
ulg width;
ulg height;
void *png_ptr;
void *info_ptr;
void (*mainprog_init)(void);
void (*mainprog_display_row)(ulg row_num);
void (*mainprog_finish_display)(void);
uch *image_data;
uch **row_pointers;
jmp_buf jmpbuf;
int passes;
int rowbytes;
int channels;
int need_bgcolor;
int done;
uch bg_red;
uch bg_green;
uch bg_blue;
} mainprog_info;
I'll explain each member as we need it, but it is clear that many of the variables that were formerly global or passed as arguments to functions now reside in this struct. Note that similar variable types have been grouped, with the smallest ones at the end, so that the larger types will be aligned on even memory boundaries by default, minimizing the amount of padding the compiler has to add to the structure.
readpng2_init() begins by calling libpng to allocate the two PNG structs:
png_structp png_ptr;
png_infop info_ptr;
png_ptr = png_create_read_struct(PNG_LIBPNG_VER_STRING,
mainprog_ptr, readpng2_error_handler, NULL);
if (!png_ptr)
return 4; /* out of memory */
info_ptr = png_create_info_struct(png_ptr);
if (!info_ptr) {
png_destroy_read_struct(&png_ptr, NULL, NULL);
return 4; /* out of memory */
}
I have used a pair of local variables here, png_ptr and info_ptr, for convenience. The mainprog_info struct also includes these variables, but because it's used in the main program, which has no knowledge of libpng datatypes, the struct versions of the two variables are simply declared as pointers to void. To use them directly in readpng2_init(), we would need to typecast them repeatedly, which is annoying and makes the program harder to read and somewhat slower. So I spent a few bytes on the temporary (local) variables to make life easier.
The next step is to set up one of those setjmp() calls. This differs from the previous version only in that now we're using our own struct's jmpbuf member instead of the one in the main PNG struct:
if (setjmp(mainprog_ptr->jmpbuf)) {
png_destroy_read_struct(&png_ptr, &info_ptr, NULL);
return 2;
}
The second big difference from the basic PNG reader is what comes next:
png_set_progressive_read_fn(png_ptr, mainprog_ptr,
readpng2_info_callback, readpng2_row_callback,
readpng2_end_callback);
Here we get a glimpse of the inversion of the program logic. The original approach was to call libpng and wait for it to return the requested image data, whether header information or actual pixels. That doesn't really work in a progressive program--if you give the library a hunk of data and wait for it to return, you may end up with nothing if the hunk was too small, or you may get the entire image back. More commonly, it is impossible to return a completely sensible result, due to the way compression works. The end of a buffer of compressed data may correspond to the first two bits of the red sample of a single pixel, for example, or it may cut off a piece of a compressed token that is therefore meaningless. Either way, what we really want is a way for the decoding library to provide us with data in a more controlled manner. Callbacks are the answer.
A callback is just what it sounds like: if our main routine calls the library with a chunk of data, the library will call us back when a certain amount has been processed--say, one row of image pixels. The function it calls (back in the main program, presumably) can then handle the decoded data, return, possibly get called again, and so forth. Eventually the library will exhaust the data it was given and return to the original routine. That routine can then read some more data from the network and pass it back to libpng, go and decode part of another image, respond to user input, or do anything else that needs doing.
The progressive handler in libpng is set up to work with three callback functions: one to be called when all of the header information has been read (i.e., everything prior to the first IDAT), one for when each row of the image is decoded (which includes ``short'' rows if the image is interlaced), and one for when the complete PNG stream has been read. These are the last three arguments to png_set_progressive_read_fn(), and our versions are called readpng2_info_callback(), readpng2_row_callback(), and readpng2_end_callback(), respectively. They are all required to have the same two arguments: png_ptr and info_ptr, the pointers to the two standard PNG structs. But in order for the application to associate image-specific data with each callback, libpng makes available a user-specified pointer, embedded somewhere within the PNG structs; it can be retrieved via a libpng function. In our case, we provide a pointer to the mainprog_info struct for the image. This is the second argument to png_set_progressive_read_fn(). (The first argument is just the png_ptr itself.)
As it turns out, the call to png_set_progressive_read_fn() is essentially the whole point of our readpng2 initialization routine. The only remaining detail is to save the two temporary pointers into the mainprog_info struct before returning to the main program:
mainprog_ptr->png_ptr = png_ptr;
mainprog_ptr->info_ptr = info_ptr;
return 0;
These pointers will be used in the readpng2 decoding routine that calls libpng, which in turn sends the pointers back to the callback functions.
Back in the main program, after dealing with various windowing-system chores, the code sets a few variables in the mainprog_info struct. The following excerpt is from the X version of the code, but the Windows code is the same, aside from prefixing function names with rpng2_win_ instead of rpng2_x_:
if (user_did_not_specify_a_background_color_or_pattern)
rpng2_info.need_bgcolor = TRUE;
rpng2_info.mainprog_init = rpng2_x_init;
rpng2_info.mainprog_display_row = rpng2_x_display_row;
rpng2_info.mainprog_finish_display = rpng2_x_finish_display;
Unlike the basic viewer, where the main program called a special function to check for and retrieve the image's background color, the progressive viewer simply sets the need_bgcolor flag in the struct. It also sets three function pointers corresponding to the three readpng2 callbacks. The reason for this apparent duplication will become clear when we look at the callbacks in detail.
Having prepared everything for decoding, the main program begins the data loop that is at its core, reading file data into a buffer and passing it to the PNG-decoding function:
for (;;) {
if (readpng2_decode_data(&rpng2_info, inbuf, incount))
++error;
if (error || feof(infile) || rpng2_info.done)
break;
if (timing)
sleep(1);
incount = fread(inbuf, 1, INBUFSIZE, infile);
}
Note the call to readpng2_decode_data() at the beginning of the loop, before fread(); it handles the initial chunk of data we read prior to calling readpng2_init().
The only remarkable feature of the loop itself is the conditional call to the sleep() function. Because this is a demo program, and because it is intended to be a rough simulation of how a web browser functions, I chose to give the user the option of simulating how an image download over a fast modem would appear. The sleep() function is an extremely crude method of doing this--it has only one-second precision, which is too coarse to allow for a smooth simulation--but it is relatively portable and ubiquitous. Less portable but more precise alternatives include usleep() and various Windows API calls. But since no sane programmer would intentionally add a delay like this to the inner loop of a program except for demonstration purposes, I judged that sleep() was good enough for this. The combination of a one-second sleep interval and the default buffer size of 4096 bytes results in an apparent download speed that is 10% to 20% faster than a 33.6K modem can manage. In fact, it's close to the average connection speed of a 56K modem over typical phone lines.
As to readpng2_decode_data() itself, it is little more than a wrapper function for the libpng routine png_process_data(). Its arguments include a pointer to our mainprog_info struct, a pointer to the input buffer, and the number of bytes of input data; the only things it does besides calling libpng are copy the struct pointers and set up the usual error-handling code:
int readpng2_decode_data(mainprog_info *mainprog_ptr, uch *rawbuf,
ulg length)
{
png_structp png_ptr = (png_structp)mainprog_ptr->png_ptr;
png_infop info_ptr = (png_infop)mainprog_ptr->info_ptr;
if (setjmp(mainprog_ptr->jmpbuf)) {
png_destroy_read_struct(&png_ptr, &info_ptr, NULL);
mainprog_ptr->png_ptr = NULL;
mainprog_ptr->info_ptr = NULL;
return 2;
}
png_process_data(png_ptr, info_ptr, rawbuf, length);
return 0;
}
The struct pointers are copied merely because the alternative is to typedef them; the latter may be more efficient (though not necessarily, due to the extra level of indirection inherent in the -> operator), but it is also uglier and makes the code somewhat less readable.[101]
[101] Clarity and expediency, that's what we like. Well, we like efficiency, too, but not at the cost of clarity when writing a book on programming PNG.
png_process_data() is, in some sense, the last real libpng function that the main program calls--yet so far we haven't set any transformations and have virtually no information about the PNG image except that its signature is correct. The solution to these little mysteries lies within the first of the callback routines, readpng2_info_callback(). In most respects, it functions as the second half of our libpng initialization routine: it gets the PNG image's header information, including the image dimensions and perhaps the background color; it sets all of the transformations, including gamma correction; and it calls a routine in the main program to initialize the viewing window. In short, it does everything except handle actual pixels.
One important thing it does not do, however, is set up the usual error-handling code via the setjmp() function. The reason for this is simple: libpng requires that control never return to it when an error occurs; ordinarily, it longjumps to a user routine, which then returns an error value to the main program. But in this case it is libpng itself that calls readpng2_info_callback(), so a longjump back to here would make no sense--the only things we could do would be to return to libpng or call exit() without cleaning up, which is a rather brutal method of handling an error. (Well, actually we could do our own longjump back to the main program, but that's effectively what we are already doing. And in the last chapter I noted my dislike of big goto statements.) By not calling setjmp() within the callback, any errors will return to the location of the previous setjmp() call, which was in readpng2_decode_data(). It can then return a proper error value to the main program.
There is a feature in the callback routine that has no analogue in the basic PNG reader, however:
mainprog_info *mainprog_ptr;
mainprog_ptr = (mainprog_info *)png_get_progressive_ptr(png_ptr);
if (mainprog_ptr == NULL) {
fprintf(stderr, "readpng2 error: "
"main struct not recoverable in info_callback.\n");
fflush(stderr);
return;
}
This is the way we retrieve our image-specific pointer from the bowels of the PNG structs. (If it's invalid, we're in big trouble already, but there's no need to compound the problem by dereferencing a NULL pointer and crashing immediately.) Having done so, we can now stuff the image dimensions into it, where they'll be used by the main program very shortly:
int color_type, bit_depth;
png_get_IHDR(png_ptr, info_ptr, &mainprog_ptr->width,
&mainprog_ptr->height, &bit_depth, &color_type, NULL, NULL, NULL);
As before, we called a libpng utility routine to retrieve information about the image. There are also so-called easy access functions to retrieve each item separately; the choice of one function call or several is purely a matter of taste.
See Chapter 13 for the detailed explanation, but trust me: it's not good karma.
As soon as we know the bit depth and color type of the image (via the png_get_IHDR() call we just made), we can check for a PNG bKGD chunk and, if it's found, adjust its values in exactly the same way as before:
![[!]](images/warning.png){kind=small}
if (mainprog_ptr->need_bgcolor &&
png_get_valid(png_ptr, info_ptr, PNG_INFO_bKGD))
{
/* do the same png_get_bKGD() call and scale the RGB values as
* required; put results in mainprog_ptr->bg_red, bg_green,
* and bg_blue */
}
This time, instead of passing the red, green, and blue values back through the arguments to a readpng2 function, we place them into the bg_red, bg_green, and bg_blue elements of our mainprog_info struct.
The next step is to set up the desired libpng transformations; this is completely identical to the code in the first demo program. It is followed by the gamma-correction setup, but here we depart slightly from the previous example:
if (png_get_gAMA(png_ptr, info_ptr, &gamma))
png_set_gamma(png_ptr, mainprog_ptr->display_exponent,
gamma);
else
png_set_gamma(png_ptr, mainprog_ptr->display_exponent,
0.45455);
Because this program is intended to provide an example of how to write a PNG reader for a web browser, we imagine that the files it will be viewing are coming from the Internet--even though the front ends we provide only read from local files, just as in the basic version. Because images from the Internet are more likely to have been either created on PC-like systems or intended for display on PC-like systems, we follow the recommendation of the sRGB proposal (see Chapter 10, "Gamma Correction and Precision Color") and assume that all unlabeled images live in the sRGB color space--which, among other things, means they have a gamma of 1/2.2 or 0.45455, the same as most PCs and workstations. This does mean that unlabeled images created on a Macintosh, SGI, or NeXT workstation and intended for display on one of these systems will appear too dark. But that, of course, is why including a gamma value in the image file is so vitally important.
There is one last ``transformation'' to register after the gamma handling is out of the way; we want libpng to expand interlaced passes for us. This is signaled by calling png_set_interlace_handling(). It returns the number of passes in the image, which we save in case the main program wants to report to the user whether the image is interlaced (seven passes) or not (one pass):
mainprog_ptr->passes = png_set_interlace_handling(png_ptr);
Then we have libpng update the PNG struct information and return to us the final number of channels in the image and the size of each row:
png_read_update_info(png_ptr, info_ptr);
mainprog_ptr->rowbytes = png_get_rowbytes(png_ptr, info_ptr);
mainprog_ptr->channels = png_get_channels(png_ptr, info_ptr);
The very last thing readpng2_info_callback() does is call its corresponding function in the main program, which allocates the image memory, initializes the windowing system, and creates the display window with the proper dimensions:
(*mainprog_ptr->mainprog_init)();
return;
Recall that we saved pointers to three functions in the mainprog_info struct; this calls the first of the three. If we didn't care about separating PNG code from the main program routines, we could use just one routine per callback. But this way is a bit cleaner, and the performance hit is minimal.
The heart of the progressive reader is the row-callback function. As with the other two callbacks, it is called by png_process_data() after some amount of image data is read; unlike them, it gets called multiple times, at least once for every row in the image.[102] readpng2_row_callback() has four arguments: the main PNG struct pointer, a pointer to the row of image data, the row number, and the pass number. Its structure is actually quite simple; most of the action occurs within libpng or back in the main program:
[102] For interlaced images, it gets called (with real data) an average of 1.875 times per row and at most 4 times per row (for a one-row image that is more than four pixels wide). Once per row is still a possibility, however, if the image has only one column.
static void readpng2_row_callback(png_structp png_ptr,
png_bytep new_row,
png_uint_32 row_num,
int pass)
{
mainprog_info *mainprog_ptr;
if (!new_row)
return;
mainprog_ptr = (mainprog_info *)png_get_progressive_ptr(png_ptr);
png_progressive_combine_row(png_ptr,
mainprog_ptr->row_pointers[row_num], new_row);
(*mainprog_ptr->mainprog_display_row)(row_num);
return;
}
The first thing the routine does is check whether libpng provided any row data; if not, it returns immediately. Otherwise the function needs access to our mainprog_info struct, so it retrieves the pointer to that. Recall that the definition of this struct included two members that should look familiar: image_data and row_pointers. The first is the pointer to our image buffer; the second is an array of pointers giving the locations of every row within the buffer. Both were allocated and initialized when readpng2_info_callback() called its corresponding function in the main program. libpng does not require a row-pointers array in a progressive reader, but it happens to be a convenient and reasonably efficient way to access the image buffer.
In any case, the row-callback function calls png_progressive_combine_row() to combine the new image data with the existing pixels or, in the case of a noninterlaced image, to copy the row of data into the image buffer. Then it transfers control to its counterpart in the main program in order to composite the new pixels with the background, convert the row to a platform-dependent format, and optionally display it.
The main-program code to do all of this is almost identical to that in the first demo program, but this time around we've added a small twist: the code now supports not only a user-defined background color but also a background image of sorts. Specifically, the user has the option of choosing one of a set of predefined background patterns that simulate a tiled background image. The patterns currently include gradient-filled checkerboards (three of which are shown in the second row of Figure C-5 in the color insert), smoothly interpolated diamonds (third row of Figure C-5), and radial waves (Figure C-1 and fourth row of Figure C-5); eventually, other patterns may be defined. This approach is simple enough that it could be generated on the fly, as the image is displayed, but in the interests of speed and simplicity, I chose to define a second complete image buffer in the mainprog_init() function. The background buffer is filled as follows for the diamond pattern (contributed by Adam M. Costello):
hmax = (bgscale-1)/2; /* half the max weight of a color */
max = 2*hmax; /* the max weight of a color */
for (row = 0; row < rpng2_info.height; ++row) {
yidx = row % bgscale;
if (yidx > hmax)
yidx = bgscale-1 - yidx;
dest = bg_data + row*bg_rowbytes;
for (i = 0; i < rpng2_info.width; ++i) {
xidx = i % bgscale;
if (xidx > hmax)
xidx = bgscale-1 - xidx;
k = xidx + yidx;
*dest++ = (k*r1 + (max-k)*r2) / max;
*dest++ = (k*g1 + (max-k)*g2) / max;
*dest++ = (k*b1 + (max-k)*b2) / max;
}
}
With this approach, the inner display loop requires only a tiny change to support the background image instead of just a background color:
r = *src++;
g = *src++;
b = *src++;
a = *src++;
if (bg_image) { /* NEW */
bg_red = *src2++; /* NEW */
bg_green = *src2++; /* NEW */
bg_blue = *src2++; /* NEW */
} /* NEW */
if (a == 255) {
red = r;
green = g;
blue = b;
} else if (a == 0) {
red = bg_red;
green = bg_green;
blue = bg_blue;
} else {
/* this macro (copied from png.h) composites
* the foreground and background values and
* puts the result into the first argument */
alpha_composite(red, r, a, bg_red);
alpha_composite(green, g, a, bg_green);
alpha_composite(blue, b, a, bg_blue);
}
In other words, the background color used for compositing is now changed once per pixel. (Note that the src2 pointer is initialized just once per call. That's the only other change to the display routine to support the background image.) The cases in which the alpha component is either 255 or 0 are handled separately for performance reasons only; using the alpha_composite() macro would produce identical results. But because the macro employs multiplication, addition, and bit-shifting for every pixel (in fact, three times per pixel) and because fully opaque and fully transparent pixels are generally by far the most numerous, the difference in speed would probably be noticeable. It therefore makes sense to handle the two special cases separately. Whether full opacity or full transparency is handled first is less obvious; I guessed that opaque pixels are likely to be more common in images with transparency, so the 255 case is checked first.
The results, using one of the more exotic radial-wave patterns as the background, are shown in Figure C-1 in the color insert. The base image consists of partially transparent icicles hanging from opaque tree branches, seen against a completely transparent sky. The figure is a composite of the appearance after the first PNG pass (left half) and the final pass (right half).
Once the last row-callback has been made, the program is basically done. Because of the way the main program's row-display code was written to deal with interlaced images, when the last row of pixels is sent, it is guaranteed to be flushed to the display immediately. Thus, when libpng calls our final callback routine, readpng2_end_callback(), it does nothing more than retrieve the pointer to our mainprog_info struct and call the corresponding mainprog_finish_display() function, which in turn merely sets a ``done'' flag and lets the user know that the image is complete:
static void rpng2_x_finish_display()
{
rpng2_info.done = TRUE;
printf("Done. Press Q, Esc or mouse button 1 to quit.\n");
}
It would also have been reasonable to free both the image_data and bg_data buffers at this point, and a memory-constrained application certainly would do so--or, more likely, it would never have allocated full buffers in the first place, instead handling everything on a per-row basis and calculating the background pattern on the fly. Regardless, I chose to free all front-end buffers in the front-end cleanup routine, which is the last function called before the program exits.
Before that happens, though, the mainprog_finish_display() routine returns control through readpng2_end_callback() to libpng and eventually back to the main program loop, which is now finished. The main program then closes the PNG file and calls readpng2_cleanup() to deallocate the PNG structs:
void readpng2_cleanup(mainprog_info *mainprog_ptr)
{
png_structp png_ptr = (png_structp)mainprog_ptr->png_ptr;
png_infop info_ptr = (png_infop)mainprog_ptr->info_ptr;
if (png_ptr && info_ptr)
png_destroy_read_struct(&png_ptr, &info_ptr, NULL);
mainprog_ptr->png_ptr = NULL;
mainprog_ptr->info_ptr = NULL;
}
Once that is done, the program waits for user input to terminate, then it exits.
All of the source files for the rpng2 demo program (rpng2-x.c, rpng2-win.c, readpng2.c, readpng2.h, and makefiles) are available via the web, under a BSD-like Open Source license. The files will be available for download from the following URL for the foreseeable future:
http://www.libpng.org/pub/png/pngbook.html
Bug fixes, new features and ports, and other contributions may be integrated into the code, time permitting.