We have seen so far that upsampling can improve image presentation and communication and that increases in processing power and algorithm sophistication can improve the quality of the upsampled image. One also sees that interactive zoomable presentation improves the viewer experience, draws people into images and keeps them on the image longer. The interactive interface also opens doors to numerous new strategies for communications, including wormholes to new images, refocus, golden keys, etc.
Image upsampling is not a panacea, however. There are limits to how much upsampling is desireable (~3x in each dimension typically) and there are images for which upsampling yields unsatisfactory results.
Four effects in particular limit the utility of upsampling:
1. Highly compressed images tend to look worse when upsampled. The 1992 jpeg standard uses an 8 by 8 dct block. Compression consists of selecting features from this block. The structure of the block is apparent in upsampled compressed images. See for example this image:
These artifacts dominate the upsampled versions of the small images a couple of pages ago (click here to refresh your memory.)
2. Images at low exposure values or with high fixed pattern noise have speckle-like structure at the pixel level. Such images look extremely noisy when viewed at pixel level resolution and this noise pattern is amplified for human view when upsampling. This structure appeared to some degree in all images processed. It can be ameliorated by integrated demosaicing, denoising and upsampling in a common algorithmic structure.
3. Aliasing cannot be easily removed in upsampling and highly aliased features become more apparent in upsampled images.
4. Images that are already oversampled relative to their resolution (e.g. poorly focused images) do not look better when upsampled.
Simple Fourier analysis can be used to determine the utility of upsampling. Artifact 1 is indicated if Fourier features are correlated to the jpeg block size, 2 is indicated if the image has broad high frequency components with no pattern and 4 is indicated if the image is low frequency relative to the sampling grid. Aliasing artifacts are more challenging to detect.
Given the very large number of images and users available to Flickr, one may also choose to mine user feedback to adjust image servers dynamically. With tile servers, one knows where people are looking and the strategy they are taking to look. One may monitor the path viewers take through images and viewer satisfaction to adjust server characteristics for a given image and for similar images.
Archie's email also wonders how super-res effects image/face recognition? For a human observer, upsampling improves image communication and thus will improve feature and face analysis. For computers, however, upsampling increases the dimensionality of the data and degrades the performance of automated image analysis. One does better, in fact, with computer analysis to reduce dimensionality with feature based filters prior to image analysis.