Texture Synthesis Using Image Quilting

In addition to recognition of specific objects in a given image, the eye has the ability to recognize "texture", a portion of the image that has a consistent statistical pattern that appears to represent the same type of image. Examples include the bark of a tree, grass on a lawn, stucco on a wall, and so on. In computer graphics applications, such texture is mapped onto the surfaces of objects to better evoke a sense of realism in the viewer.

One frequent problem, however, is that arbitrarily large geometric models may require an arbitrarily large amount of texture data, which often tends to be quite limited, especially in the case where it is taken from photography. A solution is texture synthesis by example, as implemented in this project by use of the Texture Quilting Algorithm of Efros and Freeman. Given a source example of texture, the algorithm can create an arbitrarily large target texture that appears to be of the same texture as the original.

It does this by sampling tiles from the source image, and placing entire tiles at a time in the destination. First, a random tile from the source is placed in the top left corner of the destination. Then, the system will look through all possible pixels in the source to find a tile that will best overlap with the tiles placed so far. A new tile is randomly selected from the best matches, and the algorithm computes the best cut along the overlapped region so that the new tile will be placed into the image with the minimum error. The advantage of this method over existing per-pixel methods is that entire blocks may be placed at a time, greatly acclerating speed.

We show a series of examples. The first image in each row is the source. The second image is a synthesized texture with blocksize 80, which appears to give good results for all source images. The third image in each row is a synthesized texture with blocksize 10. Clearly, we can see many artifacts introduced in the output image. This is because the blocksize is not large enough to pick up distinguishing detail, and simply repeats areas of constant texture. The effects are less noticable when there is more texture detail. Notice the image of the radishes, where the most of the image looks acceptable, except for parts of the leaves.

To run the examples, assuming you have all the files in the same directory, run something like the MATLAB command:

  Y = imagequilt(imread('brick.jpg'),80,5);

You may use this code however you wish, for either commercial or non-commercial projects. If you find it useful, please drop me a line, I'd love to hear about it.