CS 7490 Spring 2001 Final Project Painterly Renderer Justin Jang, May 2001

Introduction

The goal of this project was to implement a painterly renderer based on the architecture in the the paper Painterly Rendering for Animation (Barbara J. Meier, SIGGRAPH 96, pp. 477-484).

The diagram below is the painterly rendering pipeline that I used (Figure 3 of the Meier paper). The C program I wrote implemented every stage (the particle placer, shaders, camera transform, and painterly renderer) shown in the diagram. For the geometry, I borrowed a triangle mesh of a familiar triceratops model. I drew four 64x64 greyscale images with my favorite painting program to use as brush images.

The Particle Placer

The Meier paper computed the surface area of each triangle in the model and randomly distributed particles within each triangle. The number of particles for a triangle is determined by the ratio of its surface area to the total surface area of the entire model. I implemented two different algorithms for placing particles in this manner, a global approach and a local approach.

For the global approach, I pick triangles at random with the probability of a triangle being picked as described above. Whenever a triangle is picked, a random point on that triangle is selected as a particle. (Joshua Gargus introduced this algorithm to me.)

In local approach, each triangle is considered once. For each triangle, the ratio of its area to the total area is multiplied by the total desired number of points. This many points are randomly picked on the current triangle. This approach has problems when many small triangles are present in the mesh, especially if there are many small triangles adjacent to each other. Because C rounds down, certain areas of the mesh may receive no points at all. To compensate for this, I made every triangle get atleast one point minimum which is little more than a hack. Of course, this hack has the opposite but more acceptable problem, dense patches of points. I also made it round correctly up to 6, resulting in slightly more particles picked overall (for all triangles that previously would have had less than 6 points).

(Click on images for larger views.)
Source triangles	Local particle picking (with no one particle minimum)
Global particle picking	Local particle picking (with one particle minimum per triangle)

The Shaders

The Color Reference Picture

To create the color source image (color reference picture), I texture mapped the triceratops model and wrote some simple shading code to make it look lit from the right.

The Orientation Reference Picture

To create the orientation source image (orientation reference picture), I drew the triangle mesh and flat shaded each triangle with a color encoding of the orientation. To get this color, I projected the triangle's normal onto the viewplane and encoded this 2D vector in the red (x component) and green (y component) channels of the color. I mapped the vector component range [-1, 1] to the color component range [0, 1]. Therefore, regions with extremely high red and extremely low red content specify right and left pointing vectors respectively. Likewise, regions with extremely high green and extremely low green content specify up and down pointing vectors respectively.

The Size Reference Picture

The size source image (size reference picture) was created similarly to the orientation source image, except the x-size and y-size are encoded instead of the x and y vector components. Also, instead of projecting the normal to the viewplane, I dot the normal with the view direction vector. As a result, triangles that face the viewer will force larger brush strokes. In my implementation of the size shader, I clamp the range [0, 1] to [0.2, 0.5] so that the brush can never get too small. This value is the same for both x-size and y-size. I multiply this value by user-defined x and y brush scale factors to get different brush shapes. Because x-size and y-size are encoded in red and green, yellow is a prominent color in this image.

(Click on images for larger views.)
Color Reference Picture	Orientation Reference Picture
Size Reference Picture	Painted Image

The Painterly Renderer

The final step is to paint the picture by drawing brush strokes. A brush stroke is draw for every particle in the model. Each particle is transformed and sorted from back to front with respect to the viewpoint (i.e. the eye or the center of projection). The particles are then projected onto the viewplane. The location of the projection a particle onto the viewplane is used to grab the brush parameters for that particle from the source images. The color from the color source image is combined with the brush alpha map to make the base brush stroke. This stroke is then scaled according to the x-size and y-size from the size source image. Then, the stroke is rotated according to orientation from the orientation source image. Finally, the brush is draw onto the image.

The four brush images used.

Automatic Brush-Size Scaling

When an object is moved closer or further away from the viewer, the object appears larger and smaller. The brush strokes should account for this change in size. Since the 3D view is a perspective projection, a good strategy for proper scaling should come out of the perspective projection equations. Since x'=d*x/(d+z), we can scale accordingly with z-scale-factor=zsf/(d+z). Assuming the initial brush sizes are correct for the initial position of the object, we can calibrate the z-scale-factor with the initial d and z values, zsf=d+z. This makes z-scale-factor=1 initially. As the object moves further from the eye, z increases and z-scale-factor decreases, and vice versa. This scale factor is applied to the brush strokes in addition to the x-size and y-size scale factors.

A separate zsf should be maintained for each object that moves independently of the rest of the scene. This generalizes to particles, as long as a zsf is maintained for each particle that moves independently.

Results

My implementation of the painterly renderer allows for interactive tweaking and comparing of various approaches and parameters in the painterly rendering pipeline. Here are some examples of the painterly renderer at work.

(Click on images for larger views.)
Painted images with different views, brushes, and brush sizes.

(Click on images for larger views.)
Global particle set	Local particle set
Paintings with the two different particle sets.

(Click on images for larger views.)
Near	Far
Object at different distances from the eye.

Future Additions

• Add more brushes. Add brushes with color and alpha (as opposed to only alpha).
• Add background scenery (land, sky, trees) and multiple objects into the scene.
• The brush parameters in my implementation all came from the source images As mentioned in the paper, the brush parameters could also be attached to the particles or a combination of the two could be employed. Facilitate the comparison of the two techniques and a hybrid.

Future Work

• Further investigate the generalization of the z-scale-factor to particles.
• Investigate the use of a heirarchical scheme for organizing the particles such that fewer brush strokes are used for further objects.
• Investigate the use of a hybrid scheme of a particle picking scheme and a particle scaling scheme.
• Explore other orientation and size schemes. Consider a tight coupling between the two schemes. Furthemore, consider a tight coupling between the two schemes and deformable brushes. (The paper mentions deformable brushes in the Future Directions section.) This information can also be stored in the source images, e.g. using the blue channels.

Appendix

Keyboard Controls

  e
s d f  - rotate object

  E
S D F  - translate object

Q/q    - translate object away ("zoom out")
A/a    - translate object towards ("zoom in")

SPACE  - reset transformations

1-4    - use brushes 1-4

  u
h j k  - brush scale factor adjustment; range=[0.2, 2.0], delta=0.1

0      - show particles
9      - show triangles

ENTER - show painting 
i     - show color source image
o     - show orientation source image
p     - show size source image

I     - dump color source image to file "source_color.raw"
O     - dump orientation source image to file "source_orientation.raw"
P     - dump size source image to file "source_size.raw"
[     - dump painting image to file "painting.raw"

,     - particle set 1 (global selection)
.     - particle set 2 (local selection)

ESC - quit

Notes

The file debug.txt contains some run-time statistics.

Contact

Justin Jang
College of Computing
Georgia Institute of Technology
jang@cc.gatech.edu