On the main page is a link to a mini-tutorial on how to load in a texture image into an array of packed r,g,b pixels.

As I said in class, I'm going to open
things up a bit between now and the end
of the semester.
You can focus on one or both of two directions
moving forward:
*(i)*
texturing, and
*(ii)*
animation.
You can also choose one or more other
topics if you check with me first.
As always, your job is to implement
the requisite technology,
and then to create interesting
original content that shows
that technology in action.

I have to get your grades in by Dec 23. I am going to try to give you as much time as possible between now and then to get your projects completed. Please note that I am going to spend some time in class next Tuesday (Nov 30) looking on-line at select projects that students have implemented to date. If you want me to look at something in particular then, please let me know.

In these on-line notes I am going to be focusing on textures. Meanwhile, you can also go ahead and start to play with animation instead. All you really need in order to do that is to build a model that has rotatable joints (such as the swinging arm example we looked at in class earlier this semester), and then start using the cubic spline curves that you have already implemented in order to create time-varying joint angles for rotating the joints in your animatable figure. For example, you might want to try to build a simple two-legged human figure and implement a walk cycle. Next Tuesday I am going to talk more extensively about animation topics.

In class we went over
texturing. First we covered
*texture mapping*
and *antialiasing*,
with particular emphasis on
*MIP mapping*.
Then we covered
*procedural texturing*,
with a mini-lecture on the
*noise function*
and how to use it to make
interesting textures.
Here are some notes
that follow that discussion.

At its most basic, texture mapping
is quite simple.
When you build a geometric mesh,
you place *parametric coordinates*
(u,v) at each vertex, in addition
to (x,y,z) and (n_{x},n_{y},n_{z}).
Note that you have just "fattened" each
vertex from six floating point numbers
to eight floating point numbers.

For most of the types of geometric meshes that you have been implementing, it is very straighforward to assign a (u,v) value to each mesh vertex - you can just use the parametric coordinates that you used to build the shape (sphere, cylinder, bicubic patch, ...) in the the first place.

When you move through the rendering
pipeline all the way to the pixel,
the (u,v) value at each vertex becomes a value of (u,v) at
each pixel to be shaded.
At that point
you can use this interpolated (u,v) to do a look-up into
a stored texture image, from which you can retrieve
a texture color
(r_{t},g_{t},b_{t}).

Note that (u,v) are not transformed by the matrix that transforms the location and normal of the vertex. But (u,v) are indeed linearly interpolated during scan conversion, along with all the other geometric data at each vertex.

If you do pure *vertex shading*
(as we have done up until now),
then you will only have (r,g,b) information
at each pixel once you have done the
z-buffer step.
In this case, the only thing you can
do with the texture color is
to use it to modulate the color
at that pixel:

r *= r_{t}

g *= g_{t}

b *= b_{t}

On the other hand, you can
defer shading until later in the pipeline,
by interpolating the surface normal
vector coordinates down to the
pixel level, and doing the Phong algorithm
calculation at each pixel.
In this case, you can use the
retrieved texture to
modify *any* input parameter
to the Phong shading algorithm.
This includes any
information about the color or location of
light sources, as well as any
material surface parameter
such as the Ambient, Diffuse, or Specular
color, or the Specular power,
in addition to geometric data
such as the surface normal.

If you interpolate surface normals
between vertices
rather than colors, and then do the Phong algorithm at the pixel
level, then you are implementing
a *pixel shader*.
This can be much slower than doing a vertex
shader, but it is also much more flexible
and can produce more dramatic and varied
surface effects.
As I mentioned in class, pixel shaders
are extensively used for feature films,
when rendering time is not a critial issue,
as well as in hardware-accelerated shaders
enabled by GPUs (Graphics Processing Units),
such as the nVidia GForce and
ATI Radeon boards.

*Antialiasing*

As we discussed in class, you will generally get bad results if you simply use the (u,v) value at each pixel to do a look-up into your source texture image, because you will end up sampling improperly. The problem is that one image pixel can actually cover many texture source pixels, and what you ideally want is to perform an integral over the entire sub-area of the source texture image that is covered by each image pixel.

Doing a brute-force
calculation of this area integral
can be prohitibively expensive,
so people have devised various
methods to approximate this integral.
The most commonly used even today is
MIP mapping
(where "MIP" stands for *multim in parnum*),
developed by Lance Williams about 23 years ago.
Lance adapted the power-of-two image
pyramid strategy first developed
by Tanimoto in the late 1970's for
computer vision.
This is one example of many in which
computer vision algorithms for scene
analysis have a parallel in
computer graphics algorithms for
scene synthesis.

Conversion to an image pyramid recursively
proceeds by recursively halving
the resolution of the texture image.
At each stage of the recursion, every 2×2 pixels
is replaced by a single pixel that contains
the average value of those four texture pixels.
The recursion stops when you are left with
only a 1×1 image.
It is simplest to begin with a source
texture image that is a power of two in dimension,
256×256.
This is converted into a *pyramid*
of images:

128×128

64×64

32×32

16×16

8×8

4×4

2×2

1×1

For any pixel texture look-up,
we look not just at the value (u,v)
at that pixel, but also at the (u,v)
at one or more neighboring pixels in the
image to be rendered,
in order to get an approximation of how much
u and v change from one pixel to the next.
The magnitude of this variation *s* is used to create
a square shaped extent
(u__+__s/2,v__+__s/2),
over the texture image, which
approximates the shape in the texture
image that is covered by one pixel.

Now we can use this square shaped extent
to do a *tri-linearly interpolated* look-up
into the image pyramid,
in which we linearly interpolate in the three dimensions
of u, v, and scale.
As I discussed in class, this will
involve a total of eight accesses into
the image pyramid (four each at two
neighboring levels of the pyramid),
and a total of 2^{3}-1, or seven,
linear interpolation computations.

*Procedural texture*

*Noise function*

You can grab a copy of the source code for the noise function at: http://mrl.nyu.edu/~perlin/noise.

In class we went over an on-line tutorial about the noise function. You can review that tutorial at http://www.noisemachine.com/talk1/.

*Using the noise function to make procedural textures*

In addition to the examples at that tutorial, here is an on-line example of the use of noise function to vary surface normal: http://mrl.nyu.edu/~perlin/bumpy/ Feel free to look at the source code for class Sphere.java, in order to see how it all works.

The examples at that URL show procedural textures being used to vary the surface normal vector at each pixel, prior to performing the Phong shading algorithm at each pixel. Please note that this example does not use a real geometric model. Rather, the geometry of the spherical ball is faked by an image-based procedure, just for the purposes of the on-line demo.

In this on-line example, I first approximate the three partial derivatives of the noise function by taking differences in each dimension:

(noise(x+ε,y,z) - noise(x,y,z)) / ε

(noise(x,y+ε,z) - noise(x,y,z)) / ε

(noise(x,y,z+ε) - noise(x,y,z)) / ε

Then I *subtract* this
vector-valued derivative function from the
surface normal, to do *bump mapping*
prior to performing the Phong shading algorithm.
Bump mapping doesn't actually perturb the geometry
of the surface, but rather
fools the eye into thinking that the
surface is perturbed, by changing
the surface normals so that the surface
responds to light the way an actual bumpy surface would.

If you choose to implement procedural textures,
you shouldn't just duplicate
the textures that I show in that example.
Rather, you should play around with
the technique, and try to create
your *own*
examples of procedural textures that
vary Phong shading parameters or surface normal,
or some combination, to create an interesting
procedural texture.