Project 3 - Face Morphing

Part 1: Defining Correspondences

Here are the original images of Chris Evans and Chris Hemsworth.

I established point correspondences between two images to determine how one image maps onto the other. Using the provided correspondence tool, I selected key facial features and included the image corners to ensure accurate mapping. After computing the average points, I created a Delaunay triangulation based on this average shape. Below are two images displaying the triangulated faces of Chris Evans and Chris Hemsworth.

Part 2: Mid-way Face

I generated a mid-way face by blending both the shape and appearance of the two original images. This process involved several steps:

1. Computing the Average Shape:

I calculated the average positions of the corresponding key points from both images using the formula:

average_points = (1 – warp_fraction) × imageA_points + warp_fraction × imageB_points

With a warp fraction of 0.5, this effectively averages the coordinates of each corresponding point.

Delaunay Triangulation:

Using the averaged key points, I created a Delaunay triangulation. This divides the face into a network of non-overlapping triangles, which is crucial for piecewise affine transformations.

Affine Transformations for Each Triangle:

For each triangle in the triangulation, I computed two affine transformation matrices:

• One mapping the triangle from the average shape to the corresponding triangle in the first image.
• Another mapping to the corresponding triangle in the second image.

This was achieved using the computeAffine function, which solves for the affine transformation matrix that aligns the points of one triangle to another.

Warping Both Images into the Average Shape:

1. Apply the affine transformations to warp the pixels from both original images into the average shape.
2. Transform the coordinates of the pixels within each triangle to align with the average shape.
3. Interpolate the pixel values at these transformed coordinates.

Cross-Dissolving Colors:

After warping, I blended the colors of the two images by averaging their pixel values using the formula:

blended_pixels = (1 – dissolve_fraction) × pixels_from_imageA + dissolve_fraction × pixels_from_imageB

With a dissolve fraction of 0.5, this results in an equal mix of colors from both images.

Reconstructing the Mid-way Face:

I assembled the blended pixels of all triangles to form the complete mid-way face.

Part 3: The Morph Sequence

A morph sequence is generated by creating a series of intermediate images that gradually transform one face into the other(from part 2). I increased the warp and dissolve fractions [0,1] over 45 frames to produce a smooth animation.

Reset GIF

Part 4: The "Mean Face" of a Population

In Part 4, I computed the mean face of a population using a set of faces from a danes database. The goal was to find the average facial structure and appearance.

The process involved the following steps:

1. Calculate the average shape of all the faces.
2. Warp each face in the database into the average shape using the same logic as Part 2, which involves affine transformations and pixel interpolation.
3. Compute the mean face by averaging the pixel values of all the warped images.

Below is the resulting "Mean Face" of the population:

Here are some examples of the population morphed into the mean face:

Morphing my face and the "mean" face:

Part 5: Caricatures - Extrapolating from the Mean

I created caricatures by exaggerating the differences between my face and the mean face. This is done by extrapolating from the mean, pushing the features further away from the average. The same affine transformations and pixel interpolation logic from Part 2 are used to warp the faces, but this time I manipulated the warp_frac value to amplify or reduce the differences from the mean face.

Negative alpha (α = -0.5): This creates a caricature where the features are pulled toward the mean face, reducing unique characteristics.

Positive alpha (α = 0.5): This creates a caricature where the unique features are pushed further away from the mean, emphasizing their differences from the average.

Bells & Whistles

As part of the Bells & Whistles, I changed the gender of my friend's face using average Korean women face:

I tinkered around with the image by only warping, only cross dissolving, and both.