Cheese! Turning Unsmiling into Smiling
Final Project for 2025 Winter Introduction to Deep Learning (MIE 1517), University of Toronto
Project teaser image

Abstract

Taking a photo with friends or family is always fun and memorable, but sometimes people in the photo may not be smiling by mistake. Our project covers this situation by constructing inpainting-based smiling generation. First, we carefully detect all possible faces and classify them as smiling or not smiling. Then, we generate a smiling emotion for each unsmiling face using a pre-trained inpainting model after detecting the lips region with a face parser. Finally, we forward masked image and the original image to the inpainting model to generate a smiling face. To validate practically, we implement a simple web demo via Gradio.

Overview

Explain the main intuition and provide a simple pipeline.

Method pipeline

1. Face Detection

Detect all faces given an image.

2. Smile Classification

Classify each detected face as smiling or not smiling. (fine-tuned)

3. Smile Generation

Generate a smiling emotion based on pre-trained inpainting model.

Applications

Our framework provides useful and funny applications via Gradio. Please refer to our code repository.