
Abstract
Taking a photo with friends or family is always fun and memorable, but sometimes people in the photo may not be smiling by mistake. Our project covers this situation by constructing inpainting-based smiling generation. First, we carefully detect all possible faces and classify them as smiling or not smiling. Then, we generate a smiling emotion for each unsmiling face using a pre-trained inpainting model after detecting the lips region with a face parser. Finally, we forward masked image and the original image to the inpainting model to generate a smiling face. To validate practically, we implement a simple web demo via Gradio.
Overview
Explain the main intuition and provide a simple pipeline.

1. Face Detection
Detect all faces given an image.
2. Smile Classification
Classify each detected face as smiling or not smiling. (fine-tuned)
3. Smile Generation
Generate a smiling emotion based on pre-trained inpainting model.
Applications
Our framework provides useful and funny applications via Gradio. Please refer to our code repository.