Face Image Animation
Introduction
We discuss the details about the Face Image Animation here. We use the edge maps as the structure guidance of target frames. Given an input reference image and a guidance video sequence, our model generating a video containing the specific movements.
Left: Input image; Right: Output results.
Dataset Preparation
We use the real video of the FaceForensics dataset. It contains 1000 Videos in total. We use 900 videos for training and 100 videos for testing.
- Crop the videos so that the movement of faces is dominant in the videos.
- Extract video frames and put the images into the forder
dataset/FaceForensics/train_data
- Download face key point extractor
shape_predictor_68_face_landmarks.dat
form here. Put the file under the forder dataset/FaceForensics
.
- Run the following code to generate the keypoint files for the extracted frames.
python script/obtain_face_kp.py
Training and Testing
Run the Following code to train our model.
python train.py \
--name=face \
--model=face \
--attn_layer=2,3 \
--kernel_size=2=5,3=3 \
--gpu_id=0,1 \
--dataset_mode=face \
--dataroot=./dataset/FaceForensics\
You can download our trained model from here.
Put the obtained weights into ./result/face_checkpoints
. Then run the following code to generate the animation videos.
python test.py \
--name=face_checkpoints \
--model=face \
--attn_layer=2,3 \
--kernel_size=2=5,3=3 \
--gpu_id=0 \
--dataset_mode=face \
--dataroot=./dataset/FaceForensics \
--results_dir=./eval_results/face \
--nThreads=1
The visdom is required to show the temporary results. You can access these results with:
http://localhost:8096