PIESC: Person of Interest Extraction, Segmentation and Classification Project

Overview: PIESC is a project that involves AI to extract, segment, and classify all the people
that appear in hundreds of images and video clips.

Imagine a tragic event that involves thousands of people, for which we have hundreds or
thousands of different images and videos taken by many participants from different devices
(smartphones, security cameras, dashboard cameras, etc).

The aim is to list all unique persons and extract good-looking images, especially facial ones, of
all individuals who appear on the sources the media files and match them / cross reference
against all other source media files.

Input:

A folder containing X number of videos and Y number of images in .mp4, .mv, .png, and jpg
formats.

Output:

A directory with processed images of persons who appear in the input videos and images in the
following tree example:

├── input
│ └── processed
│ ├── images
│ │ └── some_image.png
│ └── videos
│ └── some_video.mp4
└── output
└── persons
└── person_1
├── full_body_images
│ └── 1.png
├── headshots
│ └── 1.png
├── images
│ └── 1.png
├── profile
│ ├── face_aligned.png
│ ├── face_aligned_animation.gif
│ └── face_aligned_inhanced.png
└── videos
└── 1.mp4
Each unique person will have a directory created named person_[UNIQE_ID] that contains all
the instances of his/her appearance:

If it's an image and it hasn't been processed:

1. Create a copy of that image and save it with name iteration (1.png, 2.png, etc)

2. Extract full body images (if applicable) and headshot images of the person and save
them.

Note: the existence of the same image should be evaluated using something similar to
https://github.com/nikvoronin/we-aint-same

If it's a video and it hasn't been processed:

1. Extract a video cut from the input video from when he first appeared until when he
finished to appear in the input video. Save it with naming iteration (1.mp4, 2.mp4, etc)

2. Extract full body images (if applicable) and headshot images of the person and save
them.

Create a profile image for the person that contains the below 3 elements:

1. An aligned face photo (face_aligned.png).

2. An enhanced face photo using
(https://huggingface.co/spaces/clem/Image_Face_Upscale_Restoration-GFPGAN)
(face_aligned_inhanced.png)
3. An aligned face gif using NextFace (https://github.com/abdallahdib/NextFace)
(face_aligned_animation.gif)

The program should be able to handle new images/videos in the future. If a person was
previously identified and a person matched with a previous one, it would add his/her
videos/images to their already-made directory.

Note:
 ● We need to think about a way to match persons/suspected same person as the same
and merge them.
● We need to figure out how to avoid matching incorrect people as the same by mistake.
● We need to use open-source projects whenever it is possible.
● We need to figure out which AI tools are best for each step and what server we need to
spin in order to facilitate continuous results as new images/videos will keep rolling in.

Please devise a plan on how you would tackle the task and what tools, including which specific
AI models you would use to facilitate the results.