Go, change the world

RV College of
Engineering

MINOR PROJECT(18CS64)

Topic: Mutual Action Recognition Using

Deep Learning
Submitted by,
Shivashankar K -1RV19CS149
Shadakshari A -1RV19CS143
Tirumalesh M Naik -1RV19CS175
 RV College of
Engineering Contents Go, change the world

● Problem domain
● Problem statement
● Literature survey
● Objectives
● Dataset details
● Methodology
● Expected outcome
 RV College of
Engineering Problem Domain Go, change the world

Action recognition
Action recognition is a computer technology related to computer vision and image processing that
deals with detecting instances of semantic objects of a certain class in digital images and videos.
Applications:
● Games faul detection (penalty in football)
● security
● Video summarization
RV College of
Engineering Problem Statement Go, change the world

Our problem is to recognize the human-human interaction in a given video data.

build a LSTM based model which is trained on the selected UCF101 categories
which include interaction between them.
RV College of
Engineering Objectives Go, change the world

● Collecting the datasets.

● Takes the input from the video as well as the image.
● Integrating the two or more models for better accuracy and efficiency.
● Train and test dataset for action recognition and build the model.
● Recognition of mutual interaction between humans.
RV College of
Engineering Dataset Details Go, change the world

● UCF101 is an action recognition data set of realistic action

videos, collected from YouTube, having 101 action categories.
● This data set is an extension of UCF50 dataset which has 50
action categories.
● The videos in 101 action categories are grouped into 25 groups,
where each group can consist of 4-7 videos of an action.
● The videos from the same group may share some common
features, such as similar background, similar viewpoint, etc.
 RV College of
Engineering Dataset Details Go, change the world

The action categories can be divided into five types:

1. Human-Object Interaction
2. Body-Motion Only
3. Human-Human Interaction
4. Playing Musical Instruments
5. Sports.
RV College of
Engineering Methodology Go, change the world

step1:Download the Data with its Labels

Step 2: Preprocess the Dataset
Step 3: Split the Data into Train and Test Set
Step 4: Implement the ConvLSTM Approach
Step 5: Construct the Model
Step 6: Train the Model
Step 7: Test the Model on input videos
RV College of
Engineering CNN and LSTM Go, change the world
RV College of
Engineering Result and Analysis Go, change the world
 RV College of
Engineering Conclusion and Future Enhancement Go, change the world

● In this project , for mutual action recognition we propose a CNN, LSTM

method to recognise the relation between interacting people over time
Experimental results on person-person interaction recognition and
prediction have demonstrated the superior performance of the proposed
model.
● In future , we will use the Co-LSTM(Concurrence Aware Long short term
sub memories) for better accuracy and for activity recognition for the
multiple people environment.