Joint Master Degree in

Image Processing and Computer Vision (IPCV)

Course overview and logistics

Applied Video Sequence Analysis
2023
Video Processing
and Understanding
Lab
Escuela Politécnica Superior
 Applied Video Sequence Analysis
• Teaching staff

Dr. José Mª Martínez Dr. Juan C. San Miguel Ms Paula Moral

Office C201 (C Building) Office C205 (C Building) Office C111 (C Building)
josem.martinez@uam.es juancarlos.sanmiguel@uam.es paula.moral@uam.es

Lectures for Unit I-II Lectures for Unit III Teaching assistant for
Labs for Unit II Labs for Unit III evaluation support
Course Coordinator of Labs for Unit II and III

Course overview “Applied Video Sequence Analysis” 2

 COURSE PREREQUISITES
• Fundamental notions of probability and mathematics
• Fundamental notions of signal processing
• Basics of image processing
• Programming experience in C or Java

Course overview “Applied Video Sequence Analysis” 3

 LEARNING OUTCOMES
• By the end of the course, the student must be able to:
−Identify major challenges for video processing
−Identify major challenges in current video-surveillance systems
−Identify the required tasks for a given video application
−Knowledge of advantages/drawbacks for main video processing tools
−Implement and evaluate video processing algorithms

• Transversal skills
−Write scientific and technical reports
−Structured peer collaboration

Course overview “Applied Video Sequence Analysis” 4

 SCHEDULE
• Course duration: 2nd February – 12th May
• Lesson types & activities: Lectures & Labs
• Schedule
−02/Feb/23 to 16/Mar/23: Tuesday 12–14 & Thursday 10–12
−20/Mar/23 to 12/May/23: Tuesday 12–14 & Wednesday 12–14
−See Moodle course for the detailed schedule of lectures and labs
• Room: Laboratorio 13 (A building, 1st floor)
• Out-of-class support: ask for appointment by email

Attendance marking for every 2h-slot:

Look for the icon in the introduction tab of the Moodle course
Course overview “Applied Video Sequence Analysis” 5
 LIST OF ACTIVITIES
Modality Activity Hours % Hours %

PRESENTIAL Lecture sessions 24 16%

Practical programming 22 14%

48 32%
sessions
Test and exams 2 2%

NON Weekly study of lectures 24 16%

PRESENTIAL (~2h/week)
Practical work 60 40%
(~15h/assignment) 102 68%

Preparation of exams 18 12%

(~8-10 hours/exam)
TOTAL WORKLOAD: 25 hours x 6 ECTS 150 100%

Course overview “Applied Video Sequence Analysis” 6

 “APPLIED”
• The course is focused on a wide area of application:

Video-surveillance or video-monitoring

Course overview “Applied Video Sequence Analysis” 7

 ACTIVITIES: LECTURES
UNIT I – Introduction to video-monitoring systems (2h)
 Systems, open challenges, camera networks,…

Video Input

Foreground
Segmentation

Blob Tracking

Person-Object
Classification

Feature
Extraction

Event
Detection

Course overview “Applied Video Sequence Analysis” 8

 ACTIVITIES: LECTURES
• UNIT II: Foreground/Objects detection and segmentation (8h)
−Background subtraction: parametric and non-parametric models
−Shadow detection
−Specific object detectors

Course overview “Applied Video Sequence Analysis” 9

 ACTIVITIES: LECTURES
• UNIT II: Foreground/Objects detection and segmentation (8h)
−Background subtraction: parametric and non-parametric models
−Shadow detection
−Specific object detectors

Course overview “Applied Video Sequence Analysis” 10

 ACTIVITIES: LECTURES
• UNIT II: Foreground/Objects detection and segmentation (8h)
−Background subtraction: parametric and non-parametric models
−Shadow detection
−Specific object detectors
Moving object extraction

AVSS2007 - ftp://motinas.elec.qmul.ac.uk/pub/iLids/ PETS2001 - http://ftp.pets.rdg.ac.uk/PETS2001/

Group Person Vehicle Object Train

Course overview “Applied Video Sequence Analysis” 11
 ACTIVITIES: LECTURES
• UNIT II: Video object tracking (16h)
−Template matching & Mean-shift tracking
−Kalman and Particle Filters
−Tracking by detection

Course overview “Applied Video Sequence Analysis” 12

 ACTIVITIES: LECTURES
• UNIT III: Video object tracking (16h)
−Kalman and Particle Filters
−Template matching & Mean-shift tracking
−Tracking by detection

Detection
Tracking

Course overview “Applied Video Sequence Analysis” 13

 ACTIVITIES: LECTURES
• UNIT III: Video object tracking (16h)
−Kalman and Particle Filters
−Template matching & Mean-shift tracking
−Tracking by detection

Course overview “Applied Video Sequence Analysis” 14

 ACTIVITIES: LECTURES
• UNIT III: Video object tracking (16h) – Multiple objects
−Template matching & Mean-shift tracking
−Kalman and Particle Filters
−Tracking by detection

Course overview “Applied Video Sequence Analysis” 15

 ACTIVITIES: LABS
• Programming assignments every 3-4 weeks in C++/OpenCV
−Introduction to OpenCV (~4h)
−Four assignments (9-10 sessions, ~18h)
LAB1 Foreground segmentation and shadow detection (3 sessions)
LAB2 Blob extraction & classification (3 sessions)
LAB3 Object tracking using Kalman (1.5 sessions)
LAB4 Model-free object tracking (1.5 sessions)

• Work in pairs
• Submission via Moodle
−Code & how to run
−Report to asked questions

• 4-day late policy

for all assignments
Course overview “Applied Video Sequence Analysis” 16
 ACTIVITIES: LABS
• Programming assignments every 3-4 weeks in C++/OpenCV
• Methodology based on pair programming

…so by the end of the course,

the full pair-programming methodology will be implemented

LAB1 & LAB2: focus on understanding the tools & environment

LAB3 & LAB4: focus on the pair-programming methodology

Course overview “Applied Video Sequence Analysis” 17

 ACTIVITIES: LABS
• Programming assignments every 3-4 weeks in C++/OpenCV
• Methodology based on pair programming
(mixing hybrid pair programming1 and moderated two-stage projects2)
−Three stages: individual + followup + collaborative (in pairs)

Proposed methodology Working material: guidelines with 1-2

programming exercises and 1-2 open
questions. Peer interaction not allowed.
Stage 1 (individual) These tasks are required for the
1 week

collaborative work in stage 2.

3 weeks

Followup session
2 weeks

Assignment submission: code for each

Stage 2 (collaborative) task and brief report answering the
questions (max 2 pages)

1Chen, J. et al. Improve student performance using moderated two-stage projects. ACM Conference on Global Computing Education, 2019
2Yuan, H. et al. Hybrid pair programming-a promising alternative to standard pair programming. ACM Tech. Symp.on Computer Science Education, 2019

Course overview “Applied Video Sequence Analysis” 18

 ACTIVITIES: LABS
• Programming assignments every 3-4 weeks in C++/OpenCV
• Methodology based on pair programming
(mixing hybrid pair programming1 and moderated two-stage projects2)
−Three stages: individual + followup + collaborative (in pairs)

Proposed methodology

Stage 1 (individual)
1 week

Address major limitations found in the

3 weeks

individual work and provide feedback

Followup session to improve in the collaborative stage
2 weeks

May not be needed if marks of stage 1

Stage 2 (collaborative) are sufficiently good.

1Chen, J. et al. Improve student performance using moderated two-stage projects. ACM Conference on Global Computing Education, 2019
2Yuan, H. et al. Hybrid pair programming-a promising alternative to standard pair programming. ACM Tech. Symp.on Computer Science Education, 2019

Course overview “Applied Video Sequence Analysis” 19

 ACTIVITIES: LABS
• Programming assignments every 3-4 weeks in C++/OpenCV
• Methodology based on pair programming
(mixing hybrid pair programming1 and moderated two-stage projects2)
−Three stages: individual + followup + collaborative (in pairs)

Proposed methodology Working material: guidelines with 3-4

programming exercises and 2-4 open
questions. Focus on collaboration. Work
Stage 1 (individual) from stage 1 is required.
1 week
3 weeks

Followup session
2 weeks

Assignment submission: code for each

Stage 2 (collaborative)
task, brief report answering the
questions (max 4 pages) & reflection
on the individual/collaborative work
1Chen, J. et al. Improve student performance using moderated two-stage projects. ACM Conference on Global Computing Education, 2019
2Yuan, H. et al. Hybrid pair programming-a promising alternative to standard pair programming. ACM Tech. Symp.on Computer Science Education, 2019

Course overview “Applied Video Sequence Analysis” 20

 ACTIVITIES: LABS
• Why OpenCV and C++?
−Developed by Intel in 2000 and now open source
−Very popular tool in the Computer Vision community
−Supports many OS: Windows, Linux, Mac-OS,…
−Supports many languages:
 Python, Java, C++, MATLAB…

General Image Processing Functions Image Pyramids

Geometric
descriptor
Segmentation s Camera
calibratio
n,
Features Stereo,
Transforms 3D
Utilities and
Data Structures

Tracking
Machine
Learning: Fitting
• Detection,
• Recognition
Matrix Math
21 © Gary Bradski

More info at https://bit.ly/2YapEGX

Course overview “Applied Video Sequence Analysis” 21
 ACTIVITIES: LABS
• Why OpenCV and C++?
−Key for real-time operation and embedded systems
−Professional deployment based on C++ (OpenCV or NVIDIA libraries)

https://www.tiobe.com/tiobe-index/

https://www.techrepublic.com/article/the-best-
programming-languages-to-learn-in-2022/

Course overview “Applied Video Sequence Analysis” 22

 ACTIVITIES: LABS
• Programming environment
Eclipse, gcc,… - Linux/MAC

You can use other IDEs of your choice

(but we only provide support to setup and run Eclipse)
Course overview “Applied Video Sequence Analysis” 23
 ACTIVITIES: LABS
• Writing task for each lab assignment
− Objective: accurate technical writing
− Iterative learning process
− Incremental weight in the evaluation

Write Feedback
(student) (lecturer)

Revise
(student)

Course overview “Applied Video Sequence Analysis” 24

 EVALUATION

𝐹𝐹𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 𝐺𝐺𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 = 40%TE + 60%PR

−TE: Grade obtained from Lectures (Theory)
𝑇𝑇𝑇𝑇 = 50%TE1 + 50%TE2
TE1 – Test via Moodle for unit I & II –16/March/2023 (tentative)
TE2 – Test via Moodle for unit III – 10/May/2023 (tentative)
Course slides and notes can be used

−PR: grade obtained from programming assignments (Practice)

𝑃𝑃𝑃𝑃 = 25%PR1 + 25%PR2+25%PR3 + 25%PR4
Tentative deadlines: 03/03/23, 24/03/23, 25/04/23, 16/05/23
(final deadlines will appear in the Moodle course)

If final grade is below 5.0 (or attendance requirements are not met)
students must do the retake exam the 7th June 2023
which considers the whole content of the course
Course overview “Applied Video Sequence Analysis” 25
 SURVEY TO UNDERSTAND YOU BETTER

Course overview “Applied Video Sequence Analysis” 26

 TASKS FOR YOU (before the end of this week)

In the AVSA course of Moodle

(1) select your partner for the
programming assignments
(2) Indicate your prior level
knowledge of some topics
(3) Upload a picture of yourself
to your Moodle profile
Course overview “Applied Video Sequence Analysis” 27
 Any Questions?

Course overview “Applied Video Sequence Analysis” 28