Professional Documents
Culture Documents
USING GAIT
The Viterbi algorithm finds the most likely sequence of hidden states (gait
states) given the observed sequence of sensor data.
It is used to decode the underlying sequence of gait states from observed
sensor data.
This can provide valuable information about the individual's gait pattern and
behavior, such as identifying different phases of gait (e.g., walking, running,
standing) or detecting abnormal gait patterns.
Inference Algorithms: Difference between the two
Step 3: Calculate the expected number of transitions between states and the
expected number of times each state emits a particular observation, based on
the forward and backward probabilities computed in the previous step
Step 4: Update the model parameters: transition probabilities, emission
probabilities, initial state probabilities, based on the expected counts calculated.
This step involves maximizing the likelihood of the observation sequences
given the current model parameters.
Repeats these steps iteratively until the parameters of the HMM converge to
stable values.
The parameters obtained after convergence represent the trained HMM,
which provides a good model of the underlying gait patterns captured by the
observed sensor data.
View Invariant Gait Recognition
Imagine capturing a person's walking outline, like a shadow puppet. This essentially
describes the initial approach in gait recognition. Techniques relied on silhouettes, the
body's outline without details.
These early methods, while groundbreaking, had limitations. Variations in clothing and
viewing angles significantly impacted their accuracy.
SILHOUETTE-BASED METHODS
Imagine a person walking past a brightly lit wall. Early gait recognition techniques mirrored this scenario,
relying on silhouettes, the body outline without details. These silhouettes offered a basic understanding of a
person's body proportions and posture during walking.
1. Single Silhouettes: A simpler method captured a single silhouette at a specific point in the gait cycle,
typically when both feet were on the ground (stance phase). This provided a snapshot of the person's
posture at a particular moment, offering information about body proportions like leg length and torso
width.
• Clothing Variations: Changes in clothing significantly altered the captured silhouette. A bulky
jacket or loose pants could drastically affect the body outline, making identification challenging.
• Viewing Angle Dependence: Recognition performance dropped significantly when the view
wasn't ideal. Looking at someone from the side or back presented a completely different
silhouette compared to a head-on view, hindering accurate identification.
Due to these limitations, researchers began to explore the complexities of human movement itself,
leading to the development of dynamics-based gait recognition methods. The next section will delve
into how researchers shifted their focus to capture the essence of how a person walks, leading to
more robust identification systems.
Gait Energy Images (GEI): Capturing the Rhythm of
Movement
The GEI Concept:
Imagine stacking a sequence of silhouettes captured throughout a person's gait cycle, one on top of another.
Areas where the silhouettes frequently overlap will appear brighter in the resulting image. This cumulative
effect creates a "heat map" of gait patterns, where brighter regions highlight areas of the body that move more
frequently or with greater range during walking.
Benefits:
• Dynamic Glimpse: GEI captures some level of movement, highlighting frequently used body regions during
gait (e.g., pronounced hip swing).
• Data Efficiency: It compacts a silhouette sequence into a single image, potentially saving storage and
processing resources.
Drawbacks:
• Limited Dynamics: GEI only considers overlap frequency, not actual movement direction. It doesn't fully
capture the intricacies of gait.
• Noise Sensitivity: Clothing variations or changes in walking speed can affect the silhouette shape and
consequently the resulting GEI.
Gait Energy Images (GEI):
(Heatmap becomes more defined as the number of snapshots increase)
LIMITATIONS OF THE “SHAPE” APPROACH
While these initial methods provided a foundation, they faced limitations:
1. Improved Accuracy: Combining shape and dynamics provides a more comprehensive and distinctive
signature for identification. This can significantly improve recognition accuracy compared to pre-deep
learning approaches.
2. Enhanced Robustness: Deep learning models become less susceptible to factors that can affect gait
recognition, such as:
• Viewpoint variations: CNNs can learn to recognize gait patterns even if the person is captured from
different angles.
• Clothing and carrying objects: Deep learning models can focus on the underlying movement patterns
that are less affected by clothing or carried objects.
• Limited training data: Deep learning models have the ability to learn more effective representations
with smaller datasets compared to traditional methods.
WORKING OF A CNN BASED GAIT RECOGNITION MODEL
FACTORIAL HMM AND
PARALLEL HMM FOR GAIT
RECOGNITION
Feature and Decision Level Fusion with
FHMM and PHMM
1. While deep learning offers a powerful solution for gait recognition, other techniques can be valuable
depending on the specific application and data availability. Here, we explore two methods for
information fusion:
• Factorial Hidden Markov Models (FHMMs)
• Parallel Hidden Markov Models (PHMMs).
2. As discussed earlier, information fusion refers to the strategy of combining information from multiple
sources (e.g., different gait features) to create a more robust and accurate representation of an
individual's gait pattern. In gait recognition, this typically involves fusing features like shape
information (body proportions) and dynamic information (joint angles). By combining these, we can
overcome limitations associated with relying on a single feature, leading to:
•Improved Accuracy: Fusing information provides a richer description of gait, potentially leading to
more accurate identification compared to single-feature approaches.
•Enhanced Robustness: Recognition becomes less susceptible to factors that can affect gait
analysis, such as viewpoint variations, clothing, and limited training data.
FIRST OF ALL, A DIFFERENCE :
HMM VS NEURAL NETWORKS
HIDDEN MARKOV MODELS NEURAL NETWORKS
1. Focus on States and Transitions: HMMs represent a 1. Learn from Data: Neural networks are machine
system as a series of hidden states that you can't learning models that learn complex patterns from
directly observe. The model captures the probability of data. They consist of interconnected layers of artificial
transitioning between these states and the probability neurons that process information and improve their
of emitting certain observations from each state. predictions through training.
2. An HMM would model: 2. Light Bulb Example (using a Simple Neural Network):
• The probability of the bulb being burnt out (State 1) Imagine you have a dataset that includes information
or working (State 2). about the bulb's age, usage history (on/off cycles),
• The probability of the bulb transitioning from burnt and whether it's currently on/off. A neural network
out to working (unlikely) or vice versa (more likely). could learn from this data to:
• The probability of the bulb emitting light (observed • Predict the bulb's current state (burnt out or
as "on") when it's working (State 2). working) based on its age and usage history.
• The probability of the bulb emitting no light • Predict the likelihood of the bulb burning out in the
(observed as "off") even when it's working (due to a future.
switch being off) or because it's burnt out (State 1).
FACTORIAL HIDDEN MARKOV MODELS(FHMM)
One approach to information fusion in gait recognition is through Factorial Hidden Markov Models
(FHMMs). Unlike deep learning models that can learn complex relationships directly from data, FHMMs
offer a structured approach to combine information from multiple gait features.
Working Principle :
Imagine a layered cake, where each layer represents a distinct gait feature. Within each layer, a hidden
Markov model operates. These hidden Markov models capture the underlying dynamics of the specific
feature. For instance, the silhouette layer might have hidden states representing different poses during
the gait cycle (stance, swing).These hidden states can have multiple hidden sub-states that capture the
independent influence of factors affecting the system. Similarly, the joint angle layer might have hidden
states corresponding to different angle ranges for knees and ankles.
The magic happens during the training process. Here, information exchange occurs between these layers.
Imagine frosting flowing between the cake layers. This allows the FHMM to learn the relationships
between features. For example, the model might learn that a specific silhouette pose (e.g., heel strike)
often coincides with a particular range of knee and ankle angles.
Question - HOW GAIT RECOGNITION IS PERFORMED
BY FHMM’S?
In gait recognition with Factorial Hidden Markov Models (FHMMs), hidden states play a crucial role in capturing the
complex and nuanced aspects of human walking patterns. Here's a deeper dive into their power:
1. Human gait is influenced by underlying factors like muscle movements, posture, and coordination, which are not
directly observable.
2. FHMMs introduce hidden states, which represent these underlying factors influencing the observed gait data.
3. Each hidden state captures a specific combination of sub-states representing independent factors like:
• Leg Movement: "Swing" or "Stance" for each leg.
• Upper Body Movement: "Bent Torso" or "Straight Torso.“
4. The true power lies in combining these sub-states into factorial hidden states. These combined states represent
specific moments in a person's gait cycle:
• Example: (Left Leg - Swing, Right Leg - Stance, Upper Body - Bent Torso) depicts a specific walking posture with
a left leg swing and a bent torso.
FACTORIAL HIDDEN MARKOV MODEL
HIDDEN LAYERS
WITHIN EACH HIDDEN LAYER, A HIDDEN
MARKOV MODEL OPERATES
OBSERVABLE LAYER/S
BENEFITS OF USING FHMM
1. Reduced Dimensionality: Compared to single-feature HMMs that concatenate features, FHMMs avoid the issue of
high dimensionality. By modeling features in separate layers, FHMMs can improve training efficiency and potentially
reduce overfitting, especially with limited training data.
2. Effective with Weakly Correlated Features: FHMMs are well-suited for scenarios where the gait features exhibit weak
correlations. By capturing the underlying relationships, FHMMs can potentially improve recognition accuracy
compared to single-feature models.
3. Capture Individuality: By modeling these combinations, FHMMs can capture the unique aspects of a person's gait,
even with variations in clothing, walking speed, or carrying objects.
4. Handle Asynchronous Movements: Human gait isn't always perfectly symmetrical. FHMMs can model independent
leg movements effectively, allowing for asynchronous leg swings.
5. Robustness to Variations: Variations in walking surface or carrying objects might influence observed data (video
frames) but the underlying gait pattern remains similar. FHMMs with rich hidden states can focus on these underlying
patterns for better recognition.
PARALLEL HIDDEN MARKOV MODELS(PHMM)
Another approach to information fusion in gait recognition is through Parallel Hidden Markov
Models (FHMMs). Unlike deep learning models that can learn complex relationships directly from
data, Parallel Hidden Markov Models (PHMMs) offer a more refined approach by considering gait
as a combination of independent factors analyzed in parallel.
Working :
1.Factorization: The initial step involves decomposing the gait data (e.g., video frames) into
independent factors influencing the overall walking pattern. These factors could include:
• Leg Movements: Each leg's movement can be considered a separate factor.
• Upper Body Posture: The body's posture (bent or straight torso) can be another factor.
• Additional factors (optional): Depending on the application, arm swing patterns or head
orientation could also be included.
2. Sub-models with Hidden States: Each independent factor is modeled by a separate Hidden Markov Model
(HMM). These sub-models have their own set of hidden states representing the unseen aspects influencing
that specific factor. For example:
• Leg movement sub-model might have hidden states like "Stance" or "Swing" for each leg.
• Upper body posture sub-model might have hidden states like "Bent Torso" or "Straight Torso.“
3. Parallel Analysis: Unlike standard HMMs, PHMMs analyze these sub-models simultaneously. This allows for
the consideration of how different factors influence each other during gait.
4. Interaction Modeling: PHMMs learn the transition probabilities between hidden states within each sub-
model. Additionally, they learn how the hidden states in different sub-models interact and influence each
other. For instance, the model might learn that a specific leg movement ("Heel Strike") often coincides with a
particular upper body posture ("Straight Torso").
5. Recognition: When presented with a new gait sequence, the PHMM analyzes the hidden states in each sub-
model and considers their interactions. By comparing these states and their interactions to the learned
patterns, the model can potentially identify the individual based on their unique gait signature.
BENEFITS OF USING PHMM
1.Robustness to Variations: PHMMs excel at handling variations in external factors like clothing,
carried objects, or even walking surfaces. This is because they focus on independent factors
influencing gait. Each factor (e.g., leg movements) is analyzed separately, allowing the model to
capture the underlying gait patterns that remain consistent despite external variations.
2.Detailed Feature Analysis: By analyzing each independent factor with its own hidden states,
PHMMs provide a more detailed understanding of gait compared to traditional models. This allows
for a richer picture of the walk, potentially capturing specific details about individual gait styles.
3.Simpler Model Structure: Compared to FHMMs (Factorial Hidden Markov Models), PHMMs have
a simpler model structure. They utilize separate HMMs for each factor, which can be easier to train
and interpret. This simplicity can be beneficial, especially when dealing with limited training data.
THANK YOU!!