Baggingand Boosting

prathameshlavekar@gmail.
com
YHZEPDBA51
Bagging and Boosting
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by prathameshlavekar@gmail.com only. 1
Sharing or publishing the contents in part or full is liable for legal action.
Bagging and Boosting
Bagging is a homogeneous weak learners’ model that learns from each other independently in parallel and combines them for
determining the model average.
prathameshlavekar@gmail.com
YHZEPDBA51
Boosting is also a homogeneous weak learners’ model but works differently from Bagging. In this model, learners learn
sequentially and adaptively to improve model predictions of a learning algorithm.
Bagging
● Bagging stands for ‘Bootstrap Aggregation,' and it is a technique for reducing variation in a prediction model.
● Bagging is a parallel method where learners separately learn from one another, allowing them to be trained at the same time.
● From the dataset, bagging creates extra data for training using random sampling with replacement from the original dataset.
● In each new training data set, sampling with replacement may repeat certain observations.
● Multiple models are trained in parallel using these multi datasets.

● The average of all the predictions from different ensemble models is calculated for regression. The majority voting mechanism
YHZEPDBA51
is considered for classification..
● Bagging decreases the variance and tunes the prediction to an expected outcome.
Boosting
● Boosting is a sequential ensemble method that increases the weight of observations according to the previous classification
repeatedly.
● When an observation is mistakenly classified, the weight of that observation increases.
● In layman's terms, boosting refers to algorithms that convert a poor learner into a stronger one. It reduces bias error and
creates robust predictive models.
● The weights of data points that were incorrectly predicted in each cycle are increased.
YHZEPDBA51
● During training, the Boosting algorithm assigns weights to each of the generated models.
● A bigger weight will be provided to a learner who has good training data prediction outcomes.
Similarities and differences
Similarities
● Bagging and boosting are ensemble strategies that aim to produce N learners from a single learner.
● Bagging and boosting generate several training data sets and random sampling.
● Bagging and Boosting arrive at a final conclusion by averaging N learners' votes or choosing the majority voting rank.
● Bagging and boosting lower variation and increase stability while reducing errors.
YHZEPDBA51Differences
● Bagging is a technique for combining forecasts of the same type. Boosting is a technique for combining various forms of predictions.
● Bagging overcomes over-fitting difficulties in a model by reducing variance rather than bias. Boosting reduces bias rather than
variance.
● Each model is given the same weight in Bagging. Models are weighed in Boosting based on their performance.
● In Bagging, models are created individually. The performance of a previously built model in Boosting has an impact on new models.
Bagging and Boosting models
Bagging Boosting
● Bagging Meta estimator ● Adaptive Boosting
● Random Forest ● Gradient Boosting

YHZEPDBA51
● Xtreme Gradient Boosting
● Light Gradient Boosting
● Cat Boost
Important Questions
YHZEPDBA51
4) Is Random Forest bagging or boosting?
The random forest algorithm is actually a bagging algorithm: also here, we draw random bootstrap samples from your training set.
However, in addition to the bootstrap samples, we also draw random subsets of features for training the individual trees; in bagging,
we provide each tree with the full set of features. Due to the random feature selection, the trees are more independent of each
other compared to regular bagging, which often results in better predictive performance (due to better variance-bias trade-offs), and
I’d say that it’s also faster than bagging, because each tree learns only from a subset of features.
YHZEPDBA51

Baggingand Boosting

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Baggingand Boosting

Uploaded by

Copyright:

Available Formats

prathameshlavekar@gmail.

● Multiple models are trained in parallel using these multi datasets.

is considered for classification..

● Bagging Meta estimator ● Adaptive Boosting

● Random Forest ● Gradient Boosting

● Xtreme Gradient Boosting

● Light Gradient Boosting

You might also like