4 Decomposition

BAN430: Adjustments, Transformations and Time Series Decomposition
Sondre Hølleland
1/26/23
Sondre Hølleland BAN430: Adjustments, Transformations and Time Series Decomposition 1/26/23 1
Outline
Transformations and adjustments

▶ Calender adjustments
▶ Population adjustments
▶ Inflation adjustments
▶ Mathematical transformations
Time series components
Moving averages
Different decomposition techniques/methods
Calender adjustments
$75k
$70k
Total production
$65k
$60k
2015 Jan 2016 Jan 2017 Jan 2018 Jan 2019 Jan 2020 Jan
Figure 1: production YearMonth
Calender adjustments
$3.18k
$3.16k
Mean production
$3.14k
$3.12k
$3.1k
2016 Jan 2018 Jan 2020 Jan

Figure 2: production YearMonth
Calender Adjustments
Mean production is better than total

Avoid the effect of holidays, month lengths, etc.
Not a big deal, but keep in mind.
Population adjustment
Let Yt denote a quantity of interested that is somehow related to population size Popt .
Population adjusted quantity: Yt /Popt .
Examples:
▶ GDP per capita
▶ Car accidents per driver / driven 1000km
scandinaviaUSA <- global_economy %>%

filter(Country %in% c("Norway","Sweden","Denmark", "United States"))
knitr::kable(
scandinaviaUSA %>% head() )
Country Code Year GDP Growth CPI Imports Exports Population
Denmark DNK 1960 6248946880 NA 8.245702 34.32181 32.31046 4579603

Denmark DNK 1961 6933842099 6.3788325 8.530313 32.34234 29.99696 4611687
Denmark DNK 1962 7812968114 5.6668222 9.159664 32.50493 28.61946 4647727
Denmark DNK 1963 8316692386 0.6370212 9.718865 30.79128 30.40296 4684483
Denmark DNK 1964 9506678763 9.2699382 10.019448 32.62801 29.85216 4722072
Denmark DNK 1965 10678897387 4.5552548 10.565508 31.54015 29.29713 4759012
scandinaviaUSA %>%
autoplot(GDP)+
labs(title= "GDP", y = "$US")
GDP
2.0e+13
1.5e+13
Country
Denmark
$US
1.0e+13 Norway
Sweden
United States
5.0e+12
0.0e+00
1960 1980 2000

Year [1Y]
scandinaviaUSA %>%
autoplot(Population)+
labs(title= "Population", y = "Number of people")
Population
3e+08
Number of people
Country
2e+08
Denmark
Norway
Sweden
United States
1e+08
0e+00
1960 1980 2000
Year [1Y]
scandinaviaUSA %>%
autoplot(Population)+
scale_y_log10()+
labs(title= "Population (log-scale)",y = "Number of people")
Population (log−scale)
3e+08
1e+08
Number of people
Country
Denmark
Norway
3e+07
Sweden
United States
1e+07
3e+06
1960 1980 2000
Year [1Y]
scandinaviaUSA %>%
autoplot(GDP/Population) +
labs(title= "GDP per capita = GDP / Population", y = "$US")
GDP per capita = GDP / Population
100000
75000
Country
Denmark
$US
Norway
50000
Sweden
United States
25000
0
1960 1980 2000
Year [1Y]
Inflation adjustment
scandinaviaUSA %>%
autoplot(GDP/Population / CPI*100) +
labs(title= "GDP per capita (inflation adjusted)", y = "$US (2010)")
GDP per capita (inflation adjusted)

100000
75000 Country
$US (2010)
Denmark
Norway
Sweden
50000
United States
25000
1960 1980 2000

Year [1Y]
Consumer price index
CPI %>%
autoplot(CPI, color = "blue", lwd = 1.2) +
labs(title= "Consumer Price Index", y = "NOK",
subtitle = "Data source: Statistics Norway")+
geom_hline(yintercept = 100, lty = 2) + geom_vline(xintercept = 2015, lty = 2)+
scale_x_continuous(breaks = seq(1925,2025,10))+
scale_y_continuous(breaks = seq(0,120,10))
Consumer Price Index

Data source: Statistics Norway
120
110
100
90
80
70
NOK
60
50
40
30
20
10
0
1925 1935 1945 1955 1965 1975 1985 1995 2005 2015 2025
Year [1Y]
Inflation adjusted big mac prices in Norway
norBigMac %>%
autoplot(local_price) +
labs(title= "Big Mac price in Norway", y = "NOK",
subtitle = "Data source: The Economist") +
geom_smooth(method = "lm", se=FALSE)
Big Mac price in Norway

Data source: The Economist
55
50
NOK
45
40
35
2005 2010 2015 2020

date [1D]
Inflation adjusted big mac prices in Norway
norBigMac %>%
mutate(cpiAdjusted =local_price / CPI * 100) %>%
as_tsibble(index = date) %>%
autoplot(cpiAdjusted)+
labs(title= "Inflation adjusted Big Mac price in Norway", y = "NOK (2015)",
subtitle = "Data sources: The Economist (big mac index), Statistics Norway (CPI)") +
geom_smooth(method = "lm", se=FALSE)
Inflation adjusted Big Mac price in Norway

Data sources: The Economist (big mac index), Statistics Norway (CPI)
48
NOK (2015)
44
40
2005 2010 2015 2020

date [1D]
Mathematical transformations
Goal: transform the data to something that is more linear and/or stationary.
Logarithm: wt = log yt
Difference: wt = yt − yt−1
Power transformation: wt = ytp for p ∈ {1/2, 1/3, . . .}.
Box-Cox transformation: (
log yt if λ = 0,
wt = λ
(yt − 1)/λ if λ ̸= 0.
Mathematical transformations: Differencing for a trend
Say
70
yt = µ + α t + ϵt
y
60
Then
50
wt = yt − yt−1
= µ + α t + ϵt − µ − α (t − 1) − ϵt−1 4
= α + ϵt − ϵt−1 0
w
−4
−8
0 25 50 75 100
Mathematical transformations: Stock prices
Apple Closing Price transformations
5
Log : wt = log yt 150
4
logclose
close
100
Diff : wt = yt − yt−1 50
3
2
√ 1/2 0 1
Squareroot : wt = yt = yt
5
√
squareroot
10
1/3
thirdroot
4
Thirdroot : wt = 3 yt = yt 3
5
yt − yt−1 2
Returns: wt = .
yt−1 6 10
diffclose
boxcox
Log-returns: wt = log yt − log yt−1 4
0
−5
yt 2
= log . −10
yt−1 0.1 0.1
logreturn
yλ − 1 0.0
return
0.0
Box-Cox: wt = t . −0.1 −0.1
λ −0.2
2010 2015 2020 2010 2015 2020
(For the Box-Cox: λ

b = 0.08)
Time series components: Additive
Yt = Tt + St + Rt
Observations: Yt
Trend-Cycle: Tt
Season: St
Remainder / Residual: Rt
Time series components: Additive
300
200
Yt
100
Yt = Tt + St + Rt
0
Observations: Yt 200
Trend-Cycle: Tt
Tt
100
Season: St
Remainder / Residual: Rt 20
10
St
0
Simulated data where we know −10
−20
each component.
30
0
Rt
−30
2016 2018 2020
Time series components: Multiplicative
300
Yt = Tt · St · Rt
log Yt
200
100
or
0
log Yt = log Tt + log St + log Rt
200
log Tt
Observations: log Yt 100
Trend-Cycle: log Tt 20
Season: log St 10
log St
0
Remainder / Residual: −10
log Rt −20
30
Simulated data where we know

log Rt
each component. −30
2016 2018 2020
Seasonal adjustment
Additive: Yt − St Seasonally adjusted

Multiplicative: Yt /St or 300
log Yt − log St .
Seasonal variation not of
primary interest 200
Yt − St
Ex: State of economy
(positive/negative cycle) vs
monthly variations 100
2016 2018 2020
Decomposition methods
Classical decomposition
X11
Seats
STL
But first, some moving averages. . .
Moving averages
Taking the mean of a sliding window of observations

Often defined with a odd (number) window width
k
1 X
T
bt = yt+k
2k + 1 j=−k
where k = 0, 1, 2, . . ..
The window width is then m = 2k + 1.
m-MA
Ex: 3-MA means k = 1.
If the order m is even, the window become non-symmetric.
Moving average
set.seed(123)
MAex <- tibble(y = rnorm(10))
MAex
# A tibble: 10 x 1
y
<dbl>
1 -0.560
2 -0.230
3 1.56
4 0.0705
5 0.129
6 1.72
7 0.461
8 -1.27
9 -0.687
10 -0.446
Moving average
MAex %>% mutate(

`3-MA` = slider::slide_dbl(y, mean, .before = 1, .after = 1, .complete = TRUE)
)
# A tibble: 10 x 2
y `3-MA`
<dbl> <dbl>
1 -0.560 NA
2 -0.230 0.256
3 1.56 0.466
4 0.0705 0.586
5 0.129 0.638
6 1.72 0.768
7 0.461 0.304
8 -1.27 -0.497
9 -0.687 -0.799
10 -0.446 NA
Moving average
MAex %>% mutate(

`3-MA` = slider::slide_dbl(y, mean, .before = 1, .after = 1, .complete = TRUE),
)
# A tibble: 10 x 3
y `3-MA` `4-MA`
<dbl> <dbl> <dbl>
1 -0.560 NA NA
2 -0.230 0.256 0.210
3 1.56 0.466 0.382
4 0.0705 0.586 0.868
5 0.129 0.638 0.594
6 1.72 0.768 0.260
7 0.461 0.304 0.0560
8 -1.27 -0.497 -0.484
9 -0.687 -0.799 NA
10 -0.446 NA NA
Moving average
MAex %>% mutate(

)
# A tibble: 10 x 4
y `3-MA` `4-MA` `5-MA`
<dbl> <dbl> <dbl> <dbl>
1 -0.560 NA NA NA
2 -0.230 0.256 0.210 NA
3 1.56 0.466 0.382 0.194
4 0.0705 0.586 0.868 0.649
5 0.129 0.638 0.594 0.787
6 1.72 0.768 0.260 0.222
7 0.461 0.304 0.0560 0.0707
8 -1.27 -0.497 -0.484 -0.0443
9 -0.687 -0.799 NA NA
10 -0.446 NA NA NA
Moving average
MAex %>% mutate(

)
# A tibble: 10 x 5
y `3-MA` `4-MA` `5-MA` `6-MA`
<dbl> <dbl> <dbl> <dbl> <dbl>
1 -0.560 NA NA NA NA
2 -0.230 0.256 0.210 NA NA
3 1.56 0.466 0.382 0.194 0.447
4 0.0705 0.586 0.868 0.649 0.617
5 0.129 0.638 0.594 0.787 0.445
6 1.72 0.768 0.260 0.222 0.0706
7 0.461 0.304 0.0560 0.0707 -0.0154
8 -1.27 -0.497 -0.484 -0.0443 NA
9 -0.687 -0.799 NA NA NA
10 -0.446 NA NA NA NA
Moving average
MAex %>% mutate(

)
# A tibble: 10 x 6
y `3-MA` `4-MA` `5-MA` `6-MA` `7-MA`
<dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 -0.560 NA NA NA NA NA
2 -0.230 0.256 0.210 NA NA NA
3 1.56 0.466 0.382 0.194 0.447 NA
4 0.0705 0.586 0.868 0.649 0.617 0.449
5 0.129 0.638 0.594 0.787 0.445 0.348
6 1.72 0.768 0.260 0.222 0.0706 0.283
7 0.461 0.304 0.0560 0.0707 -0.0154 -0.00311
8 -1.27 -0.497 -0.484 -0.0443 NA NA
9 -0.687 -0.799 NA NA NA NA
10 -0.446 NA NA NA NA NA
Moving average
MAex %>% mutate(
`10-MA` = slider::slide_dbl(y,mean, .before = 4, .after = 5, .complete = TRUE),
`11-MA` = slider::slide_dbl(y,mean, .before = 5, .after = 5, .complete = TRUE)
)
# A tibble: 10 x 10
y `3-MA` `4-MA` `5-MA` `6-MA` `7-MA` `8-MA` `9-MA` `10-MA` `11-MA`
<dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 -0.560 NA NA NA NA NA NA NA NA NA
2 -0.230 0.256 0.210 NA NA NA NA NA NA NA
3 1.56 0.466 0.382 0.194 0.447 NA NA NA NA NA
4 0.0705 0.586 0.868 0.649 0.617 0.449 0.235 NA NA NA
5 0.129 0.638 0.594 0.787 0.445 0.348 0.219 0.132 0.0746 NA
6 1.72 0.768 0.260 0.222 0.0706 0.283 0.192 0.145 NA NA
7 0.461 0.304 0.0560 0.0707 -0.0154 -0.00311 NA NA NA NA
8 -1.27 -0.497 -0.484 -0.0443 NA NA NA NA NA NA
9 -0.687 -0.799 NA NA NA NA NA NA NA NA
10 -0.446 NA NA NA NA NA NA NA NA NA
Moving averages of moving averages
When? - m is even
Why? - To get symmetric weighting
MAex %>%
mutate(`4-MA` = slider::slide_dbl(y , mean, .before = 1, .after = 2, .complete = TRUE),
`2x4-MA` = slider::slide_dbl(`4-MA`, mean, .before = 1, .after = 0, .complete = TRUE),
`5-MA` = slider::slide_dbl(y , mean, .before = 2, .after = 2, .complete = TRUE)
)
# A tibble: 10 x 4
y `4-MA` `2x4-MA` `5-MA`
<dbl> <dbl> <dbl> <dbl>
1 -0.560 NA NA NA
2 -0.230 0.210 NA NA
3 1.56 0.382 0.296 0.194
4 0.0705 0.868 0.625 0.649
5 0.129 0.594 0.731 0.787
6 1.72 0.260 0.427 0.222
7 0.461 0.0560 0.158 0.0707
8 -1.27 -0.484 -0.214 -0.0443
9 -0.687 NA NA NA
10 -0.446 NA NA NA
(4) 1
4-MA : T
b
t = (yt−1 + yt + yt+1 + yt+2 )
4
(2×4) 1 b (4) b (4)

2 × 4-MA : T
b
t = Tt−1 + T t
2
1 1 1
= (yt−2 + yt−1 + yt + yt+1 ) + (yt−1 + yt + yt+1 + yt+2 )
2 4 4
1 1 1 1 1
= yt−2 + yt−1 + yt + yt+1 + yt+2
8 4 4 4 8
(5) 1
5-MA Tt = (yt−2 + yt−1 + yt + yt+1 + yt+2 )
5
Example: Quarterly observations
2 × 4-MA : b (2×4) = 1 y
T
1 1 1 1
Q3,Y Q1,Y + yQ2,Y + yQ3,Y + yQ4,Y + yQ1,Y+1
8 4 4 4 8
To estimate a trend-cycle using MA, m is set to the seasonal period of the data. If m is even,
use 2 × m-MA, else m-MA.
Quarterly observations –> m = 4 –> use 2 × 4-MA.
Monthly observations –> m = 12 –> use 2 × 12-MA.
Daily observations (with weekly season) –> m = 7 –> use 7-MA.
Can also use moving average of moving averages when m is odd (to get a weighted average),
but then both averages should be odd (e.g. 3 × m, 5 × m, etc.). Similarly, even m should be
followed by even order (e.g. 2 × m, 4 × m, etc.).
Classical decomposition (additive)
1 Estimate trend Tbt using 2 × m-MA if period m is even, m-MA if not.

2 Detrend yt − Tt .
b
3 Estimate season by calculating the average detrended value for each season
4 Adjust the seasonal components such that they add to zero:
P
Sr
St⋆ = St − |St | · P r .
r |Sr |
5 b t = yt − T
Calculate remainder: R bt − S
bt .
Norwegian employment
Source: Statistics Norway
2750
Employed
2500
2250
2000 Q1 2010 Q1 2020 Q1

Quarter [1Q]
dat %>%
model(classic = classical_decomposition(Employed, type = "additive")) %>%
components()
# A dable: 107 x 7 [1Q]

# Key: .model [1]
# : Employed = trend + seasonal + random
.model Quarter Employed trend seasonal random season_adjust
<chr> <qtr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 classic 1996 Q1 2090 NA -14.6 NA 2105.
2 classic 1996 Q2 2131 NA 8.79 NA 2122.
3 classic 1996 Q3 2159 2140. 10.9 8.39 2148.
4 classic 1996 Q4 2148 2156. -5.01 -2.74 2153.
5 classic 1997 Q1 2152 2172. -14.6 -5.10 2167.
6 classic 1997 Q2 2197 2187. 8.79 0.959 2188.
7 classic 1997 Q3 2221 2203. 10.9 6.77 2210.
8 classic 1997 Q4 2210 2219. -5.01 -4.37 2215.
9 classic 1998 Q1 2219 2233 -14.6 0.645 2234.
10 classic 1998 Q2 2258 2244. 8.79 5.33 2249.
# ... with 97 more rows
dat %>%
model(classic = classical_decomposition(Employed, type = "additive")) %>%
components() %>%
autoplot()
Employed = trend + seasonal + random
Employed
2750
2500
2250
2700
trend
2500
2300
10
seasonal
5
0
−5
−10
−15
20
10
random
0
−10
−20
2000 Q1 2010 Q1 2020 Q1
Quarter
Problems
Missing values
Over-smooth rapid changes
Not flexible seasonal component
Not robust against outliers
X11
Based on the classical approach

Trend-cycle available for all observations
Seasonal component allowed to vary slowly
Handles calender effects
Additive and multiplicative
Robust against outliers and level shifts
Entirely automatic
Only for quarterly and monthly data
dat %>%
model(X11 = X_13ARIMA_SEATS(Employed ~ x11())) %>%
components() %>%
autoplot()
SEATS
Seasonal extraction in ARIMA time series

Will not go into details
Similar to x11
Only for quarterly and monthly data
dat %>%
model(
SEATS = X_13ARIMA_SEATS(Employed ~ seats())
) %>%
components() %>%
autoplot()
X11 and SEATS vs classical
Classical & X−13ARIMA−SEATS using X−11 adjustment & X−13ARIMA−SEATS decompositi
Employed = trend * seasonal * random
Employed
2750
2500
2250
2700
trend
2500 .model
2300
classical
2100
seats
seasonal
1.004
1.000 x11
0.996
0.992
1.005
random
1.000
0.995
2000 Q1 2010 Q1 2020 Q1

Quarter
STL method
Seasonal and Trend decomposiotion using LOESS

LOcally Estimated Scatter plot Smoothing
Any type of seaonality
Seasonal component can change
Controllable smoothness
Can be robust against outliers
Does not handle trading day/calender variation
Only additive
STL: Parameters
Trend window
Season window
dat %>%
model(STL=STL(Employed ~ trend(window = 7)+ season(window = 11), robust = TRUE) ) %>%
components() %>%
autoplot()
STL decomposition
STL decomposition
Employed = trend + season_year + remainder
Employed
2750
2500
2250
2900
2700
trend
2500
2300
2100
season_year
10
0
−10
20
remainder
0
−20
−40
2000 Q1 2010 Q1 2020 Q1
Quarter
Your turn!
Wholesale and retail sales index for Norway
Source: Statistics Norway
150
Wholesale and retail sales index
125
100
75
50
2000 Jan 2005 Jan 2010 Jan 2015 Jan 2020 Jan
yearmonth [1M]
Exercise
1 Load the data

wholesale <- readRDS(file= "data/wholesale_and_retails_index_norway.rds")
2 Make a time plot

3 Decompose the time series using the classical, x11, seats and STL methods
4 Can you detect any prominent differences between the methods?
5 Try adjusting the trend and season windows of the STL. What happens? (default values
are respectively 21 and 11)
6 Using your method of choice, plot the seasonally adjusted time series.
7 Using your method of choice, plot the detrended series.
8 Implement your own additive classical decomposition on this example.

4 Decomposition

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

4 Decomposition

Uploaded by

Copyright:

Available Formats

BAN430: Adjustments, Transformations and Time Series Decomposition

Transformations and adjustments

2016 Jan 2018 Jan 2020 Jan

Mean production is better than total

scandinaviaUSA <- global_economy %>%

Country Code Year GDP Growth CPI Imports Exports Population

Denmark DNK 1960 6248946880 NA 8.245702 34.32181 32.31046 4579603

1960 1980 2000

GDP per capita = GDP / Population

GDP per capita (inflation adjusted)

1960 1980 2000

Consumer Price Index

Big Mac price in Norway

2005 2010 2015 2020

Inflation adjusted Big Mac price in Norway

2005 2010 2015 2020

yt−1 0.1 0.1

Box-Cox: wt = t . −0.1 −0.1

(For the Box-Cox: λ

Simulated data where we know −10

2016 2018 2020

Simulated data where we know

each component. −30

2016 2018 2020

Additive: Yt − St Seasonally adjusted

2016 2018 2020

Taking the mean of a sliding window of observations

MAex %>% mutate(

MAex %>% mutate(

MAex %>% mutate(

MAex %>% mutate(

MAex %>% mutate(

Example: Quarterly observations

1 Estimate trend Tbt using 2 × m-MA if period m is even, m-MA if not.

2000 Q1 2010 Q1 2020 Q1

# A dable: 107 x 7 [1Q]

Based on the classical approach

Seasonal extraction in ARIMA time series

2000 Q1 2010 Q1 2020 Q1

Seasonal and Trend decomposiotion using LOESS

1 Load the data

2 Make a time plot

You might also like