Fourier Regression

Fourier Regression can be used to model periodic functions. So when there is no trendline Fourier regression
can be used. One application is modelling of errors, post regression. After we have removed the trend line
Fourier regression can work, if error terms have periodic pattern. Another application is time series modelling of
seasonal data, commonly found in domains of climate. Eg:- Pollution Data Set

So fourier regression can be used when Periodicity is involved

In [124]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import math

generating a periodic function by random combination of sines

and cosines.
In [125]:
n = np.linspace(0.1,6*math.pi,100)
a = np.random.uniform(0,10,10)
b = np.random.uniform(0,10,10)

In [126]:

lst = []
for i in range (0,len(n)):
s = 0
for j in range(0,10):
s = s + a[j]*math.sin(j * n[i]) + b[j]*math.cos(j*n[i])
lst.append(s)

In [127]:
plt.plot(lst,label= "signal")
plt.title('Periodic Function')
plt.legend()
Out[127]:
<matplotlib.legend.Legend at 0x15271ab9be0>

So we got a periodic function , now we perform fourier regression to regenrate it. Since it's formed from linear
combination of sines and cosines we should be able to model to perfectly. However, that may not be the case
with other periodic functions.

Steps Involved
Fourier regression is regression with sines, cosines and their frequencies as basis
Sines and cosines unless of same frequency are orthogonal to it.
This removes the issue of collinearity
Also since its a lineare combination the method of multiple linear regression with orthogonal basis would
work

In [128]:
pred = []
s = [0]*100
for j in range(1,25):
s = [0]*100
s1 = [0]*100
for i in range(0,len(n)):
s[i] = math.sin(j*n[i])
s1[i] = math.cos(j*n[i])
pred.append(s) #sinnt
pred.append(s1)#cosnt

SO we have formed basis with sines and cosines

Now we project the data point of our target curve onto this basis and get their linear combiation to get the
curve

In [129]:
predf = [0]*100
for i in range(0,6):
pred1 = [0]*100
slope = (np.cov(pred[i],lst)/np.var(pred[i]))[0][1]

for j in range(0,100):
pred1[j] = slope*pred[i][j]
for k in range(0,100):
predf[k] = predf[k] + pred1[k]
con = [0]*100
for k in range(0,100):
con[k] = predf[k] - lst[k]
c = sum(con)/len(con)

predf = predf - c

# plotting the predicted curve and target curve

plt.plot(predf,color = 'black', linewidth =4,label = 'Predicted with 1st 3 frequencies')

plt.plot(lst,label = 'Signal')
plt.legend()
Out[129]:
<matplotlib.legend.Legend at 0x15271b2ea30>
In [130]:
predf = [0]*100
for i in range(0,12):
pred1 = [0]*100
slope = (np.cov(pred[i],lst)/np.var(pred[i]))[0][1]

for j in range(0,100):
pred1[j] = slope*pred[i][j]
for k in range(0,100):
predf[k] = predf[k] + pred1[k]
con = [0]*100
for k in range(0,100):
con[k] = predf[k] - lst[k]
c = sum(con)/len(con)

predf = predf - c

# plotting the predicted curve and target curve

plt.plot(predf,color = 'black', linewidth =4,label = 'Predicted with 1st 6 frequencies')

plt.plot(lst,label = 'Signal')
plt.legend()
Out[130]:
<matplotlib.legend.Legend at 0x15271ba3a30>

In [132]:
predf = [0]*100
for i in range(0,20):
pred1 = [0]*100
slope = (np.cov(pred[i],lst)/np.var(pred[i]))[0][1]

for j in range(0,100):
pred1[j] = slope*pred[i][j]
for k in range(0,100):
predf[k] = predf[k] + pred1[k]
con = [0]*100
for k in range(0,100):
con[k] = predf[k] - lst[k]
c = sum(con)/len(con)

predf = predf - c

# plotting the predicted curve and target curve

plt.plot(predf,color = 'black', linewidth =4,label = 'Predicted with 1st 10 frequencies')

plt.plot(lst,label = 'Signal')
plt.legend()
Out[132]:
<matplotlib.legend.Legend at 0x15272c620d0>

Conclusion
As we increase the basis we get closer and closer to actual curve.
We have to account for bias variance trade off
Technically if n = infinity we can accurately model any periodic curve perfectly.