40 views

Uploaded by Christopher Bennett

Smoothing spline methods

- ETI30EN
- m.tech_pg_2018
- Absolute Harmonic Filter For RF
- Smart Seis Version c
- Speech Recognition project
- RF Bandpass Filters based on Toko Inductors
- how to write CV
- Adaptive Filter
- 03 Class Systems
- Review on Implementation of Fir Adaptive Filter Using Distributed Arithmatic and Block Lms Algorithm
- Date-Sheet B.E B.tech Exam 4th Sem
- IC200ALG240-DB
- Aamir Khan Cv 2014(1) (1)
- Ch13 ADC, DAC and Sensor Interfacing
- 07707428.pdf
- lm12w (1)
- EC-411EC411 LecNotes Pt1 v09
- A Complete Simulation of Target Detection and Esti
- 1660769643-DOK-TD-MRF3E
- Unit 2 Signals

You are on page 1of 6

October 31, 1996

Summary:

This guide presents an overview of filtering methods and the software which is available in the

HPL.

1. What is filtering/smoothing?

Smoothing is an operation which removes high-frequency fluctuations from a signal. Low-pass

filtering is another term for the same thing, but is restricted to methods which are linear: i.e. if you

want to filter a signal x(t)+y(t), it does not matter whether you apply the filter before or after adding the two signals. Such linear operations can be described by a frequency response. All methods

described here are linear, with the exception of curve fitting.

Random noise can greatly affect the result of certain types of analysis. Two important examples

are: quantification of peaks in the signal (noise can shift the location of the peak, and leads to systematic overestimation of the height of the peak), and differentiation (noise is amplified by differentiation). Always consider filtering before these types of data analysis. Since random (or white)

noise is distributed over all frequencies, and signal is typically limited to low frequencies, a reduction of high frequency components will improve the signal/noise ratio.

3.1. Analog (RC, resistor-capacitor) filter

R

input

output

The frequency response is a 1st order Butterworthfilter, and really easy to make (<1 Stano-hour),

1

cut-off frequency f 0 = --------------- . Connect this between your signal source (transducer or amplifier)

and the A/D converter if2RC

you want to get rid of high frequencies before the signal is sampled.

Remember the sampling theorem: if the signal contains frequencies which are higher than half the

sampling frequency, you can get erroneous results. An analog filter is the only way to make sure

that high frequencies dont reach the A/D converter.

The frequency response (defined as the ratio between input and output amplitude for a sine wave

of frequency f ) of an n th order Butterworth filter is:

A out

1

-------- H ( f ) = -----------------------------A in

f 2n

1 + -----

f 0

H(f)

f0=50 Hz

n=1

n=2

n=3

f [Hz}

Higher order analog filters are much harder to make (> 1 Stano-day).

Digital filters are implemented as a computer program that transforms sampled data. E.g. 2nd

order recursive filter:

y i = ax i + bx i 1 + cx i 2 + d y i 1 + ey i 2

The term recursive means that past y-values are fed back to the input. The relationship between

input samples ( x 1 x n ) and output samples ( y 1 y n ) is illustrated in the following diagram:

x 1, x 2, x 3,

FILTER

y 1, y 2, y 3,

KinTrak:

Uses a double recursive 2nd order Butterworth filter. This is a 4th order filter (since the asymp4

totic response is f ), but NOT a 4th order Butterworth filter because its frequency response is

the square of the 2nd order filter:

1

H ( f ) = ----------------------4f

1 + -----

f 0

Matlab functions:

BUTTER: get coefficients a,b,... for a Butterworth filter with specific order and cut-off frequency

FILTER: filter data

FILTFILT: filter data twice, forward and reverse

Notes:

1. Recursive digital filters always have a time lag between input and output, non-recursive filters

(where y is not fed back to the input side) can be made with zero-lag but do not allow a good

approximation to a Butterworth filter. Solution for the time lag problem: filter twice, second time

in reverse direction.

2. Digital filters require constant sampling interval.

3. It is not (yet) clear to me how the Matlab implementation deals with boundary conditions.

Matlab functions:

POLYFIT to get coefficients of best-fitting polynomial

POLYVAL to evaluate polynomial at arbitrary time values

POLYDER to evaluate first derivative at arbitraty time values

LEASTSQ(model, parameters) for other nonlinear regression problems

Other software:

spreadsheet (Lotus, Excel), statistics (SPSS, SAS).

Applications: e.g. quantification of force-deformation curves.

Notes:

1. Apply only when underlying function is simple; high order polynomials are dangerous when

extrapolated and give large errors in derivatives at the endpoints.

2. Does NOT correspond to a low-pass filter! (frequency response depends on data)

A n th degree spline function f ( t ) is a piecewise n th degree polynomial function. The polynomials are joined at the nodes ( t -values in the input data) in such a way that all derivatives up to the

( n 1 ) th are continuous. Within these constraints, the function f ( t ) is selected which minimizes:

n + 1

----------

2

2

+

( t ) dt

(

f

(

t

)

x

)

p

f

i i

(k)

where (t i, x i ) are the raw data samples, and f denotes the k th derivative of f ( t ) . The weight

factor p is the smoothing parameter. With p = 0 , an interpolating spline will be obtained, with

p = a least squares fit of the entire dataset using a single polynomial of degree ( n 1 ) /2 (e.g.

a straight line for n =3. Intermediate values give a compromise between good fit (the first term in

the equation) and the smoothness (the second term). The degree n should be odd: n = 3,5,7 for

cubic, quintic, heptic splines.

Matlab function:

CSAPS: 3rd degree (cubic) smoothing spline

Stand-alone program:

GCV: any odd order spline, with many options to select smoothing parameter. See documentation

(FrameMaker file) ~bogert/help/gcv.fm.

Fortran library:

GCVSPL: this library by Herman Woltring is used in the GCV program and can also be used in

your own Fortran programs. You can interface it with Matlab too. See http://www.kin.ucalgary.ca/

isb/software.

Notes:

1. Sampling interval does not have to be constant for a spline method.

2. Differentiation is not an additional finite difference approximation, as in the digital filter, but

can be done directly from the equations of the polynomials. The GCVSPL package includes a

function SPLDER (spline derivative) that does this for you.

3. Boundary conditions: f

+ 1

n---------- 2

Matlab:

FFT (fast fourier transform), followed by reduction of amplitudes of high frequencies, followed

by IFFT (inverse fft)

Notes:

1. Simply setting high frequencies to zero can cause ringing near sudden transitions in the input

signal.

(n)

(n)

2. Boundary conditions: f ( 0 ) = f ( T ) for function ( n = 0 ) and all derivatives

( n = 1, 2, ). Real movement data (especially non-cyclic data) does not satisfy these conditions.

Both drawbacks are to some extent avoided in the method published by Hatze (1981).

This is typically an interactive trial and error process. The goodness of a filter is best based on

visual inspection of the results. Also note that it depends on the subsequent steps in the analysis

too. For instance, a good filter for peak detection may be one which reduces noise to 1% of the

signal. The same filter may not be good for differentiation, because the noise in the derivative is

still to high. Filters for differentiation typically need a lower cut-off frequency.

The GCV software includes two optimization methods to select a p-value objectively. The first

requires an estimate of the magnitude of the noise, the second (the generalized cross-validation

criterion) tries to make an estimate of the noise. The latter method tends not to work well for

choosing an optimal filter for differentiation (it tends to smooth not enough), but your mileage

may vary.

5. References

Hatze, H. (1981) The use of optimally regularized Fourier series for estimating higher-order

derivatives of noisy biomechanical data. J. Biomech. 14,13-18. (describes a Fourier series

method)

Pezzack, J.C., R.W. Norman, and D.A. Winter (1977) An assessment of derivative determining

techniques used for motion analysis. J. Biomechanics 10,377-382. (describes digital filtering method and benchmark data)

Woltring, H.J., Smoothing and differentiation techniques applied to 3-D data, in Three-Dimensional Analysis of Human Movement, ed. P. Allard, I.A.F. Stokes, J.-P. Blanchi, Human

Kinetics Publishers, Urbana-Champaign, Illinois/USA. (good overview and mathematical

background)

Woltring, H.J. (1986) A Fortran package for generalized, cross-validatory spline smoothing and

differentiation. Adv. Engng. Softw. 8,104-110. (describes spline method, very technical)

Wood, G.A. (1982) Data smoothing and differentiaion procedures in biomechanics. Exerc. Sport

Sciences Rev. 10,308-362. (good overview)

NOTE: if you suggestions for additions or revisions to this document, please contact me at

bogert@acs.ucalgary.ca

- ETI30ENUploaded byDiego Cordova
- m.tech_pg_2018Uploaded byDamodar Reddy
- Absolute Harmonic Filter For RFUploaded byLeslie Green
- Smart Seis Version cUploaded byAngel Nahun Rivera
- Speech Recognition projectUploaded byabhigolwala07
- RF Bandpass Filters based on Toko InductorsUploaded byDefaultAnomoly
- how to write CVUploaded byamin
- Adaptive FilterUploaded byhammadhussain
- 03 Class SystemsUploaded byJonathan Terrón
- Review on Implementation of Fir Adaptive Filter Using Distributed Arithmatic and Block Lms AlgorithmUploaded byIJMER
- Date-Sheet B.E B.tech Exam 4th SemUploaded byTanuj Vashist
- IC200ALG240-DBUploaded byAlessandro Inviti
- Aamir Khan Cv 2014(1) (1)Uploaded byp06010101
- Ch13 ADC, DAC and Sensor InterfacingUploaded bysumbal_iqbal
- 07707428.pdfUploaded byIsma Iba
- lm12w (1)Uploaded byTom Kent
- EC-411EC411 LecNotes Pt1 v09Uploaded byAbhay Yadav
- A Complete Simulation of Target Detection and EstiUploaded byShankarPrasai
- 1660769643-DOK-TD-MRF3EUploaded byBhavik Prajapati
- Unit 2 SignalsUploaded byDigitallogicdl
- TP Modul 1 2015-2016Uploaded byIGede Agus Surya Negara
- Record VikasUploaded byElangovan Devaki
- pras DE .pptxUploaded byprasnthsm
- Booklist IIB GrpFUploaded bys_hardik89
- M380SYL Spring 2014-01-13Uploaded byMorsTorTor
- ftantcUploaded bypblhnrq
- NO 4Uploaded byRoby
- WIKA electronic pressure measurement.pdfUploaded byDragoi Alexandrina
- Noise analysisUploaded byRasoul Gmdri
- 04404890Uploaded byphineasferb00

- Factsheet SGT-700 EnUploaded byAzril Dahari
- Rapid determination of oxalic acid by reversed-phase high-perfor- mance liquid chromatographyUploaded byCarlos José Pachón Cubaque
- Intro Remotes SensingUploaded byEnrique Zadomen
- USP 281 ROI Residue on IgnitionUploaded byLisa Debrowski
- 18228-20605-1-PBUploaded bycitianz2011
- MA30087/50087: Optimisation methods of operational researchUploaded byMarcelo Ziulkoski
- 2nd order diferential equationsUploaded byRoel Moralez
- Robot Trajectory Optimization for the Relaxed End-Effector PathUploaded bywanglei2112221990
- MeasurIT KTek KM26 Dual Chamber Comparison 0808Uploaded bycwiejkowska
- Eigen value based (EBB) beamforming precoding design for downlink capacity improvement in multiuser MIMO channelUploaded byKrishna Ram Budhathoki
- Cup AnemometerUploaded bymertik
- Introduction to WaspUploaded byJeisson Jesús Coaguila Chipana
- RCN Telescope RoncellaUploaded byMartin Schwerdt
- 2-Understanding PPF and PPFDUploaded byChris Ruhland
- Condition Assessment on Waste Water Collection SystemUploaded byKifayat Ullah Khalil
- Sheet 1Uploaded bylawan
- CLUSTERING HYPERSPECTRAL DATAUploaded byCS & IT
- Full TextUploaded byRamzi Abbyad
- Technical Inspection Report Due to Corrosion Ocured on LPC Pump G-2707Uploaded byJunie Prananta
- Centralina CatalogoUploaded byFrancisco Fuentes
- A Guide to Professional CookwareUploaded byEffeciu_73
- delta CNC_C_ENUploaded byErsin Akyüz
- MellesGriot Gaussian Beam OpticsUploaded byKurt Patzik
- Solenoid Lifting Lug AnalysisUploaded byGiuliano Paolini
- Barral Et Al. 2010Uploaded byBismoy Roy Chowdhury
- Qualitative AnalysisUploaded byMaria Pauline Sarmiento Gonzales
- jeas_1017_6416Uploaded bysaeedhoseini
- Technical Lecture - EMIUploaded byMatthew Simeon
- Geometry on the Sphere- Google's S2 LibraryUploaded byAnonymous ccNLUY
- 0580_s12_qp_41Uploaded byKarim Ahmed