You are on page 1of 12

Autocorrelation of Timeseries EcoTool Version 1.

1

A Note About This Notebook
This file is a Mathematica notebook — a document which may contain executable code, text, and graphics. As the basis for an EcoTool, it serves two purposes. First, it is a development and demonstration environment. The code used in the EcoTool is developed, tested and refined in the notebook. Technical notes and other annotations are included in order to make the code as explicit as possible, even to those who are not profocient in Mathematica. Examples of the code in action may also be included. Second, this notebook is the basis for a Mathematica package — a separate file which contains only the code necessary to perform the calculations specified by the web page for this EcoTool. The package is loaded by web Mathematica when a calculation request is submitted. The package file is generated and updated automatically whenever this file is saved. Some of the cells in this notebook are initialization cells, identified by the little 'triangle with a tail' in the cell bracket on the right. The first time you attempt to evaluate any command in this notebook interactively, you will be prompted as to whether you want all the initialization cells to evaluate first (I suggest you say yes). These same cells are the ones that are saved to the corresponding package. All other cells, including text and graphics, are ignored. The beauty of this system is that is only one file to update — you can be sure that the code performing the online calculations is exactly the same as the code in this notebook.

‡ Open source development
EcoTools modules are open source. You are free to use the code herein for your own purposes, including your own installation of Mathematica. There are just two requirements: 1) If you make any improvements, extensions, etc., to the code, you should submit them back to the EcoTools site for possible inclusion in an updated version of the tool. See the Contributing link on the EcoTools main page for details. 2) If you publish work based on the code herein, or on a modification of the code herein, you should acknowledge the source. See the Authors link for this EcoTool for the details of how to format citations.

References
As far as is possible, EcoTools code is based on standard, published references. Annotations in this notebook are likely to refer to those sources extensively. The sources for this notebook are:

1<DD. ‡ Turn spelling warnings off In[64]:= Off@General::spell1D. dataProcessing@data_ListD := Module@8t. 8i. Calculation functions perform the bulk of the number-crunching.Drop@t. Miscellaneous functions are typically small. Flatten@Table@ Join@8data@@i. ‡ Miscellaneous functions This function applies to data that are 8time. as the name implies are used to generate graphics for output to the web. usually those that come with Mathematica or web Mathematica.t@@iDDL ê minInterval . checks for missing data. value< pairs. ‡ Packages In[65]:= Needs@"Statistics`DataManipulation` "D. Packages refers to other packages. t = data@@All.nb Royama. London. It works out the timing interval. -1DD. Chapman & Hall. Graphics functions. 1DD. 1. 2DD<. Length@dataD . (1989) Analytical Population Dynamics. and perform tasks are quite general but which are useful in the context of this EcoTool.13a. The Code The code is divided into four sections. minInterval = Min@Drop@t. that are required by the EcoTool-specific code. T. Max@tD.1<DD D . minIntervalD. and then strips off the time values to leave a one-dimensional vector (which is what the autocorrelationa@D and autocorrelationb@D functions expect as input). 8Ht@@i + 1DD . In[66]:= ‡ Calculation functions This is Royama 3. 1D .2 Autocorrelation. Table@"n". minInterval<. fullT = Range@Min@tD. fills in the blanks with "n".

m = Mean@DropNonNumeric@timeseriesDD. Length@DropNonNumeric@timeseriesDD . denom. Table@filtered = N@DropNonNumeric@Transpose@8Take@timeseries. filtered@@All. Take@timeseries. Length@DropNonNumeric@timeseriesDD . HDropNonNumeric@timeseriesD . Briley's original code. 2DDD * H1 .jLL<.13b. Length@timeseriesD .mL ^ 2D. 8j.jLD<DDDD. Apply@Plus.jD. 8j.13a. Apply@Plus.Autocorrelation. -HLength@timeseriesD .2<D autocorrelationaOld@timeseries_ListD := Module@8var.2<D autocorrelationb@timeseries_ListD := Module@8m. filtered<. which I think is wrong.mLD ê denom<. Take@timeseries. Table@filtered = Transpose@ N@DropNonNumeric@Transpose@8Take@timeseries. CovarianceMLE@filtered@@All. filtered<. HDropNonNumeric@timeseriesD . 8j. Hfiltered@@1DD . Length@timeseriesD . denom = Apply@Plus. In[68]:= autocorrelationa@timeseries_ListD := Module@8m. 1DD. 8j. Length@DropNonNumeric@timeseriesDD .13b (alternate method).jD.mLD * Length@timeseriesD ê Hdenom * HLength@timeseriesD . Hfiltered@@1DD . 8j.mL ^ 2D. 1DD. Length@timeseriesD . 8j.j ê Length@timeseriesDL ê var<. Table@filtered = N@DropNonNumeric@Transpose@8Take@timeseries. CovarianceMLE@filtered@@All. filtered<. 8j.mL * Hfiltered@@2DD .2<D D D D D This is Royama 3. var = VarianceMLE@DropNonNumeric@timeseriesDD. -HLength@timeseriesD .jLD<DDDD.jLD<DDD. 2DDD ê var<. filtered@@All. filtered<.nb 3 In[67]:= This is Royama 3. -HLength@timeseriesD . var = VarianceMLE@DropNonNumeric@timeseriesDD.jD. Take@timeseries. Take@timeseries.2<D autocorrelationbOLD@timeseries_ListD := Module@8var. 8j. In[69]:= This is Royama 3. Length@DropNonNumeric@timeseriesDD . denom.jD. -HLength@timeseriesD . Length@timeseriesD . m = Mean@DropNonNumeric@timeseriesDD. In[70]:= .jLD<DDD. denom = Apply@Plus.mL * Hfiltered@@2DD . Table@filtered = Transpose@ N@DropNonNumeric@Transpose@8Take@timeseries.

kDD = Hacf@@kDD . For@k = 2.f@@k. 8Length@acfD<.f@@k. 8i. j < k. H* now get f@@k.iDD.1. kDD f@@k . ListD<D . f@@1. f@@1. k ++.jDD. It uses the standard Levinson-Durbin algorithm. jDD = f@@k . f@@k. jDD = f@@k . k .kDD by taking the diagonal elements *L Transpose@8Range@1. k ++.1<DL. For@j = 1. D H* now get f@@k. 8Length@acfD<D.nb The function partialAutocorrelation@acfOutput_ListD generates partial autocorrelations from a list of previously-calculated autocorrelations.1. D. Length@acfDD. k § Length@acfD. f<. kDD = Hacf@@kDD . 8Length@acfD<. Length@acfDD.4 Autocorrelation. D. For@j = 1. jDD . j ++.1. f<.1<DL ê HProduct@1 . j ++. For@k = 2. In[72]:= partialAutocorrelationOld@acfOutput_ListD := Module@8acf. 2DD. iDD acf@@k . ListD<D D H* initialize f and establish base case *L f = Table@0. 8i. 8i.iDD. k<DL. 8i. k .1. f@@k.Sum@f@@k .kDD by taking the diagonal elements *L Transpose@8Range@1. D. iDD acf@@iDD. kDD f@@k . 2DD. j < k. iDD acf@@k . jDD . acf = acfOutput@@All. D. k . f@@k.Sum@f@@k . H* initialize f and establish base case *L f = Table@0.1.1. k . In[71]:= partialAutocorrelation@acfOutput_ListD := Module@8acf. f@@k. acf = acfOutput@@All.1. k<DL ê H1 .Sum@f@@k . 1DD = acf@@1DD.f@@i. 1DD = acf@@1DD. Tr@f. Tr@f. 8Length@acfD<D.jDD. Previous version of the above that has a few errors in the loop indices. iDD ^ 2. k § Length@acfD.

AxesOrigin Ø 80. opts___D := Module@8window. data. points = Take@acf. points. myPlotRange<. res_D := Module@8r. Map@Point. 8i. 8-1. Table@8i. In[76]:= spectrum@acfOutput_List. 81. Graphics@Line@pointsDD. PlotJoined Ø True. opts___D := Module@8plotData. Round@Length@acfD * windowSpecD. emulating the appearance of the graphics in Royama (1992. Length@wD<D D Does what its name implies. 2DDD > 0. 8-1. w. p100). In[75]:= parzenWindow@M_D := N@Table@If@i < M ê 2. On@Graphics::gptnD.Autocorrelation. windowD. 88-1. Off@Graphics::gptnD. fig D plotAutocorrelation@acf_List. 8-0. None<. Table@8w@@jDD. "1. 0. 0<. 8j. 8i. Axes Ø True. fig = ListPlot@plotData. In[77]:= plotTimeseries@data_List. D Show@Graphics@8PointSize@0. l<. optsD. 1. 0. 2 H1 . "-1.i ê ML ^ 3D. M<DD ‡ Graphics functions Does what its name implies. fig<. PlotRange Ø myPlotRange. r = acfOutput@@All.0"<. "0. w = Table@Pi j ê Hres Length@rDL. 80.02D. pointsD<D. 1 . 80. If@Max@points@@All. AxesLabel Ø TraditionalForm@"rj "D.5. 0. l@@1DD r@@1DD + 2 Sum@l@@iDD r@@iDD Cos@w@@jDD iD.5. TextStyle Ø 8FontSize Ø 14<. plotData = If@Length@data@@1DDD ã 2. 1<DD<. myPlotRange = 880. Length@dataD<DD. Length@lD<D ê Pi<. 80. 8j. 8i. windowSpec_. window = If@windowSpec § 1. res Length@rD<D êê N. TextStyle Ø 8FontSize Ø 14<.5<. 0<. In[73]:= tukeyWindow@M_D := N@Table@1 + Cos@Pi i ê MD.0"<. window + 0.6 Hi ê ML ^ 2 + 6 Hi ê ML ^ 3. This is a common way of specifying the window length. Axes Ø True. 2DD. 1<. res_D calculates a smoothed power spectrum based on a list of previousè!!! ly-calculated autcorrelations and a specified resolution res. AxesLabel Ø 8TraditionalForm@tD.5. optsD .nb 5 Kernel functions used for smoothing of the power spectrum. 8i. M<D ê 2D The function spectrum@acfOutput_List. Ticks Ø 8None. ""<. l = tukeyWindow@2 Sqrt@Length@rDDD. If@Min@points@@All. 2DDD < 0. Ceiling@windowSpecDD. It uses the Tukey window function with window length 2 n where n is the length of the list of autocorrelations.0"<<<. data@@iDD<. PlotRange Ø All.5. ""<.

optsE D .5. confidence = 2 ê Sqrt@Length@pacfDD. Graphics@8PointSize@0.5.0125.0125<D.5. 0<. p100). Axes Ø True. lines. windowSpec_. 80. If@Min@points@@All. linesDD. In[78]:= plotPartialAutocorrelation@pacf_List. p Ticks Ø 8Table@8Pi * i. pointsD. "0.5. "-1. -confidence<.0"<. confidence<. 0<. If@Max@points@@All.5<. 0D.0"<. Ticks Ø 8None. Axes Ø True. bands = Graphics@8Dashing@80. i<. 0. "1. emulating the appearance of the graphics in Royama (1992. 80. PlotJoined Ø True. 8i. 8-1. Round@Length@pacfD * windowSpecD. ""<.5. 0. ""=.5. AxesLabel Ø TraditionalForm@"fj "D. 8window + 0. 2DDD > 0. 8-0. lines = Map@88#@@1DD. TextStyle Ø 8FontSize Ø 14<. 8-1. bands. -confidence<<D<D. 2DDD < 0. optsD myPlotRange = 880.6 Autocorrelation. Graphics@Map@Line. In[79]:= This function puts everything together — it's the function that is called in the EcoTool. Automatic<. AxesOrigin Ø 80. 1<. Show@bands. window = If@windowSpec § 1. Ceiling@windowSpecDD. opts___D := Module@8window. 0. confidence<<D. points. opts___D := ListPlotAspec. confidence. RGBColor@1.5. 80. Does what its name implies. points = Take@pacf. wj AxesLabel Ø 9TraditionalFormA" ÅÅÅÅÅÅÅ "E.nb Does what its name implies. PlotRange Ø myPlotRange. 8window + 0.5. 0.2<D. PlotRange Ø 8All. 1. Map@Point. 88-1. window + 0. #< &. plotSpectrum@spec_List. ""<. Line@880. 81. 0<.0"<<<. windowD. All<. TextStyle Ø 8FontSize Ø 14<. 1<DD<.02D. pointsD<D. myPlotRange<. Line@880.

spec = spectrum@ac. specacD<. DisplayFunction Ø IdentityD<<D. autocorrelationb@dataDD. DisplayFunction Ø IdentityD<. Log@dataD êê N. 8ac. PlotLabel Ø "Differenced data". 8plotAutocorrelation@ac. window. window. window. plotAutocorrelation@ acD. Show@GraphicsArray@88plotTimeseries@data. window_D := Module@8data. -1D. bias_. DisplayFunction Ø IdentityD<. 1D . DisplayFunction Ø IdentityD<. PlotLabel Ø "Raw data". 8plotPartialAutocorrelation @pac. DisplayFunction Ø IdentityD. -1DD. PlotLabel Ø "ACF". 1D . ac = If@bias ã "yes".Autocorrelation. data = If@log ã "yes". pac. pac. log_. PlotLabel Ø "PACF". pacD = partialAutocorrelation@acDD. dataProcessing@dataInputD. plotPartialAutocorrelation@pacD. window. PlotLabel Ø "ACF". DisplayFunction Ø IdentityD. DisplayFunction Ø IdentityD< D On@General::spell1D.Drop@data.Drop@data.nb 7 In[80]:= ‡ Turn spelling warnings back on In[81]:= timeseriesAnalysis@dataInput_. . acD = autocorrelationa@Drop@data. data = If@Length@dataInput@@1DDD ã 2. ac. PlotLabel Ø "PACF". dataD. plotTimeseries@ Drop@data. PlotLabel Ø "Power Spectrum". pac = partialAutocorrelation@acD. dataInputD. DisplayFunction Ø IdentityD. autocorrelationa@dataD. 8plotSpectrum@ spec. 2D. ImageSize Ø 700. spec.

0. 100<D. 1. "yes".3D. Show@results@@4DD. 8i. "no".8 Autocorrelation.nb Example analyses ‡ White noise In[82]:= tempData = Table@8i. results = timeseriesAnalysis@tempData. DisplayFunction Ø $DisplayFunctionD. Random@D<. .

4 0. p -0.05 ÅÅÅÅÅÅÅÅÅ 0.0 PACF -1.15 0.6 -0.0 fj 1.4 0.0 .Autocorrelation.1 -0.5 0.1 wj 0.2 0.nb 9 1 0.05 0.0 ACF Differenced data 0.25 20 40 60 80 100 -0.0 Power Spectrum 0.0 fj 1.25 t -0.5 -0.0 PACF -1.75 rj 1.75 0.15 -1.81.8 0.0 ACF -1.6 0.2 Raw data t 20 40 60 80 100 rj 1.

8 1. Raw data 2 1 -1 rj 1. 101D.2 -0.60. DisplayFunction Ø $DisplayFunctionD.0 wj ÅÅÅÅÅÅÅÅÅ 0. "yes".5L &. p .0 Power Spectrum 5 4 3 2 -1.10 Autocorrelation. 0.0 fj 1.20.0.0 PACF -1.40. 100D<D. 0.0 PACF fj 1.nb ‡ Random walk In[85]:= tempData = Transpose@8Range@1.2 -0.0 t 20 40 60 80 100 ACF 0.0 ACF -1.4 0. NestList@# + HRandom@D . Show@results@@4DD.4 Differenced data t 20 40 60 80 100 rj 1.3D. "no". results = timeseriesAnalysis@tempData.

2985<. 81860. 81894. 81911. 81851. 81885. 81912. 81877. 4031<. 2536<. 81829. 1000<. 81839. 245<. 81906. 81892. 81863. 81864. 81848. 73<. 1033<. 81891. 2935<. 2685<. 2657<. 81883. 81901. 2821<. 6721<. 3574<. 3794<. 81844. 39<. 409<. 80<. 81828. 485<. 81824. 81859. 229<. 4245<. 81867. 758<. 736<. 2251<. 1824<. 81914. 81856. 3091<. 361<. 81827. 279<. 4950<. 81858. 382<. 546<. 81874. 81869. 81930. 81843. 81857. 299<. 81822. 3311<. 81934. 957<. 81862. 756<. 81916. 81846. 81836. 81826. 871<. 2811<. 552<. 3928<. 81928. 81876. 81825. 81888. 81823. 81907. 81903. 2713<. 81887. 1676<. 2119<. 674<.nb 11 ‡ Lynx data Data In[88]:= lynxData = 881821. 387<. 81915. 81898. 377<. 81847. 377<. 81919. 399<. 81886. 3396<<. 2577<. 49<. 81905. 188<. 81921. 2042<. 81866. 587<. 81840. 81852. 81925. 1307<. 269<. 81896.Autocorrelation. 81842. 81920. 68<. 81909. 81<. 6991<. 184<. 81845. 81872. 2432<. 2725<. 81917. 1638<. 3790<. 153<. 81849. 81831. 81881. 81890. 81855. 5943<. 81850. 81833. 2871<. 81923. 225<. 81878. 201<. 1426<. 784<. 81841. 81870. 108<. 731<. 59<. 808<. 81838. 2511<. 1475<. 409<. 229<. 4431<. 98<. 81926. 81854. 6313<. 473<. 151<. 360<. 81922. 81868. 81902. 81835. 81895. 81830. 81837. 81879. 299<. 81927. 1590<. . 81884. 81910. 1836<. 105<. 687<. 1537<. 81899. 81889. 523<. 321<. 81834. 81900. 81931. 3800<. 358<. 1292<. 81897. 81853. 2285<. 81924. 81882. 2129<. 1594<. 662<. 81904. 529<. 81871. 213<. 81873. 81933. 345<. 81875. 81908. 1388<. 684<. 469<. 3409<. 81865. 45<. 81880. 1132<. 81832. 81932. 81918. 3465<. 81929. 236<. 81893. 255<. 585<. 81861. 389<. 3495<. 1623<. 81913.

0 ACF -1.60.0 3 2.nb Analysis In[89]:= results = timeseriesAnalysis@lynxData.5 2 1. DisplayFunction Ø $DisplayFunctionD.5 Power Spectrum -1. 30D.0 wj ÅÅÅÅÅÅÅÅÅ 0. Raw data 8 7 6 5 20 40 60 80 100 rj 1.0 PACF -1.0 ACF t 1 -1 -2 Differenced data 20 40 60 80 100 t rj 1. Show@results@@4DD.0 fj 1.5 0.12 Autocorrelation. "yes".20.0 fj 1.8 1.0 PACF -1.40. p . "yes".