OSCAR Users Manual

Fabrice Bonjean, John T. Gunn, and Gary S.E. Lagerloef

Table of contents

1.

!er!ie" 1.1. #uic$ Start

The cean Surface %urrent &nalysis 'eal(ti)e * S%&'+ ,rocessing syste) is described in this docu)ent. This is an entirely auto)ated co),uter syste) that gathers and ,rocesses u,(to(date source data, including satellite sea surface height *SS-+, "ind and sea surface te),erature *SST+ data, calculates ocean surface !elocity, and creates surface !elocity files "hich are then used by the S%&' "eb ser!er for the dis,lay.do"nload facilities *see Figure 1Error/ 'eference source not found+. This )anual concerns only the S%&' co),uter and its interfaces "ith the source

SSH SERVER

WIND SERVER

SST SERVER

ft, . 1 1S ft, . 1 1S
OSCAR COMPUTER

ft,
OSCAR SERVER

http

USER

Figure ! OSCAR s"ur#e $ata%#al#ulat"r%OSCAR $ata ser&er inter'a#es(

data and S%&' "eb ser!ers *Figure 1Error/ 'eference source not found, in grey+. nce the S%&' ,rocessing syste) is ,ro,erly set u, *section 2+, it is able to run and u,date all the datasets auto)atically at fi2ed ti)e, inde,endently of any )anual )anage)ent. 3e!ertheless )anual )aintenance is re4uired fro) the S%&' )anager after the scheduled auto)ated session of the ,rocessing is ter)inated, in order to chec$ the data 4uality and the status of the ,rocess run. &t the end of the )aintenance, the S%&' )anager also ta$es care )anually of the transfer of the u,dated ,roducts to their storage folder. The S%&' ,rocessing syste) can be o,erated in )anual )ode at any ti)e, by )anually running any ,rocedure in off(schedule sessions. This is for e2a),le the )ethod for re(running a full S%&' ,rocessing session *if, for so)e reason, the scheduled job did not run ,ro,erly+, or for reconstructing a s,ecific dataset *SS-, "ind, SST or surface !elocity+. Se!eral data ,roducts )ay be a!ailable for one ,articular source dataset *SS-, "ind or SST+, and it is ,ossible to s"itch fro) one to the other by )odifying the ,rocessing syste) *through a relati!ely straightfor"ard )odification+. &t the ti)e of the deli!ery fro) ES' to 3ES15S, the data ,roducts that are i),le)ented and chosen as o,erational ,roducts1 are the ones that ha!e been ad!ocated since the beginning of the 3 66. S%&' ,roject, and, coincidently, allo" *for no"+ the regular u,date of the satellite( deri!ed current "ith the shortest delay2.

1.2. -ost %o),uter 3a)e/ oscaroscar+ L5378 o,erating syste). %ontrolling Bourne shell *.bin.sh+ scri,t e2ecutions. 1is$ free s,ace/ at least 19 Gbytes. :atlab soft"are !.9 or later ,lus ;Signal 6rocessing< toolbo2. 5nternet connection *high(s,eed+ 5ntranet connection to a net"or$ *3ES15S+ &ccess to a ft, ser!er located at 3ES15Slocal ft, ser!er

1.0. The ; ,e31&6.1 1S< utility

These ,roducts are the 7SF gridded alti)etry, % &6S #ui$Scat gridded "ind !ector, and 'eynolds.S)ith gridded SST datasets. 2 5n general, a )ini)u) of = days and a )a2i)u) of 1> days se,arate the current date and the date of the last u,dated )a, on the S%&' "eb site.
1

>

The ,e31&6.1 1S and ft, interfaces are the only )eans "ith "hich the data are i),orted to, and e2,orted fro) the S%&' co),uter *Figure 1Error/ 'eference source not found+. The follo"ing is a brief ,resentation of ,e31&6.1 1S. The ,e31&6.1 1S interface allo"s the ,rocessing syste) to access the re)ote data and i),ort the) "ithin a :atlab session. This :atlab.1 1S co)bination is utili?ed in auto)ated ,rocesses )anaged by Linu2 shell scri,ts and :atlab ,rogra) files. 5n a :atlab session, the :atlab co))and used to do"nload data *;loaddods<+ re4uires as in,ut the 5nternet resource location *7'L+ of the 1 1S site "here the data are located. The result of the loaddods co))and )a$es the re4uested data a!ailable as a )atri2 in the :atlab "or$s,ace, "here they can directly be )ani,ulated using the :atlab tools *inter,olation, filtering, dis,lay, calculation...+. & significant benefit of using the ,e31&6.1 1S interface, there is no need to consider the source data for)at. 1.>. 1irectory structure 1.>.1. General directories The ho)e directory of the S%&' )anager, "hose user@s na)e is ;o)<, is .ho)e.o). 1escending fro) .ho)e.o), the )ain directory and subdirectory trees associated "ith S%&' are sho"n in Figure 2. The .ho)e.o).oscarA9d directory includes all directories and files for the S%&' ,rocessing syste)0. The descendant directories of oscarA9d consist of the ;datasets< directory and its subdirectories, and additionally three folders that contain the ,rogra) files controlling the syste)/ the directories ;)files< * S%&' :atlab routines+, ;shfiles< * S%&' e2ecutable shell scri,ts+ and ;toolbo2< *free :atlab toolbo2es co),rising the 3etcdf and 1 1S ,ac$ages+. &t the sa)e directory le!el an additional ;bin< folder includes so)e utility ,rogra)s that facilitate the )anual )aintenance tas$s conducted by the o) user. The ;oscarAnetrc< file, "hich is located at the sa)e le!el, )ay be used for auto)ated ft, connection and file transfer to the local 3ES15S ft, site *ho"e!er, this )ethod is not i),le)ented any)oreB see section 2.2+. 1.>.2. The ;datasets< directory and subdirectories The oscarA9d.datasets folder includes three directories, each associated "ith a ty,e of source data/ ;SS-< for the sea surface height data, ;C531< for the surface "ind data, and ;SST< for the sea surface te),erature data. There is one directory for the surface current !elocity data,
The folder na)e ;oscarA9d< distinguishes the 9(day data fro) the data that "ere historically ,rocessed on a 1D(day ti)e base.
0

na)ed %7''E3TS. There can be )ulti,le choices of source data ,roduct ,er ty,e of data, but only one ,roduct ,er ty,e is selected for o,erational use in the ,rocessing syste). There is one subdirectory associated "ith each ,roduct. Thus, for this !ersion of this soft"are ,ac$age, the source data ,roducts selected as o,erational ,roducts for calculating the surface currents are/ the 7SF sea surface height data ,roduct located in ;oscarA9d.datasets.SS-.7SF<, the #ui$Scat surface "ind data ,roduct in ;oscarA9d.datasets.C531.SA#ui$Scat<B this dataset includes the historical SS:5 data for the ,eriod before &ugust 1EEE, hence the ;SA< suffi2B the 'eynolds and S)ith sea surface te),erature data in ;oscarA9d.datasets.SST.'EF3 L1S<. &nd the resulting o,erational surface current data are located in ;oscarA9d.datasets.%7''E3TS. S%&'<. ther directories )ay be created at this le!el in order to store other surface current ,roducts, for e2a),le, data to co),are "ith and !alidate the S%&' currents *as seen belo"+. Each ,roduct directory *e.g C531.SA#ui$Scat+ contains a ;%' 3< directory in "hich the u,dated files of the corres,onding dataset are auto)atically ,roduced according to a user( deter)ined schedule. The na)e %' 3 is chosen after the Linu2 cron facility, "hich is used in the S%&' ,rocessing in order to schedule the regular runs of the auto)ated ,rocess at a fi2ed ti)e. These scheduled runs subse4uently referred to as ; S%&' cron jobs<. The go!erning rule is that any data file ,roduced by an S%&' cron job is created in the corres,onding %' 3 directory *e.g. C531.SA#ui$Scat.%' 3+, "hereas data that "ere ,re!iously u,dated and !erified are stored in the ,arent directory *e.g. C531.SA#ui$Scat+. 5n the follo"ing sche)e, ,arent folders of the %' 3 directories are na)ed ;storage< directories. To a!oid any accidental and ,ossibly har)ful interactions bet"een inde,endent )ani,ulations of the data and those of the o!erall ,rocessing syste), the storage and %' 3 directories should be considered as strictly reser!ed for out,ut fro) the S%&' cron jobs and )aintenance tas$s. 6arallel data ,rocessing acti!ities related to S%&' should )a$e use of the 6' % directories *as sho"n after"ards+ or, of course, any ne" directory created by the o) user.

6' % directories are si)ilar to %' 3 directories but are not in!ol!ed in the scheduled runs of the ,rocessing syste). 6' % directories )ay be used to ,roduce u,dates of the data at any ti)e as needed. For e2a),le, they )ay be used for ,reli)inarily building or reconstructing datasets, testing routines, or 4uic$ly re(starting an u,date "hich, for so)e reason, did not succeed during the scheduled S%&' cron job. The te),orary directories *T:6+ )ay be auto)atically created during the S%&' cron job. &s a data safety ,rocedure, they are created and ,o,ulated "ith the u,dated ,roducts that )ay be found in the %' 3 directories at the start of a ne" session of the S%&' cron job. This situation occurs "hen the u,dated files "ere not transferred fro) %' 3 to the storage folder before the start of the ne2t session. They do not ha!e any acti!e ,art in the ,rocess, and can be deleted "hen con!enient, if their content is not needed. See the section 1.9 for further details.

The %7''E3TS directory contains se!eral subdirectories associated "ith the surface current data. ne of these, the S%&' directory, contains the deri!ed current !elocities resulting fro) the a,,lication of the S%&' )ethodology using the source data. The %7''E3TS directory also includes other folders that are associated "ith tests of the !arious source data *e.g. the !elocity data that are deri!ed fro) the %LS absolute to,ogra,hy are stored in SS-ATESTS.%LS.&1T+. There )ay also be other folders storing !arious current ,roducts for co),arison "ith S%&' *e.g 1'5FTE'S+.

Figure )! $ata tree "' the OSCAR pr"#essing s*ste+(

1.9. 6rocessing !er!ie" The S%&' ,rocessing is a co)bination of the uni4ue scheduled auto)ated ,rocedure *the S%&' cron job+ and so)e ,ost(,rocessing )anual tas$s co),leted by the S%&' )anager for 4uality re!ie" and )aintenance. 5f these )anual tas$s cannot be ,erfor)ed for a gi!en data cycle, the syste) is, ne!ertheless, set u, to continue to u,date the datasets as long as the ,rocessing is scheduled and no ,roble) occurs. Therefore, the S%&' ,rocessing can be considered fully auto)ated. 5t is, of course, reco))ended, "hene!er ,ossible, to co),lete the 4uality re!ie" and )aintenance tas$s, "hich notably include !isually chec$ing and ,ro,erly storing the data. &side fro) the ft, transfers of the final, u,dated ,roducts *the surface current dataset+, all files ,roduced by the S%&' cron job are created in the %' 3 directories. 1.9.1. S%&' cron job & uni4ue )eta(,rocedure, "hich is na)ed ;auto)aticAoscarA,rocessA9d<, is scheduled to run at a date and ti)e s,ecified in the cron ti)etable. 5t )ay run "hen no one is logged on to oscar. 6referably, the date and ti)e should be set u, to run at night ti)e *e.g D.D9a) ET+. &n ense)ble of sub(,rocedures and routines are in!o$ed fro) the )eta(,rocedure, and altogether ,erfor) the follo"ing tas$s/ 1+ %hec$ and clean the "or$ing directories/ create %' 3 folders if needed in each directory associated "ith the o,erational source data *SS-, "ind, SST+, )o!e any files found in the already e2isting %' 3 folders to T:6 folders. 2+ 7,date datasets/ do"nload the re)ote source data/ SS-, "ind, SST, ,re(,rocess the source data, ,roduce gra,hics for subse4uent !isual chec$s, calculate surface current !elocity, ,ost(,rocess the !elocity data, ,roduce gra,hics for subse4uent !isual chec$s. 0+ :anage data files/ create 3etcdf files of the ,rocessed current data and co,y files to the local 3ES15S ft, site, E

 e2,ort files to the re)ote S%&' "eb ser!er for data dis,lay.distribution/ this is actually ta$en care of by the 6:EL ser!er. 1.9.2. :anual tas$s 1+ 6ost(,rocessing chec$ %hec$ ,rocessing status fro) the S%&' cron job e()ail, if needed, chec$ the log files ,roduced by the auto)ated :atlab sessions, !isuali?e the gra,hics generated by :atlab to !erify the u,dated datasets. 2+ File transfer 5f the datasets "ere a,,ro,riately u,dated *chec$s in 1 abo!e+, )o!e all files fro) the %' 3 to the storage directories. This is done for each data ,roduct, SS-, "ind, SST and surface currents. The syste) is then ready for a ne" u,dating session. 5),ortant note/ the file transfers should be co),leted each ti)e the data are u,dated, "hich occurs "ith a fre4uency ranging fro) 1 to H days. -o"e!er, if the file transfers are not ,erfor)ed follo"ing one or )ore effecti!e sessions of the S%&' cron job, at the ne2t session all files in the %' 3 directories are auto)atically )o!ed to te),orary T:6 directories. The u,dated files thus )o!ed are not ta$en into account during this ne2t session "hich "ill then u,date the data again, ,erha,s to a )ore recent date. 1.G. Bac$u, syste) 1.G.1. Scheduled bac$u, & scheduled auto)atic general bac$u, is set u, to ,roceed fro) one dis$ to the other on the oscar co),uter. The co),uter includes t"o hard dri!es "ith a no)inal si?e of 11DGB. The S%&' ,rocessing syste) is located and runs on one hard dri!e only, )ore ,recisely on the .de!.hda9 filesyste). E!ery night, the entire contents of the .ho)e directory, that notably includes the o) directory, is du,licated on the .data.bac$u, directory located on the second hard dri!e *the .de!.!gD1.l!Adata filesyste)+. 1.G.2. ther bac$u, The S%&' )anager should regularly bac$u, all the ,rocedure files of the S%&' ,rocessing syste) *the actual ;soft"are<+ to an inde,endent storage s,ace *%1(' :, other co),uter, I+. 5n case of loss of all or ,art of the S%&' data, the datasets can al"ays be reconstructed through off(scheduled auto)ated runs controlled by the S%&' ,rogra) files *section G+. 5n case of loss of so)e soft"are co),onent, a co,y $e,t at ES'@s office in Seattle can be re4uested. &n e2act 1D

i)age of the S%&' ,rocessing syste) is ,lanned to $ee, on running at all ti)e at ES' *no" running+.

11

2. 5nstallation.%hec$ ,oints 2.1. %o),uter connection 2.2. 6reli)inary Setu, %hec$ that the follo"ing ite)s are defined in the file syste)s/ o 7serJs na)e/ o) *oscar )anager+ o -o)e directory/ .ho)e.o) o scar directory/ .ho)e.o).oscarA9d directories, ,o,ulated "ith e2ecutable *bourne(+ shell scri,t files, :atlab scri,t files, and data files. o &dditionally, it should include an oscarAnetrc file "hich )ay be used to control the auto)atic ft, connections to the 3ES15S local ft, site. %hec$ the .usr.local.)atlab.etc.license.dat file/ the :atlab license nu)ber and host 51 should be 2H9=DG for John Kuhn. :odify the ser!er address in the file if needed *at ,resent it is oscar.esr.org and "ill need to be changed at the deli!ery+. %hec$ that the userJs shell is tcsh, "hich is associated "ith the .ho)e.o)..cshrc startu, file *chec$ this file e2ists+. 3ote that the ty,e of user@s shell is rele!ant only to the direct )ani,ulations of the S%&' ,rocessing syste), and not to the auto)ated ,rocessing. %hec$ that the .cshrc file includes the follo"ing lines/ seten! S%&' Lho)e.oscarA9d seten! :&TL&B .usr.local.:atlab seten! :&TL&B6&T- L S%&'.)files set ,ath M * .usr.local.bin .bin .usr.bin .usr.811'G.bin .usr.sbin + set l,ath M * L S%&'.shfiles L:&TL&B.bin + set ),athM* L S%&'.bin + 12 o The oscar directory should include the shfiles, )files, toolbo2, datasets and bin 3a)e/ oscar 5nstallation of net"or$ connection, "ith *at least+ internet, e)ail, ft,.

set ,ath M * L,ath Ll,ath L),ath + %hec$ that the directory structure descending fro) .ho)e.o).oscarA9d.datasets e2ists and is filled "ith ,rocessed data. The directory structure *the s$eleton+ "as initially built by si),ly running the .ho)e.o).shfiles.oscarAdatatree e2ecutable shell scri,t. %hec$ that the directory .ho)e.o)..!nc e2ists and that it includes the ;,ass"d< file. The !nc facility allo"s a user to o,en a !irtual dis,lay "indo" "ith no ,ro),ted ,ass"ord re4uest, in order to create :atlab figures during the cron jobs. This set u, "as initially ,erfor)ed once and for all by e2ecuting the co))and ;!ncser!er /1< in a ter)inal "indo", then by entering a ,ass"ord at re4uest, and finally by $illing the !irtual dis,lay "ith the co))and ;!ncser!er ($ill /1<. 'eser!e a directory for the S%&' data in the local ;,ub< ft, site of 3ES15S. Let@s call this directory ;,ubA,ath.dataAdir<, and let@s call ;nesdisAser!er< the ser!er that )anages the ft, facility at 3ES15S. Files ha!e to be transferred to ,ubA,ath.dataAdir fro) the oscar co),uter, and there are at least t"o o,tions/ i+ 3FS direct co,y/ this o,tion is currently installed, but )ay only be acti!ated as such, ,ro!iding nesdisAser!er o,erates under Linu2 or 7ni2. The ,ubA,ath.dataAdir has to be shared and )ounted on the oscar co),uter through the Linu2.7ni2 3FS facility. n oscar, create as ;root< the ne" directory .ho)e.dataAdir/ )$dir .ho)e.dataAdir Edit as root the .etc.fstab file, adding the ne" line/
nesdisAser!er/,ubA,ath.dataAdir .ho)e.o).dataAdir nfs rsi?eM=1E2,"si?eM=1E2,ti)eoM1>.ho)e.ft,

Logon to nesdisAser!er as root and edit the .etc.e2,orts file, adding the ne" line/ ,ubA,ath.dataAdir 3ES15S net"or$+. Still on nesdisAser!er, e2ecute ;e2,ortfs Na< as root. Bac$ on oscar, e2ecute ;)ount Na< as root. %hec$ that any co,ied file to .ho)e.dataAdir by o) on oscar, is actually co,ied to the ft, directory ,ubA,ath.dataAdir on nesdisAser!er. &s o), edit the .ho)e.o).oscarA9d.shfiles.oscarfileA9d scri,t, changing the line ;FT6A15'MI< to FT6A15'M.ho)e.dataAdir oscar.nesdis.go!*r",sync+ *;oscar.nesdis.go!< stands for the full address of the oscar co),uter connected to the

10

The syste) is then set u, to auto)atically co,y the S%&' files fro) oscar to the ,ubA,ath.dataAdir directory. ii+ Ft, transfer/ the ser!er)achine na)e nesdisAser!er, the user@s na)e and user@s ,ass"ord ha!e to be s,ecified in the .ho)e.o).oscarA9d.oscarAnetrc file in order to o,erate auto)atic ft, transfers. For ob!ious security reasons, this should re)ain anony)ous ft, connection, "here the user@s na)e is ;anony)ous< and the ,ass"ord is the user@s e)ail. -o"e!er, this re4uires that the anony)ous user has the ,er)ission to "rite in the ,ubA,ath.dataAdir directory, "hich )ay not be con!enient or e!en ,ossible. Edit the .ho)e.o).oscarA9d.shfiles.oscarfileA9d file/ t, and the allocated ft, directory ,ath has to be s,ecified as FT6A15'M.ho)e.dataAdir This o,tion can be acti!ated and set u, in .ho)e.o).oscarA9d.shfiles.oscarfileA9d by re)o!ing ;O< at the beginning of the lines that control the ft, auto)atic )ani,ulations, co))enting out the lines that are associated "ith the 3FS file co,y fro) o,tion i abo!e *add ;O< at the beginning+, and by editing accordingly the .ho)e.o).oscarA9d.oscarAnetrc file. These t"o o,tions i and ii allo" the ,rocessing syste) to auto)atically co,ying the data files to the allocated ,ub directory once the S%&' current datasets are u,dated. This ft, address, "hich )ust be reachable fro) outside through anony)ous ft,, )ust be ,ro!ided to the S%&' "eb site )anagers *3ancy Sorreide, Cilla Phu, Cilla.PhuQnoaa.go!+ at 6:EL, and also to the )anagers of the S%&' data distribution at 6 .1&&% *Kelly %ase, Eric 'igor, erQseastar.j,l.nasa.go!+. OO S-ELLM.bin.sh - :EM.ho)e.o) 6&T-M.usr.local.bin/.bin/.usr.bin/.usr.811'G.bin/.usr.sbin/.ho)e.o).oscarA9d.shfiles/.usr.local .)atlab.bin :&5LT MbonjeanQesr.org S%&'M.ho)e.o).oscarA9d :&TL&B6&T-M.ho)e.o).oscarA9d.)files 1> %hec$ that the %' 3 jobs are acti!ated/ ;crontab (e< to edit, ;crontab (l< to chec$. The o)Js crontab file should include the follo"ing lines/

O S%&' 6' %ESS53G D9 DD R R R auto)aticAoscarA,rocessA9d .ho)e.o).oscarA9d.datasets.%7''E3TS. S%&'.%' 3

The :&5LT !ariable )ust be e4ual to the actual e()ail address of the ,erson in charge of the S%&' ,rocessing. 5f needed, the dates and ti)es that are currently set u, in the crontab file )ay be changed.

19

0.

,eration 0.1. Soft"are and )ethod

The soft"are languages used in the S%&' ,rocessing syste) e2ist in Bourne shell *sh+, and :atlab ,lus associated toolbo2es. 5n the follo"ing, a ;sh(file< is an e2ecutable shell scri,t, and ;)(file< a scri,t that is run "ith :atlab. & sh(file na)ed ;auto)aticAoscarA,rocessA9d< is in!o$ed fro) the cron ti)etable, and co))ands the e2ecution of all subse4uent S%&' tas$s *see Figure 0+. 5t is therefore at the highest ran$ in the soft"are hierarchy. The auto)aticAoscarA,rocessA9d sh(file calls another high(ran$ sh(file, ;,rocessAallA9d<, "hich controls all tas$s for the data ,rocessing by in!o$ing subordinate sh(files and associated :atlab sessions. The !elocity calculation is ,erfor)ed in one :atlab session, "hich is directly in!o$ed fro) the ,rocessAallA9d sh(file. The ,rocessAallA9d sh(file also calls the ;oscarfileA9d< sh(file "hich creates and e2,orts the !elocity data files. 6recisely, fro) ,rocessAallA9d a co)bination of one sh(file and one or )ore )(files is dedicated to each of the follo"ing tas$s/ do"nloading.,rocessing the ssh data, do"nloading.,rocessing the "ind data, do"nloading.,rocessing the sst data, calculating the surface !elocity creating.e2,orting the !elocity files.

3ote/ in this soft"are !ersion, the SS:5.#ui$Scat "ind ,roduct is the only ,roduct in!ol!ing the e2ecution of )ore than one )(file *0+. nly one ,rocedure is sho"n in Figure 0.

5n general, the sh(files ensure the )anage)ent of directories and files, ft, connections and transfers, and the e2ecution of the )(files in :atlab sessions. The )(files ta$e care of the data e2traction fro) the re)ote sites *only for those "ith 1 1S ser!ers+, the data ,rocessing and the ,roduct file creation. The )(files are located in the .ho)e.o).oscarA9d.)files directory and the sh(files in .ho)e.o).oscarA9d.shfiles. For e2a),le, to u,date the 7SF SS- data, there e2ists a single )(file *6'E6A7SFSS-A91.)+ "hich chec$s the data, adds the )ean dyna)ic 1G

to,ogra,hy *:1T+, and creates gra,hics for ,ost(,rocessing chec$sB and there is a single sh(file *,rocessAusfsshA9d+ "hich do"nloads the SS- data through ft, and runs the :atlab session in!o$ing 6'E6A7SFA91.). For the SS:5.#ui$Scat "ind and the 'eynolds SST data, the )( files control the data do"nloading through the 1 1S interface *no ft, transfer in!ol!ed+. 1 1S is also used for do"nloading the %LS *%3ES+ alti)etry ,roducts. The na)es of the associated sh(files and )(files are consistent "ith each other/ for e2a),le, the ,rocessAusfsshA9d sh(file is associated "ith the 6'E6A7SFSS-A91.) )(file, ,rocessAreysstA9d "ith 6'E6A'EFSSTA91.), ,rocessAs4s"indA9d "ith the three )(files 6'E6A#S%C531A91, 6'E6A#S%C531A3'TA91, 6'E6AS#SC531A91, etcI &s there are se!eral ,roducts ,er ty,e of data *SS-, "ind, SST+, there are also different na)es of sh(files and )(files/ for e2a),le, ,rocessAusfsshA9d and 6'E6A7SFSS-A91.) for the 7SF SS- ,roduct, and ,rocessAclssshA9d and 6'E6A%LSSS-A91.) for the %LS SS- ,roduct. &s only one of those t"o ,roducts is chosen, only one of the t"o sets of ,rocedures is selected for o,erational S%&' and is effecti!ely called by the ,rocessAallA9d sh(file. The na)es of the auto)aticAoscarA,rocessA9d, ,rocessAallA9d and oscarfileA9d sh(files are fi2ed and should stay uni4ue. 5n su))ary, the sh(files and )(files associated "ith the SS-, "ind and SST data ,rocessing confor) to the follo"ing synta2/ ,rocessAaaaBBBA9d and 6'E6AaaaBBBA91.), "here aaa is the ,roduct na)e or origin and BBB the ,roduct ty,e. The ;,rocessA!elAaaaAbbbAcccA9d< sh(files that also e2ist in the )(files folder are not in!ol!ed in the S%&' cron job. They can be run in off(scheduled jobs in order to re(,rocess or test the surface current calculation using different source data ,roducts *according to the aaa, bbb and ccc letters in the na)e+. They are co),letely inde,endent of ,rocessAallA9d. For e2a),le, ,rocessA!elAclssshAs4sAreyA9d calculates the current !elocity using the %LS SS- ,roduct, the SS:5.#ui$Scat ,roduct and the 'eynolds SST ,roduct. The alternati!e !elocity files thus created are stored in the datasets.%7''E3TS. S%&'.SS-ATESTS.%LS.SL& directory "here they can be tested. 5),ortant notes/ each sh(file can be e2ecuted indi!idually in off(schedule sessions, for e2a),le directly at the Linu2 ,ro),t in a ter)inal "indo", or using the co))and ;at< for i))ediate or delayed runs in the co),uter bac$ground. This can be used to build or reconstruct a dataset *see section G+. 5n off(schedule sessions, in,ut !alues ha!e to be entered in )ost of the sh(files, such as the end date until "hich the ,articular data ,roduct is ,rocessed. Finally, the sh(files )ust not 1H

be run )anually "ithin %' 3 directories, but fro) the a,,ro,riate 6' % directories *sections 1.>.2 and G+.
auto)aticAoscarA,rocessA9d ,rocessAallA9d ,rocessARRRsshA9d
6'E6ARRRSS-A91

TIME

,rocessARRR"ind A9d ,rocessARRRsstA9d

6'E6ARRRC531A91

6'E6ARRRSSTA91

:&KEASELA91

oscarfileA9d 3egen$! sh(file )(file call :&KEA S%&'F5LEA91

Figure ,! -eneral 'l". #hart "' the s#he$ule$ aut"+ati# pr"#ess /OSCAR #r"n 0"12(

0.2. 1ata Source 5nterfaces 1ata do"nloading fro) the re)ote data ser!ers is a crucial ,art of the S%&' ,rocessing syste) and is fully auto)ated. The )ethod used for do"nloading de,ends on the a!ailable syste) and ,rotocol currently set u, on the re)ote ser!er *seeError/ 'eference source not found Table 1+. Chen a!ailable, the 1 1S.:atlab interface is the ,referred )ethod, as it reduces the a)ount of inter)ediate ste,s fro) the connection to the ser!er to the )ani,ulation of the selected data "ithin the :atlab "or$s,ace. To retrie!e the 7SF SS- u,dated ,roduct, a ft, do"nload session is first ,erfor)ed during the e2ecution of the sh(file, and then the single do"nloaded )at(file is locally loaded "ithin a :atlab session. 5n this ,articular case, the co)bination of ft, and :atlab tas$s is si),le and short since the 7SF SS- data are already for)atted to calculate the currents on the S%&' grid. 1=

1ata source ,roduct 7SF gridded alti)etry #ui$Scat gridded "ind 'egular ,roduct #ui$Scat gridded "ind 3'T ,roduct 'eynolds T S)ith SST *!.2+ %LS *%3ES+ gridded alti)etry *2DD1( ,resent+ %LS *%3ES+ gridded alti)etry *1EE2( 2DD1+ %LS *%3ES+ gridded alti)etry *2DD1( ,resent+

5nterface

E2traction ty,e

%haracteristic

&ddress *FT6, 7'L+

FT6 ser!er 1 1S ser!er

1 "hole global file Global daily files

&nony)ous ft, 3o refined selection allo"ed 3o refined selection allo"ed

"i$i.)arine.usf.edu *,ub.T6Agrid+ htt,/..""".coa,s.fsu.edu. cgi(bin.dods.n,h(nc.dodsAdata. #ui$S%&TAC531SA1d htt,/..""".coa,s.fsu.edu. cgi(bin.dods. n,h(nc.dodsAdata. #ui$S%&TA3'TA1d htt,/..iridl.ldeo.colu)bia.edu. S 7'%ES..3 &&..3%E6. .E:%..%:B..GL B&L. .'eynAS)ith 5!2.."ee$ly..sst htt,/..o,enda,.a!iso.oceanobs.co). thredds.dods%.)erged.h. )slaAoerA)ergedAh

1 1S ser!er

Global daily files

1 1S ser!er 1 1S ser!er

Full selection allo"ed *ti)e.longitude.latitude+ Full selection allo"ed &no)aly field *ti)e.longitude.latitude+

1 1S ser!er

Full selection allo"ed &no)aly field *ti)e.longitude.latitude+

htt,/..o,enda,.a!iso.oceanobs.co). thredds.dods%. hhA)slaAoerA)ergedAh

1 1S ser!er

Full selection allo"ed &bsolute field *ti)e.longitude.latitude+

htt,/..o,enda,.a!iso.oceanobs.co). thredds.dods%.)erged.h. )adtAoerA)ergedAh

Ta1le ! Data s"ur#es an$ inter'a#es(

1E

Alth"ugh +"st "' the re+"te OpenDAP%DODS ser&ers are si+ilarl* "rgani4e$5 the alg"rith+ has t" 1e a$apte$5 an$ the parts "' the +6'iles that are $e$i#ate$ t" $".nl"a$ thr"ugh DODS +a* 1e s"+e.hat $i''erent 'r"+ "ne pr"#e$ure t" an"ther( In general5 the alg"rith+ p"rti"n "' the +6'ile that ta7es #are "' the DODS $ata i+p"rt has the stru#ture sh".n in Figure 8(

S,ecification of the 7'L 7'LMUhtt,/.."""I@

1o"nload ti)e ,eriods of dataset loaddods*7'L,@Vti)e@+

Ti)e ,eriods do"nloadedV

1is,lay error E85T :atlab

Last re)ote u,dated ,eriod ne"er than last local u,dated ,eriodV

SK56 1o"nload

1o"nload coordinates and ne" data Loaddods*7'L,@V8 or F or data@+

Figure 8! S#he+ati# 'l". #hart "' DODS $".nl"a$ing +anage+ent in a +6'ile pr"gra+(

3ote that for the #ui$Scat ,roduct s,ecifically, the algorith) directly chec$s for ne" daily files on the re)ote site, until the current date *today+. 1o"nloading )ay not ,roceed for three )ain reasons/ 1+ the connection cannot be established "ith the re)ote site, 2+ data does not e2ist yet in the re)ote siteB 0+ the re)ote dataset is found but is not u,dated. Chen 1 1S is used, the te2t generated in the J B.out log files *see section 0.0 belo"+ infor)s the S%&' )anager of the do"nloading ,rocedure result. For e2a),le the case 1+ abo!e )ay occur because the re)ote site@s 7'L has changed *due to a change of access ,ath for e2a),le+. The error )essage generated by the loaddods :atlab function )ay notify the user of this fact, de,ending on the degree of refine)ent of the re)ote 1 1S ser!er *distincti!e 2D

)essages are ,ro!ided by the 5'51L.L1E and %LS 1 1S ser!ers+. -o"e!er, if a data ,roduct is not u,dated after a long ti)e and if a sa)e non(e2,licit )essage fro) loaddods a,,ears in the J B.out log file, "e reco))end chec$ing the 7'L ,ath directly *using a bro"ser "indo"+. Chen Ft, is used, the transfer status is indicated in the %ron job e)ail at the end of the session *section 0.H+. 0.0. %' 3 files 0.0.1. Files that are created in all %' 3 directories *SS-, "ind, SST, and %7''E3TS. S%&'+ &fter a successful run of the S%&' cron job, at least three ty,es of files e2ist in the %' 3 folders/ :atlab data files *R.)at+, na)ed ;)at(files<, "hich include the u,dated source or calculated data ,roduct, gra,hics e,s(files *R.e,s+, "hich dis,lay general )a,s and ti)e series ,lots used for ,ost( ,rocessing chec$s, out,ut log files, J B.out, "hich include any te2t generated during the :atlab sessions. These J B.out files allo" the S%&' )anager to chec$ the status of the cron job. 3ote/ no J B.out file is created in the 7SF SS- %' 3 directory if the 7SF dataset is not u,dated on the re)ote ft, ser!er *indeed, in this case there@s no :atlab session+. 0.0.2. Files created in the %7''E3TS. S%&'.%' 3 directory only 5n this directory, other ty,es of file e2ist in addition to the ones abo!e/ %o),ressed 3etcdf data files *R.nc.g?+ that are e2,orted to the local 3ES15S ft, site, in order to be retrie!ed by the 6:EL S%&' ser!er, the S%&'(9d.created and S%&'(9d.latest files, "hich are te2t files that res,ecti!ely contain the date of creation and the date of the last u,dated ,eriod. These files are also e2,orted to the local 3ES15S ft, site, and their content is used by the 6:EL S%&' ser!er. The S%&'.gdr file is a te2t file that includes the ti)e ,eriod of the s"itch fro) the G1' to 5G1' )odes of the alti)etry data for J&S 3(1. This is used by 6 .1&&% for distribution ,ur,oses. 6 .1&&% gets the S%&' data directly fro) the local 3ES15S ft, ser!er. 3ote/ the out,ut job file in the datasets.%7''E3TS. S%&'.%' 3 folder is na)ed &LLAJ B.out and gathers all J B.out files fro) the SS-, "ind and SST %' 3 directories, ,lus, in the end, the out,ut te2t ,roduced by the !elocity ,rocessing. 21

& se,arate out,ut job file na)ed S%F5LESAJ B.out is also created in this directory. 5t docu)ents the :atlab session that ,roduces the 3etcdf files to be e2,orted. 7sually, after a successful run of the S%&' cron job, the o) user )o!es all files fro) the %' 3 directories to the storage directories *u,,er folders+, thus, u,dating the files in the storage directories *section >.>+.

22

0.>. 1ata %ollection 0.>.1. Ti)e and s,ace grids For all data ty,es and ,roducts the geogra,hical range includes the tro,ical 6acific area fro) 12DWE to HDWC, 09WS to 09W3, and the ti)e ,eriod generally ranges fro) ctober 1EE2 to ,resent. The s,ace grid is a 1Wlongitude X 1Wlatitude grid, starting in the northeast corner at 12D.9WE, 0>.9W3 *the latitude coordinates and data arrays are arranged fro) north to south+. The ti)e base is a 4uasi 9(day ti)e ste, !ector co),rising e2actly H2 ti)e ste,s ,er year. E2ce,t for 1EE2, the ti)e base starts on January 1st for each year, so that in year units/ TMFE&', FE&'Y1.H2, FE&'Y2.H2, I, FE&'YH1.H2, "here FE&'M1EE0, 1EE>, I, ,resent year. For 1EE2, the data series actually starts on ctober 11th *TM1EE2Y9G.H2+. 0.>.2. 1ata files &ll data )at(files and the )ain !elocity 3etcdf files are yearly files *1EE2, 1EE0,I, 2DD>, I+. The data directories associated "ith each ,roduct include the historical data files, u, until the last u,dated files. Each successful session of the S%&' cron job ,roduces yearly files for ,resent and ne2t ,re!ious year at the )ost. The data ,rior to the ,re!ious year are not ,rocessed again during an S%&' cron session, and therefore the associated data files are not )odified>. 1ata file na)es/

The na)es of the final 9(day data files for each ty,e of data are listed in Table 2Error/ 'eference source not foundError/ 'eference source not found. This list does not include the secondary *but still i),ortant+ files that are ,roduced and sa!ed at so)e inter)ediate ste,s of the u,date ,rocess. 5n the source data, there are a cou,le characteristics to ta$e note of. For the %LS SS- data, one of the t"o source ,roducts consists of sea le!el ano)aly data *to "hich the S%&' ,rocessing adds a )ean dyna)ic to,ogra,hy, :1T+, and the other is absolute dyna)ic to,ogra,hy data *&1T+ to "hich a :1T field has already been added at the source. 5n order to distinguish bet"een these t"o ,roducts, the notation &1T is used in the latter case. For the 7SF SS- dataset, a uni4ue )at(file GlobalAgridAH2Ayr.)at, "hich includes the entire data ,eriod * ct.1EE2(,resent+ is do"nloaded through ft, e!ery ti)e it is u,datedB this file is at all ti)e located in the corres,onding storage directory *it is not do"nloaded into the %' 3 folder, this is the sole e2ce,tion to the rule+. -o"e!er, the yearly files ,re,ared fro)
>

-o"e!er, this )ay be done )anually "ithin an off(scheduled session, if ,rior files need to be re,rocessed.

20

GlobalAgridAH2Ayr .)at are created in the %' 3 directory "ith na)es consistent "ith the other SS- ,roducts. SS:atlab *.)at+ aaa9SSH9:$;;;;(+at "r Cind SST aaa9ADT9:$;;;;(+at aaa9WIND9:$;;;;(+at aaa9SST9:$;;;;(+at 3one 3one oscarA!elA9dFFFF.nc sfc(unfilterFFFF.nc sfc(filterFFFF.nc S%&' currents S%&'ASELA9dFFFF.)at S%&'ASELA9dFFFFf.)at sfc(unfilter(longter).nc sfc(filter(longter).nc sfc(unfilter()onthly.nc sfc(filter()onthly.nc
Ta1le )! Main $ata 'iles #reate$ 1* the OSCAR #r"n 0"1 in the CRON $ire#t"ries( S"+e se#"n$ar* +at6'iles are n"t liste$ in this ta1le( <aaa< stan$s '"r the $ata pr"$u#t /e(g( S=S '"r SSMI%=ui7S#at25 ;;;; '"r *ear(

3etcdf *.nc+ 3one

Filtered S%&' current files are also created, and a subscri,t ;f< is added at the end of each file na)e. &lso, in addition to the final yearly 3etcdf files, so)e ;3etcdf sfc(,,,.nc< files are built and are ulti)ately utili?ed by the 6:EL S%&' ser!er *see section 0.9Error/ 'eference source not found+. These files contain the 9(day !elocity data and ,re(calculated long(ter) )eans, )onthly )eans and ano)alies in a s,ecific for)at. These are used in the S%&' data dis,lay.do"nload "eb ,age to be ,ro!ided to users at their re4uest. 5),ortant note/ as a rule, all the file na)es are ,rescribed "ithin the sh(files and ha!e to be )odified there, if needed. The root na)e and the file ty,e are ,rescribed in the sh(files *e.g. C531AF5LEMS#SAC531A9d.)at+, but the years are o)itted. The year nu)bers are inserted into the file na)es through string )ani,ulations during the )(files e2ecution. 1ata file co),ression/

The only co),ression ty,e recogni?ed by the ,rocessing syste) is the ;g?i,<.<gun?i,< facilities associated "ith the ;.g?< file na)e e2tension. The S%&' )anager should not use other co),ression ty,es for any tas$ related to S%&'. &s a rule, any )at(file created by the ,rocess is unco),ressed, and any 3etcdf file is co),ressed *g?i,,ed+. 2>

3ote/ the ,rocessing syste) is able to deal "ith any )at(file that "as co),ressed in the storage directories ,rior to a ne" session of the S%&' cron job. 5f so)e file co),ression "as done for e2a),le to o,ti)i?e the hard(dri!e s,ace *"hich is not needed for no"+, then the auto)ated ,rocedure un(co),resses the files, loads the data and re(co),resses the files. B7T, it is then re4uired to )a$e sure that follo"ing the S%&' cron session, the data file are not )anually du,licated during the )aintenance *see section >+. For e2a),le, so)e confusion "ould ensue if in the sa)e directory g?i,,ed old files cohabited "ith ne"ly u,dated unco),ressed files "ith the sa)e root na)e.

1ata file structure and notation :at(files

The )at(files are only inter)ediate files, and for this reason )at(files are not internally docu)ented *no )etadata in contrast "ith the 3etcdf files9+. -o"e!er, data ,rocessing and calculation are ,erfor)ed "ithin :atlab sessions, and the data fields ,ri)arily stored in the )at( files in order to be easily reached during subse4uent :atlab sessions. Table 0 ,ro!ides so)e useful notations for understanding the )(files that ,erfor) the ,re,aration of the source and surface current data.

Longitudes Latitudes Ti)e ,eriods SSCind !elocity SST Geostro,hic currents

9

Sariable *units+ 8 *WE+ F *W3+ Tjd *julian days+ ht *)+ C *).s+ SST *W%+ 7g *).s+ 29

&dditional Sariables Tyr *years+

7gf *).s+ filtered

The 3etcdf files are the ulti)ate files ,roduced by the S%&' cron job, and are the e2,orted ones.

E$)an.Sto))el currents Buoyancy grad. currents

7Ch *).s+ 7Bh *).s+

7Chf *).s+ filtered 7Bhf *).s+ filtered

Ta1le ,! Varia1les in +at6'iles(

3ote/ the )files directory contains the ,rocedures that con!ert year units to Julian days and !ice !ersa *J7L5&32F', F'2J7L5&3+. For e2a),le, the J7L5&321&TE )(file can be used to con!ert Julian days to the date in !arious character string for)ats.

2G

3etcdf files

&ll 3etcdf files include so)e )etadata regarding the data they collect. The )ost docu)ented 3etcdf files are the yearly oscarA!elA9dFFFF.nc files. These include unfiltered and filtered data, "hich are se,arated into the three ter)s of the S%&' !elocity *7g, 7Ch, 7Bh+, and are the)sel!es s,lit into both ?onal and )eridional co),onents. 0.9. S%&' interface

The S%&' data ser!er is located at 6:EL. The "eb site *htt,/..""".oscar.noaa.go!+, "hich is attached to the ser!er, includes data dis,lay and do"nloading facilities for general users. The "eb ,age o,tions for dis,laying or do"nloading data are interacti!ely selected by the users and s,ecifically include geogra,hical range, ti)e ,eriod, 9(day inter!als or )onthly )eans, long( ter) )ean, total or ano)alous !elocities and filtered or unfiltered data. So)e of these o,tions re4uire the use of the ,re(calculated fields contained in the sfc(,,,.nc 3etcdf files that are listed in Table >. These are the last created data files in the %' 3 current directory, and they are sent together "ith the other 3etcdf files to the local 3ES15S ft, folder.

sfc(unfilterFFFF.nc sfc(filterFFFF.nc sfc(unfilter(longter).nc sfc(filter(longter).nc sfc(unfilter()onthly.nc sfc(filter()onthly.nc

7nfiltered, 9(day yearly, total !elocity and ano)alies Filtered, 9(day yearly, total !elocity and ano)alies 7nfiltered, 1EE0(2DD0 ense)ble )ean, total !elocity Filtered, 1EE0(2DD0 ense)ble )ean, total !elocity 7nfiltered, )onthly )eans, total !elocity and ano)alies Filtered, )onthly )eans, total !elocity and ano)alies

Ta1le 8! Pre6#al#ulate$ 'iles sent t" the OSCAR ser&er(

3ote/ the reference ,eriod for the long ter) )ean and the )ean seasonal cycle changes e!ery year. -ence, in 2DD9 the reference ,eriod "ill beco)e 1EE0(2DD>. 2H

The 6:EL S%&' ser!er auto)atically chec$s the local 3ES15S ft, folder into "hich the

'"r #he#7ing the USF SSH $ataset(

S%&' data are e2,orted at the end of the S%&' cron job, and do"nloads the data files for u,dating the "eb site. This auto)atic ,rocedure is set u, to run t"ice a day at 2.2G,) and 9.9Da) ET. 0.G. &uto)ated gra,hics

2=

1uring the e2ecution of the :atlab routines, so)e gra,hics are generated and sa!ed in the %' 3 directories together "ith the data files. Their ,ur,ose is to dis,lay the data, so that the S%&' )anager can handily chec$ that the u,date ,rocess has run a,,ro,riately. These figures are of t"o sorts/ 1+ general longitude latitude )a,s, and in addition, )a,s restricted to certain areas *only for surface currents+B 2+ ti)e series ,lots of the data e2tracted at certain locations

Figure :! Maps '"r #he#7ing the USF SSH $ataset(

*usually at one location on the e4uator and at 1D3 on the dateline+. :a, figures

2E

Figure ?! Treate$ &el"#it* 'iel$( Figure >! Treate$ &el"#it* 'iel$(

There are usually three general )a,s ,er figure/ the first t"o )a,s sho" the te),oral )ean and the te),oral standard de!iation of the data ,roduct relati!e to the ,eriod that has been u,dated. The third )a, sho"s the last ,eriod of the data ,roduct *Figure 9+. TError/ 'eference source not foundhere )ay be successi!e and si)ilar )a, figures for one data ,roduct, corres,onding to !arious ste,s of the data ,rocessing *this is indicated in large bold letters on the figure+. Ponal and )eridional co),onents of the "ind are sho"n on se,arate figures. Surface current !ectors *arro"s+ are dis,layed on one figure only and su,eri),osed on their a),litude *color ,atches+. 0D

@! Treate$ 'iel$( Ti+e series "' theFigure =ui7S#at $ata5 at&el"#it* su##essi&e pr"#essing steps( Treate$ &el"#it* 'iel$(

:a,s of currents in restricted areas *far "estern and eastern regions, Figure G+ Figure Hare sho"n t"ice on t"o successi!e figures, before and after the s,ecial treat)ent that consists in filtering ,ossible s,urious !elocities deri!ed fro) the calculation *s,urious !alues )ay occur "ithin the e4uatorial areas that include coasts+. Sisuali?ing te),oral )ean and standard de!iation hel,s to chec$ that the u,dated data series is consistent "ith the ty,e of the dis,layed data *fa)iliar ,atterns+, and also hel,s trac$ing occasional large errors in the data analysis co)ing fro) the source. For e2a),le, t"o

Figure >! Ti+e series "' =ui7S#at 4"nal .in$ at a spe#i'i# l"#ati"n '"r su##essi&e steps "' the pr"#essing(

do"nloaded 1(day )a,s of the #ui$Scat "ind data at t"o successi!e ti)e ,eriods "ere found to
Figure >! Ti+e series "' =ui7S#at 4"nal .in$ at a spe#i'i# l"#ati"n '"r su##essi&e steps "' the pr"#essing(

include large errors. 5nitially, those ti)e ste,s had been auto)atically ,rocessed. &t the )aintenance chec$, the te),oral )ean and standard de!iation )a,s clearly re!ealed a ,roble) in the data, "hich, after !erification, could only co)e fro) the source. The ,roble) "as fi2ed *through so)e inter,olation ,rocedure+ and the ,rocess ran again. 01

So)e statistics relati!e to the data !alues "ithin each )a, are dis,layed on to, of the )a,, as an aid to chec$ the consistency of the data/ )ini)u) *:in+, )a2i)u) *:a2+, )ean *:ean+ and standard de!iation *Std+.

Ti)e series ,lots

These figures sho" the te),oral data !ariations at successi!e ste,s of their ,rocessing si)ultaneously *indicated in legend, seeError/ 'eference source not found Figure H+. They ,er)it the detection of any ,roble) co)ing fro) the s)oothing and inter,olation ,rocedures that are ,art of the ,rocessing, but also, as the )a, figures, so)e ,ossible issue in the source data. 0.H. 6rocess ter)ination &fter each session of the S%&' cron job, an e)ail )essage is auto)atically sent to the )anager of the S%&' syste), "hose e)ail address is set u, through the :&5LT !ariable in the crontab file. The e)ail rece,tion by the S%&' )anager confir)s that the scheduled ,rocedure is ter)inated, "hether it fully succeeded or not. The sender is ;%ron 1ae)on ZrootQoscarI[< and the subject ;%ron auto)aticAoscarA,rocessA9d<. The e)ail )essage lists each ste, of the ,rocessing and includes any te2t generated by the shell during the run of the sh( files. 5n order of a,,earance these ste,s are/ auto)aticAoscarA,rocessA9d ,rocessAallA9d ,rocessAaaasshA9d ,rocessAbbb"indA9d ,rocessAcccsstA9d oscarfileA9d

The out,ut te2t includes the start ti)e of the ,articular ,rocess, a short su))ary of "hat the ,rocess does, the directory "here the files are created, the in,ut !alues that are o,tionally read at the beginning of the ,rocess, so)e s,ecific infor)ation about the ,rogress of the ,rocess

02

*notably regarding the called )(files "ithin :atlab sessions+, a list of data files that are effecti!ely created by the ,rocedure, and a su))ary of instructions for )aintenance.

00

>. :aintenance The follo"ing is a su))ary of the S%&' )anager@s tas$s. :aintenance tas$s are si),le and of four ty,es/ chec$ing the S%&' "eb ,age, reading the out,ut e)ail, !isuali?ing out,ut gra,hics and transferring created files. >.1. Ceb ,age chec$ Before chec$ing the out,ut of the ,rocessing syste) itself, first o,en the S%&' "eb ,age/ htt,/..""".oscar.noaa.go!. There are usually three ,ossibilities/ 1+ the surface current )a, see)s ,ro,er and the date of the )a, has changed to a )ore recent date since the last ti)e it "as chec$ed. Then, it is ,ractically certain that the ,rocessing syste) ran correctly. The )aintenance tas$s described in the sections >.2, >.0, and >.> can be acco),lished ,ro),tly. 2+ 3othing has changed, and the )a, is the sa)e as before. This )eans that/ i+ either the source data at the re)ote ser!ers "ere not sufficiently *or not enough+ u,datedB a , "hich allo"s 4uic$ )aintenance can be ,erfor)ed *sections >.2, >.0, >.>+as "ellB ii+ or so)e ,roble) occurred during the run of the S%&' cron job, re4uiring a )ore careful chec$ *see ne2t ,aragra,hs+. See sections >.2, >.0, >.> as "ell. 0+ There is a !isible ,roble) on the )a, dis,layed by the "eb ,age/ 'e)o!e files fro) the local 3ES15S ft, site folder *"here 3etcdf files are e2,orted+. :anually transfer a co,y of all the last u,dated 3etcdf files, located in the surface current storage directory *.ho)e.o).oscarA9d.datasets.%7''E3TS. S%&'+, to the 3ES15Sat ft, folder. This ensures that the ne2t ti)e the "eb ser!er do"nloads the data fro) 3ES15S *t"ice a day at 2.2G,) and 9.9Da) ET+ the "eb ,age re!erts to the ,re!ious and !erified state. Sol!e the ,roble), if ,ossible, before the ne2t session of the S%&' cron job *see in ,articular section >.2+. 5),ortant note/ the ,rocessing syste) al"ays atte),ts to u,date the "eb ,age inde,endently of the )anual )aintenance tas$s, since the final surface current files are auto)atically co,ied to the 3ES15S ft, site, right after they are created. Thus, ,rocessed files and gra,hics are not chec$ed before the data are sent to the S%&' "eb site. ther"ise the ,rocessing syste) and "eb ser!er ense)ble "ould not be fully auto)ated. This is "hy it is reco))ended to chec$ the "eb ,age e!eryday in order to sol!e any ,roble) as 4uic$ly as ,ossible. 0>

>.2.

'eading out,ut job e)ail and log files >.2.1. %hec$ing the e)ail )essage fro) ;%ron 1ae)on ZrootQoscar[<

5f the e)ail )essage fro) the last S%&' cron job "as recei!ed, then the u,date ,rocess "as run and ter)inated, "hether it succeeded or not. 'ead the e)ail )essage thoroughly, in ,articular chec$ing that no fatal error occurred during the e2ecution of the shell ,rogra)s. Chile reading, see if ne" files "ere created in the %' 3 directories, es,ecially the )ost i),ortant ones/ the current !elocity )at( and 3etcdf files should be listed near the end of the e)ail. &lso, chec$ if the !elocity 3etcdf files "ere ,ro,erly transferred to the 3ES15S ft, directory. Follo" the instructions gi!en after each ,rocess ter)ination, belo" the title ;B&S5% :&53TE3&3%E T&SKS< *note/ so)e instructions )ay be redundant+. >.2.2. %hec$ing the log files fro) :atlab sessions %hec$ing the J B.out, &LLAJ B.out and S%F5LESAJ B.out )ay be acco),lished for different reasons and in !arious circu)stances. -o"e!er here are t"o situations in "hich chec$ing the .out files is reco))ended, es,ecially if these situations are recurring o!er se!eral days/ at least one data ,roduct *source or surface current+ "as not u,dated by the S%&' cron job *no .)at file "as created in the corres,onding %' 3 directory+. First, chec$ the ter)ination status of the corres,onding :atlab session in the J B.out file, and chec$ that there is no direct :atlab error *indicated by so)e out,ut te2t ,roduced by :atlab and starting by ;VVV ErrorI<+. This could occur if the )(file*s+ "as ,re!iously )odified and if an error "as introduced. Thoroughly read the te2t in the J B.out file. 5nfor)ation is gi!en regarding so)e of the $ey ste,s of the ,rocessing. 5n ,articular, J B.out confir)s if the data "ere actually u,dated at the re)ote data ser!er, and if the re)ote data could be reached at that ti)e. 5f not, then the data could not be u,dated. 3ote that the J B.out files are all gathered in a uni4ue &LLAJ B.out file located in the currents %' 3 directory.

09

The final data files "ere not created *e.g )issing 3etcdf !elocity files in the current %' 3 directory+. %hec$, at the end of the S%F5LESAJ B.out file, if the :atlab session "as interru,ted due to a :atlab error. Thoroughly read the te2t in the S%F5LESAJ B.out file. >.0. 1ata 1is,lay and 1ata #uality %hec$s

&s reco))ended in the e)ail out,ut )essage, the o) )anager should chec$ all the figure files *.e,s+ in each %' 3 directory after the S%&' cron job is co),leted. 5n a %' 3 folder, just ty,e the alias co))and ;gsall< "hich dis,lays all figures successi!ely in a ghostscri,t "indo". The order of the figures coincides "ith the chronological order of the ,rocessing, and the figures sho" the data at $ey ste,s. 5n )a,s, see if the data configuration see)s correct, or if there is any see)ingly s,urious ,attern, for e2a),le, so)e large noise s,read in ;along(trac$< directions. For the surface current data, so)e o!erly large a),litude !ectors *[1.2 ).s+ )ay be noticed in the ;untreated< data )a,s in the "estern)ost and eastern)ost e4uatorial regions *close to islands and coasts+. This is nor)al. These are corrected during the ,rocessing, as sho"n in the figure that usually follo"s. Filtered surface currents are also dis,layed on a subse4uent figure. For ti)e series gra,hics, chec$ that the ,rocessed data at successi!e ste,s, sho"n on the sa)e ,lot, re)ain consistent "ith the source data. See if there is any sus,icious ju), in ti)e. These !isual chec$s co!er )ost, but not all, as,ects of the o!erall data !erification. 5t is suggested to also re!ie" feedbac$ fro) users "hene!er ,ossible. >.>. File transfer 6reli)inary re)ar$/ the transfer of the final surface current files *3etcdf+ to the local 3ES15S ft, directory is auto)ated and occurs "hene!er those files are created. The file transfer discussed here is ,art of the )aintenance of the stored datasets after the 4uality control ste,s are ,erfor)ed, and is therefore acco),lished )anually. &s also indicated in the e)ail out,ut )essage, "e reco))end the transfer of files fro) the %' 3 directories to the storage directories as soon as ,ossible once the u,date has succeeded.

0G

Thus, "hen the abo!e !erification tas$s are done and there is enough confidence that the data are ,ro,erly u,dated/ go to the %' 3 folders and enter ;)! R.R ....< at the Linu2 ,ro),t. This has the effect of e2tending the duration of the stored datasets, either by just adding a ne" yearly file to the collection, or by re,lacing ,re!ious files "ith ne" and bigger ones. 3ote/ so)eti)es, ne" surface current files are created, but the length of the data ,eriod is not e2tended. This ha,,ens "hen so)e source data "ere not sufficiently e2tended in ti)e, but still enough data "as added so that the 4uality "as i),ro!ed. Files are transferred in this case too. Go to the storage folders *"here the data files ha!e just been )o!ed+, and )a$e sure that there are not du,licate files, "ith co),ressed and unco),ressed files "ith the sa)e root na)e *the only difference co)ing fro) the .g? e2tension+. 5f there are du,licate files, chec$ the date of creation of both files and delete the older one *or, in doubt, )o!e it to a T:6 folder+. 3ote/ this only ha,,ens if the S%&' )anager or another user )anually co),resses or unco),resses so)e data files in the storage directories before an S%&' cron job session is started. Chat ha,,ens if/ the S%&' )anager is unable to acco),lish the )aintenance tas$s for se!eral daysV 3o ,roble)/ the ,rocessing syste) runs as scheduled, and any unchec$ed ,roducts is auto)atically )o!ed to a T:6 folder in the %' 3 directory. The "eb ser!er is auto)atically u,dated each ti)e the ,rocessing syste) ,roduces u,dated data locally. the S%&' )anager returns after se!eral days, during "hich the files ha!e ne!er been transferredV The )aintenance tas$s )ust be ,erfor)ed, including file transfers as indicated in the ,re!ious section. 3ote that e)ail out,ut )essages fro) cron )ay ha!e accu)ulated in the S%&' )anager@s )ailbo2, and that these )essages should be read fro) the )ost recent to the oldest *as needed+. Then/ o 5f no u,dated files e2ist in the %' 3 directories, go to the %' 3.T:6 directories and see if u,dated files "ere created during an earlier session. 5f so, ,roceed to chec$ing those u,dated ,roducts *gra,hics, etcI+ as indicated abo!e 0H

*section >.0+. This includes, as a,,ro,riate, the transfer of the data files fro) the T:6 to the storage directory *;)! R.R .......<, 2 le!els higher+. &fter"ards, the cause of failure for the last u,date session )ust be trac$ed *read the last e)ail recei!ed fro) cron, and the J B.out file in the %' 3 directory, etcI+. o 5f u,dated files e2ist in the %' 3 directories, ,roceed to a regular )aintenance as ,re!iously described in the section *>.1, >.2, >.0, >.>+.

9. Troubleshooting 5f the e)ail )essage corres,onding to the last scheduled ,rocess has not been recei!ed fro) ;%ron 1ae)on< since it "as started the night before, or after )ore than t"o hours ha!e ,assed, then the ,rocess )ay still be running or )ay be stalled for so)e reason. %hec$ the list of ,rocesses that are currently run by o) *use the co))and ;,so)<+. 5f the scheduled ,rocess is still running or stalled, then it is listed as ;auto)aticAoscarA,rocessA9d< fro) the ,so) co))and, as "ell as all the descendant ,rocesses that are still running or stalled. 5f a listed ,rocess is stalled then $ill the auto)aticAoscarA,rocessA9d ,rocess and all secondary ,rocesses. Try to find the cause of the ,roble) as usual/ read the e)ail )essage fro) cron, "hich is recei!ed right after the ,rocess is $illed, and "hich indicates at "hich stage the ,rocedure stalledB read the J B.out files that "ere created. :a$e sure that the co),uter syste) is "or$ing ,ro,erly/ chec$ the o!erall )e)ory s,ace, list all ,rocesses and chec$ there is no ,rocess e2,loiting all the '&: or %67, see if there is so)e accu)ulation of ;?o)bie< ,rocesses. 5t is then ,ossible to re(run auto)aticAoscarA,rocessA9d "ithin an off( schedule session *section G+, or "ait for the ne2t scheduled session of the S%&' cron job. Failure or stalling of the ,rocessing syste) li$ely occurs because the job sub)itted to the co),uter is o!ersi?ed in ter)s of '&:. So)eti)es, this results fro) the fact that one or )ore datasets )ay ha!e to be reconstructed fro) the beginning *until ;today<+. ff(schedule ,rocessing has to be ,erfor)ed instead, for this ,ur,ose *section G+.

0=

0E

G.

ff(schedule ,rocessing

&ny dataset that is ,rocessed and u,dated by the S%&' syste) can be u,dated, built and rebuilt inde,endently of the S%&' cron job. This does not a,,ly to so)e ,er)anent datasets, such as the SS:5 "ind, the )ean dyna)ic to,ogra,hy or the )as$ data files, for "hich no s,ecific u,date ,rocedure is needed. For e2a),le, the #ui$Scat "ind dataset can be entirely re( built if the data "ere accidentally lost, or if their analysis at the source *% &6S+ "as u,graded. This can be done easily, since any e2ecutable sh(file in the S%&' sh(files directory can be run inde,endently "ithin an off(schedule session. &n off(schedule session )ay be of t"o ty,es/ 1+ using the Linu2 ;at< co))and allo"ing the s,ecification of the start ti)e for one single run session of the ,rocessB 2+ starting the job directly at the ,ro),t. Ce reco))end the use of the ;at< utility, since ,rocesses ta$e a "hile to run, and the user can then logoff at any ti)e. Si)ilarly to a cron job, an out,ut e)ail )essage is sent to the job o"ner after the ,rocess is ter)inated, "ith the subject ; ut,ut fro) your job< follo"ed by a nu)ber auto)atically assigned to the ,rocess. ne dra"bac$ of ;at< is that the job cannot al"ays be chec$ed in real( ti)e "hile running. To chec$ the job "hile running, just ,rint the J B.out files to the screen re,eatedly. E2a),le 1/ restarting )anually the co),lete auto)ated ,rocess i+ Go to the surface !elocity storage folder .ho)e.o).oscarA9d.datasets.%7''E3TS. S%&', and fro) there, to the 6' % directory. ii+ enter the follo"ing co))ands at the ,ro),t *denoted by ;L<+ L at no" at[ auto)aticAoscarA,rocessA9d and e2ecute a Z%T'L(1[ The u,dated files *if any+ are created in the 6' % directories of the res,ecti!e data ,roducts. 5f, by )ista$e or on ,ur,ose, neither a %' 3 or 6' % directory is chosen for running the ,rocess, then the u,dated files are all ,roduced in that sa)e directory. The ,rocedure abo!e includes sending the data to the 3ES15S local site. 5n order to s$i, this ,art, use the ,rocessAallA9d sh(file instead of auto)aticAoscarA,rocessA9d/ L at no" >D

at[ ,rocessAallA9dZZE F at[today 2>> at[E F Z%T'L(1[ The in,ut !alues ;today< and 2>> are e2,lained in the e2a),le 2 belo". 3ote/ this should not be used for reconstructing the S%&' datasets fro) the beginning. The a)ount of '&: re4uired by the job "ould be too large, and the ,rocess "ould stall or crash. See belo" for reconstructing indi!idual dataset using inter)ediate ti)e ste,s. E2a),le 2/ reconstructing the SS:5.#ui$Scat dataset, or the S%&' !elocity dataset. First, note that the SS:5 dataset already e2ists and only the #ui$Scat data is do"nloaded fro) the re)ote site. The SS:5 dataset should additionally be bac$ed u, on a different dis$, or can be re4uested fro) ES'. For the S%&' !elocity dataset, all the source datasets ha!e to e2ist and be u,dated ,rior to the run of the !elocity ,rocedure. Follo"ing is the se4uence of )anual tas$s to ,erfor)/ i+ Go to the storage folder .ho)e.o).oscarA9d.datasets.C531.SA#ui$Scat for the "ind, or to .ho)e.o).oscarA9d.datasets.%7''E3TS. S%&' for the surface currents, and )o!e all files to a te),orary folder created in that directory *e.g. datasets.C531.SA#ui$Scat.T:6, or datasets.%7''E3TS. S%&'.T:6+. ii+ Go to the 6' % directory *not %' 3+ and enter the follo"ing co))ands at the ,ro),t/ L at no" at[ ,rocessAs4s"indA9d ZZ E31 at[ today at[ E31 Z%T'L(1[ The ,rocess for building the "ind data "ill start right a"ay, and the data files "ill be created in the 6' % directory. The line containing ;today< is read by the ,rocedure in an in,ut !ariable na)ed E311&TE. & real date can be s,ecified *see i),ortant note belo"+. 5f the S%&' !elocity dataset is being reconstructed, the instruction se4uence is/ L at no" at[ ,rocessA!elAusfAs4sAreyA9dZZE31 at[ today 2>> filter at[ E31 Z%T'L(1[ >1

There are 0 in,ut !alues to be read in this case. 3otably, the 2>> nu)ber defines the nu)ber of days bac$"ard for "hich the !elocity data are re(,rocessed *retros,ect ,rocessing, see section =.2+. iii+ 6roceed to the )aintenance tas$s as indicated in the )aintenance section abo!e, "here ;%' 3< should be re,laced by ;6' %<. 5n this e2a),le, )aintenance tas$s are restricted to the "ind or surface !elocity dataset only. 5n ,articular, the e)ail )essage recei!ed fro) ;o)< "ith the subject ; ut,ut fro) your job< )ust be chec$ed. Just follo" the ;B&S5% :&53TE3&3%E T&SKS< at the end of the )essage. The job can be set u, to run at a later ti)e *see the ;at< )anual+. 5),ortant notes/ so)eti)es, it is reco))ended or re4uired to s,lit the reconstruction tas$ into se!eral jobs by s,ecifying inter)ediate dates bet"een the start of the dataset and ;today<. For e2a),le, instead of today abo!e, enter D1(Jul(2DD2/ at[ D1(Jul(2DD2 The dataset "ill then be rebuilt until this date. &fter )aintenance *including file transfer to storage folder+, another job can be started using a )ore recent date, and so onI This )ay be necessary since the a)ount of '&: used by the s,ecific job for reconstructing the "hole dataset )ay be !ery large and conse4uently the ,rocess "ould run !ery slo"ly. Fractioning the reconstruction tas$ into 0 or > jobs can considerably shorten its duration. Chen reconstructing a dataset fro) the start, the first date to be entered as in,ut should be at least 2.9 )onths later than the start date of the data ,roduct *0 )onths is safe+. This is due to the fact that the te),oral filtering ,rocedure re4uires that the ti)e series length in in,ut is at least 0 ti)es as large as the filter order *fi2ed to 9 in all cases+.

>2

H. S"itching ,roduct for source data The scar )anager )ay "ant to s"itch ,roduct ,er)anently or only te),orarilyG regarding a ,articular source dataset *SS-, "ind or SST+. The alternati!e ,roduct should already be installed and u,dated to a recent date, and include at least 12D days bac$"ards fro) ,resent *section =.2+. Belo", "e gi!e the e2a),le of s"itching SS- ,roduct fro) the 7SF to the %LS*SL&+ ,roduct, and it )ay be ada,ted to any other ,roduct s"itch. H.1. 6er)anent s"itch i+ :o!e all the files contained in the %7''E3TS. S%&' folder to another folder *not %' 3 or 6' %+. 5t could be a folder located in the %7''E3TS. S%&'.SS-ATESTS directory *and na)ed 7SF, follo"ing the current e2a),le+. ii+ %reate a co,y of the ,rocessAallA9d sh(file under a different na)e and edit the original/ change the directory ,ath in the line initiali?ing the a,,ro,riate 15'222 !ariable *15'SS-DML15'1&T&.SS-.%LS.SL&, follo"ing the e2a),le+, change the file na)e in the line initiali?ing the a,,ro,riate 222AF5LE !ariable *SS-AF5LEM%LSASS-A9d.)at+ change the na)e of the sh(file in the line that )anages the u,date ,rocessing of the ,articular source dataset *re,lace ,rocessAusfsshA9d by ,rocessAclssshA9d+ iii+ 'econstruct the S%&' !elocity dataset in the %7''E3TS. S%&' folder, using off(schedule runs in 6' % directory as sho"n in section G *,rocessA!elAclsslaAs4sAreyA9d is the ,rocedure to use+. 5n the general case, the ;,rocessA!elAaaaAbbbAcccA9d< sh(file for building the !elocity )ay not e2ist yet *for e2a),le if a ne" source ,roduct "as just installed+. There are then t"o ,ossibilities/ %reate and use a ne" ,rocessA!elAaaaAbbbAcccA9d ,rocedure. This just re4uires du,licating one of the e2isting ,rocedures under the a,,ro,riate na)e and )odifying directory and file na)e initiali?ations as sho"n abo!e for ,rocessAallA9d *ii+. 'un the ,rocessAallA9d ,rocedure in successi!e off(schedule sessions as done "ith any u,dating ,rocedure for reconstructing an indi!idual dataset/ L at no"
& te),orary s"itch )ay be re4uired if a source ,roduct cannot be u,dated for so)e li)ited duration, ,re!enting any u,date of the S%&' "eb ser!er for )ore than a "ee$.
G

>0

at[ ,rocessAallA9dZZE31 at[ today 2>> at[ E31 Z%T'L(1[ ;today< abo!e should be re,laced by successi!e and increasing inter)ediary dates *section G+. &s the source data are already constructed, only the !elocity dataset is ,rocessed during this )ani,ulation. H.2. Te),orary s"itch i+ ii+ 1o the tas$s in section H.1.ii abo!e *disregard the others+. %reate a co,y of the auto)aticAoscarA,rocessA9d sh(file under a different na)e, and edit the original/ s"itch the retros,ect ,eriod length, 2>>, "hich is in in,ut of ,rocessAallA9d, to D. The S%&' cron job then uses the alternati!e ,roduct for u,dating the surface current dataset. 1uring this te),orary s"itch, ne" S%&' )a,s are added to the initial series ,roduced "ith the regular source dataset, and the S%&' dataset beco)es ;)i2ed<. 5f the te),orary dataset is not needed any)ore just re!ert to the ,re!ious configuration, by re(installing the co,ied files as o,erational files fro) the t"o ,aragra,hs abo!e *i, ii+. 5f the regular dataset on the re)ote site is fully reco!ered and ready for the re,rocessing of the surface current data, the retros,ect ,rocessing de!ice e)bedded in the algorith) ta$es care of recalculating the S%&' dataset using the regular ,roduct. -o"e!er, iii+ if the alternati!e ,roduct "as used for a duration T larger than 0D> days *\1D)onths+ change the retros,ect in,ut !alue to T(GD *in day units+ in auto)aticAoscarA,rocessA9d. This is just for a one ti)e ,rocessing session and this !alue can be changed bac$ to 2>> after"ards.

>>

=. 1ata set ,rocessing The ,rocessing of the source SS-, "ind and SST data ,roducts is in general )inor, )ostly consisting in te),oral filtering and inter,olations to the S%&' ti)e and s,ace grid *section 0.>.1+, and arrange)ent of the data into yearly files. The ,rocessing of the surface current data includes the )odel calculation itself, outlier data treat)ent, and ti)e and s,ace filtering. =.1. Source data ,roducts =.1.1. 7SF SS- ,roduct E2ce,t for yearly file rearrange)ent, the 7SF gridded alti)etry data are not subject to any ,articular treat)ent during the S%&' ,rocessing. The 7SF SS- ti)e and s,ace grid already )atches the S%&' grid *section 0.>.1+. See Lagerloef et al *1EEE+, and Gary :itchu) *contact list in section 1D+ for reference regarding the gridding ,rocessing. &n absolute dyna)ic to,ogra,hy field is built fro) the 7SF gridded alti)etry data and through adding a )ean dyna)ic to,ogra,hy field *'5 :1TB 'io et al., 2DD>+ located in datasets.SS-.:1T.cls.clsrioAht.)at. &s the '5 :1T is rigorously relati!e to the 1EE0(1EEE ,eriod, the residual )ean relati!e to this ,eriod in the 7SF alti)etry ano)alies is re)o!ed before the :1T is added. =.1.2. #ui$Scat "ind ,roduct The source data are initially gridded on a 1day X 1deg X 1deg ti)e.latitude.longitude grid *6egion et al., 2DDD+. Chen the ne" 1day global arrays are do"nloaded, ,ossible ga,s in s,ace and ti)e are searched and filled through inter,olation. The data are then s,atially inter,olated on the S%&' grid. They are te),orally s)oothed using a Tc M2Dday lo"(,ass filter and subsa),led on the S%&' ti)e base. The last =D days of the ,re!iously u,dated and unfiltered 1day dataset *"hich is continuously stored+ are a,,ended first to the ne" 1 day ste,s before lo"( ,ass filtering is a,,lied. This is in order to a!oid any s,urious effect of the filtering ,rocedure at the series edge *the =Dday length M >XTc is o!ersi?ed for security+. Then the deri!ed 9day arrays are a,,ended to the ,re!iously u,dated 9day dataset fro) a retros,ect date that is >Ddays earlier than the beginning of the ne" data. -o"e!er the last 1Ddays of the ti)e series *last 2 ste,s of the S%&' ti)e base+ )ay still be subject to the s,urious effect )entioned abo!e, and the correction for this ta$es ,lace during the ne2t u,date session of the dataset. This is "hy e!en if no ne" 9(day ste, is created after an u,date of the 1(day data, the current 9(day ste, is )odified >9

for a better data 4uality. Fro) this, the surface !elocity dataset 4uality is conse4uently i),ro!ed as "ell. =.1.0. SS:5 "ind ,roduct *historical data only+ The SS:5 "ind data "ere do"nloaded and ,rocessed only once. The source data are a!ailable through a 1 1S ser!er at htt,/..dods.j,l.nasa.go!.dods(bin.n,h( hdf.,ub.oceanA"ind.ss)i.atlasAss)iA!er1D.data.le!el0.9A9day. , and are located in the datasets.C531.SS:5 directory. They are initially arranged on an e2act 9(day grid. &fter the source data are do"nloaded, they are ,rocessed si)ilarly as the #ui$Scat data. 3ote that a large nu)ber of s,atial ga,s had to be filled at each ti)e ste,. =.1.>. 'eynolds and S)ith SST ,roduct The source data are initially gridded on an e2act H(day ti)e base, and on the sa)e s,atial grid as S%&'. 1ata are inter,olated on the 9(day S%&' ti)e base, and s)oothed through a 2D(day lo"(,ass filter. =.1.9. %LS *%3ES+ SS- ,roducts The source data are initially gridded successi!ely on a H(day and 9(day ti)e base and on a s,atial grid "ith !arying s,acing *e.g. Le Traon et al., 1EE9+. nce they are do"nloaded, the data are si),ly inter,olated on the S%&' grid. 3o te),oral filtering is a,,lied. There are three different ,roducts that are do"nloaded through the S%&' ,rocessing *in ,arallel off(scheduled session+, but t"o distinct fields are built/ ne ano)aly field fro) ctober 1EE2 to ,resent, to "hich is added the '5 :1T *see =.1.1+. This re4uires to do"nload se,arately the historical dataset *once+ fro) 1EE2 to 2DD1, and the data fro) 2DD1(,resent "hich are u,dated in near(real ti)e. ne absolute field *&1T+ fro) 2DD1 to ,resent, u,dated in near(real ti)e. 3o historical data e2ist for this dataset so far. &t the )o)ent, these datasets are ,ublicly a!ailable "ith a 0D(day delay, but a H(day delay )ay be allo"ed. -o"e!er, the only ,rotocol for do"nloading those H(day delayed data )ay be through Ft,. %ontact "eb)aster at %LS(&S5S . =.2. Surface current data ,roduct The calculation of the surface currents fro) the SS-, "ind and SST data is described in Bonjean and lagerloef, 2DD2, and the )ethod is i),le)ented as such in the S%&' ,rocessing syste).

>G

&dditionally, a s)oothed ,roduct is co),uted, through a >Dday lo"(,ass filter, and a s,atial s)oothing ,rocedure of ;oi< ty,e "ith 1D]longitude X 2]latitude decorrelation scales.

>H

E. Soft"are 1ocu)entation The list of ,rocedures belo" is not co),lete, but should ,ro!ide a good sense of "hat ty,e of ,rogra) constitutes the ,rocessing syste). So)e docu)entation is gi!en in each ,rogra) file in the co),uter. aut"+ati#9"s#ar9pr"#ess9:$ called fro) the crontab file ,ur,ose/ )anages and calls general ,rocedures for surface !elocity ,rocessing, file creation and transfer

pr"#ess9all9:$ called fro) auto)aticAoscarA,rocessA9d ,ur,ose/ 6erfor)s all re4uired ,rocessing, do"nloads data, structures data and co),letes !elocity ,rocessing. 1is,lays o!erall status )essage. "s#ar'ile9:$ called fro) auto)aticAoscarA,rocessA9d ,ur,ose/ creates a 3etcdf !elocity file for S%&' "eb site in the current directory and co,ies the file to the 3ES15S local ft, site. 1is,lays status )essage.

pr"#ess9us'ssh9:$

>=

called fro) ,rocessAallA9d ,ur,ose/ FT6(do"nloads and ,rocesses gridded satellite sea surface height ano)aly data fro) 7SF. 1is,lays status )essage.

pr"#ess9sAs.in$9:$ called fro) ,rocessAallA9d ,ur,ose/ 1EE2( pr"#ess9re*sst9:$ called fro) ,rocessAallA9d ,ur,ose/ 1o"nloads and ,rocesses 'eynolds and S)ith SST "ee$ly files fro) L1E "ebsite. 1is,lays status )essage. pr"#ess9&el9us'9sAs9re*9:$ inde,endent fro) the S%&' cron job *not called by any routine+ ,ur,ose/ )essage. MABE9VE39:D %alled fro) ,rocessAallA9d ,ur,ose/ %alculates the !elocity fro) SS-, "ind and SST fro) the last u,dated ,eriod to a s,ecified ,eriod *today+. Generates gra,hics for )aintenance chec$. %reates &LLAJ B.out log file. >E 6rocesses the surface !elocity calculation fro) the satellite data, s,ecifically using 7SF, SS:5.#ui$Scat, and 'eynolds ,roducts. 1is,lays status do"nloads and ,rocesses the regular #ui$Scat dataset, tries the 3'T #ui$Scat dataset if the regular dataset is not u,dated on the re)ote site, builds a ,resent dataset using the SS:5 data. 1is,lays status )essage.

MABE9OSCARFI3E9:D %alled fro) oscarfileA9d ,ur,ose/ %reates 3etcdf file for trans)ission to S%&' "ebsite. %reates S%AJ B.out log file. PREP9USFSSH9:D %alled fro) ,rocessAusfsshA9d ,ur,ose/ &rranges the 7SF SS- data in yearly files. Generates gra,hics for )aintenance chec$. %reates J B.out log file. PREP9=SCWIND9:D %alled fro) ,rocessAs4s"indA9d ,ur,ose/ 1 1S(do"nloads and ,rocesses the regular #ui$Scat dataset. Generates gra,hics for )aintenance chec$. %reates J B.out log file.

6rocessing Structure/ the follo"ing is a list of all the scri,ts and the scri,ts and.or functions that they call. The ;9d< suffi2 signifies that the routine uses the fi!e(day data base for ,rocessing. auto)aticAoscarA,rocessA9d ,rocessAallA9d oscarfileA9d ,rocessAallA9d ,rocessAusfsshA9d ,rocessAs4s"indA9d ,rocessAreysstA9d 9D

:&KEASELA91 oscarfileA9d :&KEA S%&'F5LEA91 ,rocessAusfsshA9d 6'E6A7SFSS-A91 ,rocessAs4s"indA9d 6'E6A#S%C531A91 6'E6A#S%C531A91A3'T 6'E6AS#SC531A91 ,rocessAreysstA9d 6'E6A'EFSSTA91 :&KEASELA91 1&TE2J7L5&3 1&F37: B7TTE'C 'T-AF5LT:0 535TF5GL6 J7L5&321&TE J7L5&32F' LESELS L 3GAL&TA6L TS 6A53TE'62 S6-E'5%&LA15ST T5:EAG'&6:&KEA S%&'F5LEA91 1&TE2J7L5&3 J7L5&321&TE 91

L&TAL&BELS L 3GAL&BELS :E&3&3 ::0 6'E6A7SFSS-A91 1&TE2J7L5&3 1&F37: 535TF5GL6 J7L5&321&TE L 3GAL&TA6L TS :E&3&3 ::0 F'2J7L5&3 6'E6A%LSSL&A91 1&TE2J7L5&3 1&F37: 535TF5GL6 J7L5&321&TE LESELS L 3GAL&TA6L TS :2101 :0121 T5:EAG'&6F'2J7L5&3 6'E6A%LS&1TA91 1&TE2J7L5&3 1&F37: 535TF5GL6 J7L5&321&TE LESELS L 3GAL&TA6L TS 92

:2101 :0121 T5:EAG'&6F'2J7L5&3 6'E6A#S%C531A91 1&F37: B7TTE'C 'T-AF5LT:0 1&TE2J7L5&3 535TF5GL6 J7L5&321&TE L 3GAL&TA6L TS :2101 :0121 6A53TE'62 T5:EAG'&6F'2J7L5&3 6'E6AS#SC531A91 535TF5GL6 J7L5&321&TE LESELS L 3GAL&TA6L TS T5:EAG'&6F'2J7L5&3 6'E6A'EFSSTA91 B7TTE'C 'T-AF5LT 1&TE2J7L5&3 1&F37: 535TF5GL6 J7L5&321&TE 90

LESELS L 3GAL&TA6L TS :2101 :0121 T5:EAG'&6F'2J7L5&3

'eferences
&tlas, '., '. 3. -off)an, S. %. Bloo), J. %. Juse), and J. &rdi??one, 1EEG/ & )ultiyear global surface "ind !elocity dataset using SS:.5 "ind obser!ations, Bull. &)er. :eteor. Soc., HH, =GE( ==2. Bonjean, F., and G. S. E. Lagerloef, 2DD2/ 1iagnostic )odel and analysis of the surface currents in the Tro,ical 6acific cean. J. 6hys. ceanogr., 02, 2E0=(2E9>. Lagerloef, G.S.E., G.T. :itchu), '. Lu$as, and 6.6. 3iiler. J. Geo,hys. 'es., 1D>, 20010( 2002G. Le Traon, 6.(F., 6. Gas,ar, F. Bouyssel, and -. :a$h)ara, 1EE9/ 7sing To,e2.6oseidon data to enhance E'S(1 data, J. Atmos. Oceanic Technol., 12 *1+, 1G1(1HD. 6egion, 6. J., :. &. Bourassa, 1. :. Legler, and J. J. JBrien, 2DDD/ bjecti!ely(deri!ed daily ^"inds^ fro) satellite scattero)eter data. Mon. Wea Rev., 12=, 019D(01G= 'io, :.(-. and F. -ernande?, 2DD>/ & :ean 1yna)ic To,ogra,hy co),uted o!er the "orld ocean fro) alti)etry, in(situ )easure)ents and a geoid )odel. J. Geophys. Res. *acce,ted+

9>

1D. %ontact list This is for technical )aintenance only. The contacts bet"een brac$ets can be reached as a last resort for finding direction regarding technical infor)ation.

Ta1le :! #"nta#t list

%ontact Fabrice Bonjean *6.5.+, John Gunn *or Gary Lagerloef+ Cilla Phu *3ancy Soreide, 6.5.+ %hristina -olland *Gary :itchu), 6.5.+ Fan Gu *:ar$ Bourassa, 6.5.+ Ceb)aster

e)ail bonjeanQesr.org gunnQesr.org lagerQesr.org Cilla.PhuQnoaa.go! 3ancy.SoreideQnoaa.go! )unchQseas.)arine.usf.edu )itchu)Qseas.)arine.usf.edu guQcoa,s.fsu.edu bourassaQcoa,s.fsu.edu

1ata ,roduct Location S%&', ES', Seattle, C& S%&'(55 S%&' 3 &&.6:EL, Seattle, data and C& "eb ser!er 7SF gridded 7SF, St(6etersburg, FL alti)etry FS7.% &6S, Tallahassee, FL 5'5.3 &&.%olu)bia 7., 6alisades, 3F %LS.%3ES, Toulouse, France 3'L, Stennis S,ace %enter, :S

Ceb)aster

Kir$ Chit)er *Gregg Jacobs+

% &6S gridded "ind !ectors hel,Qiri.colu)bia.edu 'eynolds and S)ith SST !ersion 2 %LS aviso@cls.fr "eb)asterQa!iso.oceanobs.co) *%3ES+ gridded alti)etry 3'L Whitmer@nrlssc.navy.mil gridded jacobsQnrlssc.na!y.)il alti)etry

99