US 20100305860Al

(19) United States

(12) Patent Application Publication (10) Pub. No.: US 2010/0305860 A1
Bishop (43) Pub. Date: Dec. 2, 2010

(54) FILTERED MODEL OUTPUT STATISTICS Publication Classi?cation

(FMOS) (51) Int. Cl.
(75) Inventor: Craig H. Bishop, Monterey, CA G01 W 1/10 (200601)
(Us) G06F 19/00 (2006.01)
G06F 17/18 (2006.01)
Correspondence Address:
NAVAL RESEARCH LABORATORY (52) US. Cl. ........................................... .. 702/3; 702/ 179
ASSOCIATE COUNSEL (PATENTS)
CODE 1008.2, 4555 OVERLOOK AVENUE, S.W.
WASHINGTON, DC 20375-5320 (US) (57) ABSTRACT
(73) Assignee: The Government of the United A computer-implemented method to provide stabilized and
states “America as represented spatially smooth regression coef?cients for Weather forecast
by the seeretary of the Navy’ error correction from small training data sets. In accordance
Washington’ DC (Us) With the present invention, an MOS estimate of the regression
coe?icient calculated from a small data set can be optimally
(21) Appl. No.: 12/789,497 combined With a smooth prior estimate of the regression
_ coe?icient, an estimate of the spatial error covariance of that
(22) Flled? May 28’ 2010 prior estimate, and an estimate of the spatial error covariance
Related U‘s‘ Application Data of the MOS estimate. The result is a ?ltered MOS (FMOS)
regression coef?cient Which can be used to more accurately
(60) Provisional application No, 61/ 181,686, ?led on May estimate and correct errors in Weather forecasts even using
28, 2009' only small data sets.

Simulated true state of 2-dimensional wind component mS~1

3.5
7000 km
Patent Application Publication Dec. 2, 2010 Sheet 1 0f 9 US 2010/0305860 A1

Simulated true state of 2-dimensional wind component mS-1_

7000 km

0 M 7000 km
Patent Application Publication Dec. 2, 2010 Sheet 2 0f 9 US 2010/0305860 A1

7000 km

0
7000 k
0 FIG. 2A m
Prior Art

MOS a error

7000 km

FIG. 28
Prior Art
Patent Application Publication Dec. 2, 2010 Sheet 3 0f 9 US 2010/0305860 A1

7000 km

7 OO0 km

7000 km
FIG. 2D
Patent Application Publication Dec. 2, 2010 Sheet 4 0f 9 US 2010/0305860 A1

Column of Ga (est)
70 O 0 km

7000 km
FIG. 2E

Column of Ra (est)
7O00 km

7000 km
FIG. 2F
Patent Application Publication Dec. 2, 2010 Sheet 5 0f 9 US 2010/0305860 A1

FMOSa
7000 km

0 7000 km
FIG. 2G

FMOS a error

7000 km

0 7000 km
FIG. 2H
Patent Application Publication Dec. 2, 2010 Sheet 6 0f 9 US 2010/0305860 A1

MUS b
7000 km

7000 km
FIG. 3A

MOS b error
7 O O O km

FIG. 3B
Patent Application Publication Dec. 2, 2010 Sheet 7 0f 9 US 2010/0305860 A1

TRUE b

..

FIG. 3C

7000 km . up

7000 km
FIG. 3D
Patent Application Publication Dec. 2, 2010 Sheet 8 0f 9 US 2010/0305860 A1

Column of Gb (est) h
7000 km

0
7000 k
0 FIG. 3E m

Column of Rb (est)
7000 km

0
7000 km
FIG. 3F
Patent Application Publication Dec. 2, 2010 Sheet 9 0f 9 US 2010/0305860 A1

FIG. 36

FIVIOS b error
7000 Km -

FIG. 3H
US 2010/0305860 A1
FILTERED MODEL OUTPUT STATISTICS
(FMOS)
CROSS-REFERENCE
[0001] This application claims the bene?t of priority based
on US. Provisional Patent Application No. 61/181,686 ?led
on May 28, 2009, the entirety of Which is hereby incorporated
by reference into the present application.
TECHNICAL FIELD
[0002] The present invention relates to Weather forecasting
and statistical corrections of forecast errors.
BACKGROUND
[0003] Weather forecasts are of high value to a Wide range
of activities in agriculture, energy, transportation, defense
and supply chain management. See K. Rollins and J.
ShaykeWich, “Using Willingness-to-pay to assess the eco
nomic value of Weather forecasts for multiple commercial
sectors,”MeZe0r0l. Appl. Vol. 10, pp. 31-38 (2003);Y. Zhu, Z.
Toth, R. Wobus, D. Richardson, and K, Mylne, “The Eco
nomic Value of Ensemble-Based Weather Forecasts,” BAMS,
January 2003, pp. 73-83. Of particular value are Weather
forecasts tailored for the particular Weather related decisions
that individuals, companies and organiZations must make.
Such tailored Weather forecasts are often provided by private
Weather companies such as those listed on the government
Web site http://WWW.Weather.gov/im/more.htm.
[0004] The starting point for these Weather forecasts is a
computer simulation or Numerical Weather Prediction
(NWP) in Which approximations to the equations governing
the evolution of the atmosphere propagate a recently esti
mated Weather state forWard in time. In general, substantial
improvements to forecasts made in this Way can be achieved
by comparing histories of forecasts With corresponding his
tories of verifying observations. By mathematically modeling
the differences betWeen the NWP and observations, it has
been shoWn that one can produce a forecast that is signi?
cantly better than that from NWP.
[0005] Glahn and LoWry describe hoW the theory of mul
tivariate linear regression can be used to correct systemati
cally predictable aspects of Weather forecast models such as
the NWP, and refer to their Weather forecast error corrector as
Model Output Statistics (MOS). See H. R. Glahn and D. A.
LoWry, “The Use of Model Output Statistics (MOS) in Obj ec
tive Weather Forecasting,” J. Appl. Meleon, Vol. 11, Issue 8,
pp. 1203-1211 (1972). The MOS forecast error corrector is
estimated from a historical record of forecasts together With
corresponding verifying observations. Wilson and Vallée
(2002) suggest that the historical record needs to contain at
least tWo years Worth of daily Weather forecasts. L. J. Wilson
and M. Vallée, “The Canadian Updateable Model Output
Statistics (UMOS) System: Design and Development Tests,”
Wea. Forecasting, Vol. 17, Issue 2, pp. 206-222 (2002).
[0006] MOS methods assume that on the kth day/ event, the
ith model forecast variable fl-k is related to the corresponding
verifying analysis/observation variable yl-k via the stochastic
relation
Where the true regression coe?icients al- and bl- of the model
forecast variable fl-k can be considered to be the “slope”
parameter and the “intercept” parameter, respectively, of a
Dec. 2, 2010
plot of the relation betWeen f and y, and el-k is a random
number associated With the kth forecast that is statistically
independent of yl-k.
[0007] With a ?nite sample of kIl, M realiZations, using
MOS methods one can obtain estimates al-MOS and bl-MOS of
the regression coe?icients al. and bl. using the relations
(2)
» H
Where fl-k is the ith model forecast variable and yl-k is the
verifying analysis/observation variable in Equation (1) above
and and y,- are the average values over M realiZations.
[0008] If tWo years of historical data are available for the
forecast model of interest, MOS can be relied on to provide
signi?cant forecast improvements. HoWever, in the continu
ing effort to improve Weather forecasting models, signi?cant
changes are frequently made to the forecasting model. The
MOS equations developed for an old model may be entirely
inappropriate for the changed neW model. Consequently,
after every model change, neW MOS equations need to be
developed. For example, typical model changes have
included increases in model resolution, changes in the repre
sentation of sub sub-grid scale physics, and changes in the
forcings associated With radiation.
[0009] If su?icient manpoWer and computer resources
Were available, a historical record of the performance of the
neW system required for neW MOS equations could be gen
erated fairly quickly by running the neW system on historical
data. HoWever, such resources are generally unavailable to
the responsible parties. Instead, a historical record of tWo
years is usually obtained by archiving forecasts and the cor
responding verifying analyses and forecasts in real time.
Using such a method, a tWo-year historical record Would
require tWo years’ time to compile.
[0010] This state of affairs led Wilson and Vallée (2002),
supra, to propose an Updateable Model Output Statistics
(UMOS) system that blended regression equations based on
earlier versions of the model With equations based on more
recent versions of the model. UMOS implicitly assumes that
signi?cant model changes are typically separated by more
than tWo years. HoWever, this assumption is not satis?ed at
many forecasting centers. For example, the limited area
Weather forecasting models of the US. Navy are typically
deployed in a speci?c theatre for a period betWeen tWo Weeks
and tWo years. In principle, neW MOS equations Would need
to be developed for each unique theatre in Which the forecast
ing model is deployed. Hence, the Navy’s deployment peri
ods are too short for the estimated regression coe?icients to
stabiliZe, and any attempt to estimate the regression coef?
cients using MOS or UMOS With such a small data set Will
result in very noisy regression coe?icients.
[0011] One alternative Would be to assume that regression
coef?cients Were identical in various sub-regions of the
model. This Was assumed by, for example, M. J. Schmeits, K.
J. Kok, and D. H. P. VogeleZang, “Probabilistic Forecasting of
US 2010/0305860 A1
(Severe) Thunderstorms in the Netherlands Using Model
Output Statistics.” Wea. Forecasting, Vol. 20, Issue 2, pp.
134-148 (2005).Although this approach Will give stable coef
?cients for small data training sets it does not converge to the
(superior) MOS coef?cients in the limit of in?nite training
data. In addition, the MOS equations are likely to have erro
neous jumps betWeen sub-regions.
[0012] Thus, there is a need for a method for producing less
noisy MOS regression coef?cients that canbe used With small
data sets.
SUMMARY
[0013] This summary is intended to introduce, in simpli?ed
form, a selection of concepts that are further described in the
Detailed Description. This summary is not intended to iden
tify key or essential features of the claimed subject matter, nor
is it intended to be used as an aid in determining the scope of
the claimed subject matter. Instead, it is merely presented as
a brief overvieW of the subject matter described and claimed
herein.
[0014] The present invention provides a computer-imple
mented method to provide stabiliZed and spatially smooth
regression coef?cients for Weather forecast error correction
from small training data sets. In accordance With the present
invention, the MOS estimates al-MOS and bl-MOS of regression
coef?cients calculated from a small data set can be optimally
combined With a smooth prior estimate of the regression
coe?icient, an estimate of the spatial error covariance of that
prior estimate, and an estimate of the spatial error covariance
of the MOS estimate to produce a set of Filtered MOS
(FMOS) regression coef?cients aFMOS and bFMOS:
Where
are n-vectors listing all the estimates of alMOS and bl-MOS for
iIl, . . . , n, n being the number of grid points de?ning the ?eld
of interest aprior and bp’ior are n-vectors listing prior estimates
or guesses of the regression coef?cients a and b over that ?eld,
Ga and Gb de?ne the error covariances of a‘’ '1'” and bprior and
Ra and Rb de?ne the error covariance of aMOS and bMOS.
[0015] These ?ltered regression coef?cients aFMOS and
bFMOS obtained in accordance With the present invention can
be used to more accurately estimate and correct errors in
Weather forecasts even using only small data sets.
BRIEF DESCRIPTION OF THE DRAWINGS
[0016] FIG. 1 depicts an exemplary random random
meridional Wind ?eld (v) generated using simulated observed
idealiZed Weather ?elds.
[0017] FIGS. 2A-2H illustrate aspects of the differences
betWeen the MOS and FMOS estimates of the slope coef?
cient a obtained from 16 trials/events. FIGS. 2A and 2B
illustrate an exemplary set of MOS estimates (prior art) of the
true a-?eld and the error in this estimate, respectively. FIG.
2C shoWs a plot of an exemplary true a-?eld. FIG. 2D shoWs
the true spatial covariance function of the a-?eld. FIGS. 2E
Dec. 2, 2010
and 2F illustrate estimated error covariances of the prior and
MOS estimates of a, respectively. FIGS. 2G and 2H, respec
tively, illustrate an exemplary set of Filtered MOS (FMOS)
estimates of a obtained in accordance With the present inven
tion and the errors in the FMOS estimates as compared to the
true a-?eld.
[0018] FIGS. 3A-3H illustrate corresponding aspects of the
differences betWeen the MOS and the FMOS estimates of the
slope coe?icient b. Thus, FIGS. 3A and 3B, illustrate an
exemplary set of MOS estimates (prior art) of the true b-?eld
and the error in this estimate, respectively. FIG. 3C shoWs a
plot of an exemplary true b-?eld. FIG. 3D shoWs the true
spatial covariance function of the b-?eld. FIGS. 3E and 3F
illustrate estimated error covariances of the prior and MOS
estimates of b, respectively. FIGS. 3G and 3H, respectively,
illustrate an exemplary set of FMOS estimates of b obtained
in accordance With the present invention and the errors in the
FMOS estimates as compared to the true b-?eld.
DETAILED DESCRIPTION
[0019] The aspects and features of the present invention
summariZed above can be embodied in various forms. The
folloWing description shoWs, by Way of illustration, combi
nations and con?gurations in Which the aspects and features
can be put into practice. It is understood that the described
aspects, features, and/or embodiments are merely examples,
and that one skilled in the art may utiliZe other aspects,
features, and/or embodiments or make structural and func
tional modi?cations Without departing from the scope of the
present disclosure.
[0020] The present invention provides a computer-imple
mented method for statistically deriving Weather forecast
improvement functions from small training data sets. As Will
be appreciated by one skilled in the art, a method for provid
ing stabiliZed and spatially smooth regression coef?cients for
Weather forecast error correction in accordance With the
present invention can be accomplished by executing one or
more sequences of instructions contained in computer-read
able program code read into a memory of one or more general
or special-purpose computers con?gured to execute the
instructions, Wherein MOS regression coef?cients that have
large errors because of the paucity of training data are trans
formed into a set of ?ltered modeled output statistics (FMOS)
regression coef?cients using information about the spatial
covariance of the prior estimate of the coef?cients and the
spatial error covariance of the MOS estimate of the coef?
cients. The addition of the spatial covariance information
causes the FMOS coef?cients to be much more accurate than
the noisy MOS coef?cients from Which they Were derived.
[0021] As described in more detail beloW, the FMOS tech
nique of the present invention stabiliZes and spatially
smoothes noisy regression coef?cients produced by conven
tional MOS techniques using small data sets by optimally
combining a data-poor MOS estimate With a prior estimate
based on an assumption, for example, that model forecasts are
unbiased. To optimally combine these tWo estimates, infor
mation about the spatial covariance of the prior estimate of
the coef?cients and the spatial error covariance of the MOS
estimate of the coef?cients is used. An estimate of the spatial
error covariance G of the prior estimate can be obtained from
a spectral analysis of the (noisy) difference betWeen the data
poor MOS estimate and the prior estimate, and an estimate of
the spatial error covariance R of the data-poor MOS estimate
can obtained from a spatial analysis of the equations govern
US 2010/0305860 A1
ing the grid-point Wise error variance of the MOS equations.
In accordance With the present invention, these error covari
ances G and R are combined With the MOS coef?cients to
produce a set of ?ltered MOS regression coef?cients Which
can be used to more accurately estimate and correct errors in
Weather forecasts even using only small data sets.
[0022] As noted above, at the kth event (e.g., an observation
day) a forecast fl-k at the ith grid point from the true state yl-k can
be estimated from Equation (1) set forth above and replicated
here for convenience:
ik:aiyik+bi+eik' (1)
Where the regression coef?cients al- and bl- of the modeled
forecast variable fl-k can be considered to be the “slope”
parameter and the “intercept” parameter, respectively, of a
plot of the relation betWeen f and y, and el-k is a random
number associated With the kth forecast that is statistically
independent of yl-k. Although Equation (1) expresses the fore
cast as function of the observation rather than the observation
as a function of the forecast, one skilled in the art Would
readily understand that the methods described herein can be
applied equally Well to counterparts of Equation (1) that
express the observation as a function of the forecast. It may be
noted, hoWever, that the form of Equation (1) presented
herein may be preferred When one is attempting to produce
reliable probabilistic forecasts from ensemble forecasts. See
C. H. Bishop and K. T. Shanley, “Bayesian Model Averag
ing’s problematic treatment of extreme Weather and a para
digm shift that ?xes it,” Mon. Wea. Rev. 136, pp. 4641-4652
(2008).
[0023] Also as noted above, Equation (1) contains a set of
estimated regression coef?cients al-MOS and bl-MOS can be cal
culated using Equation (2) previously presented above 1 and
replicated here for convenience:
a. = —,
[0024] HoWever, there often are signi?cant errors in the
estimated regression coef?cients aiMOS and biMOS, particu
larly in cases Where only a small data set is present.
[0025] The present invention provides a method to improve
on the MOS estimates of the regression coef?cients al- and bi,
Which in turn can be used to provide more accurate Weather
forecasts and modeling, even in cases Where only small data
sets are available. In accordance With the present invention,
each MOS estimated coe?icient can be combined With a
smooth prior estimate of that regression coef?cient, an esti
mate of the spatial error covariance of that prior estimate, and
an estimate of the spatial error covariance of the MOS esti
mate.
[0026] The result is a set of ?ltered MOS (FMOS) regres
sion coef?cients aFMOS and bFMOS:
aFMOS: apn'or: GA Ga:R6941 ( aM0s_ aprior)
bFMOS: bpn'or: Gb( Gb:R1941 (bM0s_bprior)
Dec. 2, 2010
are n-vectors listing all the estimates of aiMOS and biMOS for
iIl, . . . , n, n being the number of grid points de?ning the ?eld
of interest aprior and bp'ior are n-vectors listing prior estimates
or guesses of the regression coef?cients a and b over over that
?eld, Ga and Gb de?ne the error covariances of a‘’ "by and bPn-Or
and Ra and Rh de?ne the error covariance of aMOS and bp’ior as
described in more detail beloW. The improved regression
coef?cients aFMOS and bFMOS can be used to more accurately
estimate and correct errors in Weather forecasts even using
only small data sets.
[0027] A method for estimating aFMOS and bFMOS in accor
dance With the present invention Will noW be described in
detail.
[0028] As described above, the error covariances Ga and Gb
give the error covariance of the prior estimates ap’ior and
bprior. n accordance With the invention, Ga and Gb can be
de?ned as
Where a and b are n-vectors listing the true values of the
coef?cients al- and bl- and ap’ior and bp’ior are n-vectors listing
prior estimates or guesses of the regression coe?icients. As
noted above, after a change in the Weather forecast model, one
Would typically assume that (aP'i°')T:[l,l, . . . , l] and (bprior)
T:[0,0, . . . , 0] because, in large measure, the purpose of
model changes is to make forecasts unbiased and to make the
(Z)
approximation fl-kzyl-k as accurate as possible. Thus, in the
examples presented here, ap’ior and bprior Will be based on the
assumption that the model is unbiased, Where all of the ele
ments of ap’ior are set equal to unity and all of the elements of
bprior are set equal to Zero. HoWever, in general aprior and
bprior can represent any prior guess of the regression coef?
cients a and b.
[0029] An exemplary method for estimating the covari
ances Ga and Gb in accordance With the present invention Will
noW be described. Since the estimation method is the same for
both Ga and G1,, only the method for estimating Ga Will be
described here, as one skilled in the art Would understand hoW
to estimate Gb from the description herein. In addition, one
skilled in the art Would readily appreciate that other covari
ance matrices Ga and Gb and methods for estimating thereof
also can be used, and any appropriate covariance matrices Ga
and Gb and/or methods for estimating thereof can be used in
estimating regression coef?cients aFMOS and bFMOS in accor
dance With the present invention.
[0030] Let an n><n matrix E list the real set of discrete
orthonormal sinusoidal/cosinusoidal basis functions for the
domain of interest and ET be the transpose of E. These basis
functions can be readily accessed via Fast Fourier Transform
subroutine libraries knoWn in the art. If the domain under
consideration lies on a sphere, then a basis of spherical har
monics should be used rather than sinusoids. If the assump
tion of periodicity is invalid on a ?nite domain then a pure
(3) sinusoid or pure cosinusoid basis should be used.
US 2010/0305860 A1 Dec. 2, 2010

[0031] It can be assumed that [0037] We also note that from Equation (6)
Ga:E(diag(f))ET (5)
[0032] The term diag(f) in Equation (5) is a diagonal matrix
Whose non-Zero elements are given by

Where the subscripts k and 1 refer to a sinusoidal basis func

tion With Wavenumber k in the x-direction and Wavenumber l [0038] so that using the relations in Equation (10) in Equa
in the y-direction. tion (9) gives us
[0033] The broader the covariance functions associated
With the covariance matrices Ga and G1,, the smaller the his
torical data set required for FMOS to yield accurate regres
sion coef?cients. This is because When these covariance func
01 "k
M5
tions are broad, Equation (3) can remove the noise from the and
MOS estimates of a and b by spatially averaging the noisy
MOS estimates. The breadth of the covariance functions 61 (12)
depends on the positive de?nite parameter b used in deriving
the diagonal matrix elements fkJ, With a larger value of b k2 +
leading to broader covariance functions and a smaller value of
b leading to tighter covariance functions. The parameter B
Z2] .2 18W
used in de?ning the diagonal matrix elements fk,Z determines
the amplitude of the covariance functions, With larger
(smaller) B values giving larger (smaller) amplitudes.
[0034] The prime objective of the estimation procedure
described here is to choose B and b so that the vector (aP”°’—a)
Would appear to be a random draW of an n-vector from a [0039] The derivatives
normal distribution With covariance Ga. In other Words, f
should be chosen so that the assumption
01 d 01
?an m2)
Where Q is some random normal vector With mean 0 and
covariance I, is as plausible as possible. can then be used in any suitable minimization algorithm to
To do this, We begin by taking the transform ?nd the values of B and b2 that minimiZe J. With B and b
de?ned in this Way, the optimum error covariance GGIE (diag
(f)) ET can then be estimated using Equations (5) and (6)
Where the symbol O indicates the elementWise vector prod described above. As noted above, Gb can be estimated in the
uct. The transform yields the vector 0t of spectral coef?cients same Way as can Ga, and so the procedure for estimating Gb
de?ning aprior and aMOS. Since 0t is real and dIOtOOt, there are Will not be repeated here.
no negative elements in d. For a tWo-dimensional domain, [0040] As can be seen from Equation (3) and as described
each element dk,Z gives the square of the coe?icient of the above, in addition to Ga and G1,, the method for estimating
sinusoid With Wavenumber k in the x-direction and Wavenum improved forecast error regression coef?cients aFMOS and
ber l in the y-direction. bFMOS in accordance With the present invention also utiliZes a
[0036] To make the assumption in Equation (7) plausible, second set of error covariance equations Ra and Rb, Which
the parameters B and b in Equation (6) can be chosen to de?ne the error covariance of the MOS estimates of the
minimize the function regression coef?cients aMOS and bMOS:
a

Me Ms
l (13)
by setting [0041] It should be noted that unlike the error covariances
Ga and G1,, each of the error covariance matrices Ra and Rb
must be estimated separately.
[0042] As described in more detail beloW, these spatial
and error covariance matrices can be estimated from an analysis
of the knoWn equations de?ning the error variance of esti
mates of the regression coef?cients at each grid point. The
aspects of this analysis relating to spatial covariances are neW
and are not found in the prior art. In addition, the magnitude
of the elements in Ra and Rb tend to Zero as the length of the
US 2010/0305860 A1 Dec. 2, 2010

historical record M used as a training data set tends to in?nity in?nity. HoWever, When the number of training data sets M is
so that in this limit, bFMOSIbFMOSIb and aFMOSIaMOSIa. signi?cantly smaller than the number of grid points of inter
[0043] An exemplary method for estimating Ra and Rb is est, this approximation of the regression coe?icient Rb pro
presented below. However, as in the case of Ga and G1,, other vides results that are far superior than that obtained using
forms of Ra and Rb and methods for estimating thereof may conventional methods.
exist and any appropriate form and/or method for estimation [0050] As noted above, Ra and Rb must be estimated sepa
may be used in the method for estimating the regression rately, and so having obtained an estimate of Rb, We can noW
coef?cients aFMOS and bFMOS in accordance With the present
invention. estimate Ra.
[0044] We begin With the estimation of Rb. From Equations [0051] As described above, the forecast variable is
(1) and (3) set forth above, related to the corresponding verifying analysis/observation
variable y,” by fi’":aiyi’"+bi+‘Ql-'". Therefore,
since biMOSfi—aiyi, byes-big, Where the overbar indicates a
sample mean over the M training events, e. g., Where the barred variables and y, are mean variables as
described above.
[0052] Rearranging Equation (17), We get
ai:[(12-12)(n-YO-‘QKn-YOlKn-i? (18)
[0053] From Equation (2) described above, the value of
[0045] In conventional methods, the element {RbL-j on the aiMOS can be derived:
ith roW and jth column of Rb can be estimated using the values
of biMOS and (byes-bl.) from Equation (14) above: a I'MOS: [Uri-Ti) Orin/(M502, (l 9)
[0054] Using Equations (1 8) and (l 9) above, We can derive
a-MOS—a-:

and therefore can derive Ra:

Where the triangular brackets indicate the expectation opera

tor knoWn in the art.
[0046] Using the approximation <Qhd imzfYLal-MOSyi’Lbf
MOS in Equation (1 5), an estimate of the covariance betWeen
the error of the bias estimate at the ith grid point and the bias
estimate at the jth grid point can be obtained.
[0047] The approximation given by Equation (15) is a
trivial extension of formulae Well knoWn in the ?eld of mul
tivariate regression. HoWever, this approximation is
extremely poor When the number of data sets M is smaller
than the number of grid-points in the horiZontal domain, and
unfortunately, this data-poor state of affairs is What Would [0055] Using the facts that (yZ-MCZ" Hyf'C/“FO and
typically be expected in meteorological applications. (‘QZ-M‘QJ-k) :6mk( ‘QZ-M‘QJ'" ), Equation (21) simpli?es to
[0048] Consequently, in accordance With the present inven
tion, Rb is assumed to be diagonal in spectral space and can be
estimated using the relation E (22)
E

EM:
[0049] Note that in both of Equations (15) and (16), the
magnitude of the elements of Rb approaches Zero as M tends
EIH m: l
to in?nity; i.e., the error in the MOS coef?cients aMOS and
bMOS as compared to the true regression coef?cients a and b
goes to Zero as the number of training events M goes to
US 2010/0305860 A1
[0056] Using Equation (15) in Equation (22) gives
at,- -
[0057] To get Equation (23) into matrix form, We ?rst
de?ne the matrix Y such that
24
M (W-WW’f-W ( )
mil: 2T
[0058] Let the diagonal matrix D list the inverse of the
diagonal elements of Y such that
D:{di?g(Y)}’l (25)
[0059] Equation (23) may then be reWritten in the matrix
form
Where O denotes the elementWise matrix product. Note that
Equation (26) states that each element of Ra is directly pro
portional to a corresponding element of Rh. Consequently,
just as the elements of Rb tend to Zero as the number of
training data events M tends to in?nity, the elements of Ra Will
also tend to Zero as M a0O.
[0060] Thus, in accordance With the present invention, With
the MOS regression coe?icients, the prior estimates of the
regression coef?cients, and the covariances Ga G1,, Ra, and Rb
knoWn, an improved set of regression coef?cients al. and bi,
i.e., a-FMOS and bl-FMOS, can be obtained in accordance With
1 coef?cients using the MOS method according to Equation (2)
Equation (3) set forth above:
aFMOS:apn'or: GAGQIRQA (a?/IOS_aprior),
[0061] These improved regression coef?cients in turn can
be used to provide an improved statistical relationship
betWeen the forecast at the ith grid point fl-k and the observa
tion yik at the ith grid point from the FMOS coef?cients
al-FMOS and bl-FMOS at the ith grid point Where:
[0062] Equation (27) represents an improved estimate of
the relationship betWeen observations and forecasts than that
obtained using MOS coef?cients because FMOS coef?cients
are more accurate than MOS coe?icients. Using the methods
of Bishop and Shanley (2008), supra, Equation (27) can be
used to improve the accuracy and reliability of probabilistic
forecasts of Weather.
[0063] It Will be noted here that in some embodiments of
the present invention, all of aMOS, bMOS, a‘’ '1'”, bprior, Ga and
G1,, and Ra and Rb may already have been computed and
stored in an internal or external memory or other storage
medium, so that they need simply to be loaded as inputs into
Dec. 2, 2010
a computer programmed With appropriate softWare Which can
then estimate the aFMOS and bFMOS coef?cients using the
input values. In other embodiments, the values of aMOS and
bMOS and aprior and bprior may have been previously computed
and loaded as inputs, With Ga and G1,, Ra and Rb being com
puted as part of the process of estimating the aFMOS and
bFMOS coef?cients. In still other embodiments, none of aMOS,
bMOS, Ga and G1,, Ra and Rb may have been already computed
so that all ofaMOS, bMOS, Ga and G1,, Ra and Rh are computed
during the process of estimating the aFMOS and bFMOS coef
?cients.
[0064] To test the FMOS approach, random correlated
?elds of true values yT:[yly2, . . . , yn] Were created. An
example of such a random ?eld of true values is shoWn in FIG.
1, Which illustrates a simulated true state of a 2-dimensional
Wind component in m/ s across idealiZed Weather ?elds hav
ing a scale of 1000 km. From these true values, corresponding
forecast values fT:[fl, f2, . . . , fn] Were created at each grid
point using the true values of spatially varying ?elds of a and
b in Equation (1). To give these ?elds meteorological rel
evance they Were made to propagate from West to East and
evolve in time in a Way that is qualitatively similar to the Way
meteorological ?elds of, for example, temperature, evolve
With time. In this Way, time series of pairs of observations and
forecasts that obey Equation (1) could be easily created for
testing the MOS and FMOS approaches for estimating the
coef?cients a and b from the data ?elds. The ?elds together
With the MOS ?elds Were also used to test the aforementioned
methods for estimating Ga and G1,, R1, and Rh.
[0065] The bene?cial effects of using the method of the
present invention are clearly illustrated by the plots shoWn in
FIGS. 2A-2H and FIGS. 3A-3H.
[0066] As noted above, FIGS. 2A-2H illustrate aspects of
the differences betWeen the MOS and FMOS estimates of the
slope coe?icient a obtained from 16 trials/events. FIGS.
3A-3H illustrate corresponding aspects betWeen the MOS
and FMOS estimates of the slope coe?icient b.
[0067] Each of the 16 trials produces an observation y and
a forecast fat each gridpoint of the model domain. From these
16 data pairs, attempts Were made to estimate the unknown
set forth above. The results of these estimates are shoWn in
FIGS. 2A-2B and 3A-3B.
[0068] FIGS. 2C and 3C are plots of an exemplary true
a-?eld and an exemplary true b-?eld obtained using a minor
adjustment of the method described in the appendix of Bishop
and Hodyss (2007). See C. H. Bishop and D. Hodyss, “Flow
adaptive moderation of spurious ensemble correlations and
its use in ensemble based data assimilation.” Quart. J. Roy.
Mel. Soc. 133, pp. 2029-2044 (2007). FIGS. 2D and 3D
shoWs the true spatial covariance function of the a- and
b-?elds shoWn in FIGS. 2C and 3C, respectively, and are
equivalent to an error covariance function of the prior esti
mates of a and b that sets a and b equal to unity at all grid
points.
[0069] FIGS. 2E and 2E shoW plots of Ga, the estimated
error covariance function of the prior estimates of a (FIG. 2E),
and Ra, the estimated error covariance function of the MOS
estimates of a (FIG. 2F), With FIGS. 3E and 3E shoWing the
corresponding plots for Gb and Rb.
[0070] FIGS. 2A/3A and 2B/3B illustrate aspects of the
MOS estimates of the coef?cients a and b obtained using the
methods of the prior art. Note the noisy nature of the a and b
?elds shoWn in FIGS. 2A and 3A as compared With the true
US 2010/0305860 A1
?elds shown in FIGS. 2C and 3C. In contrast, the FMOS
estimates of the coef?cients a and b obtained using the meth
ods described herein are smooth, and resemble the plots of the
true a and b ?elds shoWn in FIGS. 2C and 3C.
[0071] In addition, a comparison of the error of the FMOS
estimates of a and b shoWn in FIGS. 2H/ 3H With the error of
the MOS estimates shoWn in FIGS. 2B/3B shoWs that the
FMOS errors are about an order of magnitude smaller than the
MOS estimate. A comparison of these Figures shoWs that the
maximum errors incurred by the MOS estimate are more than
3-10 times larger than those incurred by the FMOS estimate.
For example, the error of the al-MOS ?eld illustrated in FIG. 2B
shoWs error extrema With magnitudes greater than 0.2, and in
fact shoWs mean square errors (mse) of the MOS estimates
that are 19 times larger than the mse of the FMOS estimate
shoWn in FIG. 2H. The MOS estimate of the regression coef
?cients based on 16 events is of little practical use because the
noise (error) is larger than the signal. On the other hand, the
error obtained from the FMOS estimates shoWn in FIG. 2F
have maximum magnitudes of 0.04, Which is much smaller
than the 0.2 amplitude variations of ai. Thus, as can readily be
appreciated, the FMOS estimates of the regression coef?
cients in accordance With the present invention represent a
signi?cant improvement over the MOS estimates for use With
small data sets.
[0072] In addition, to test the statistical signi?cance of the
superiority of FMOS over MOS, 16 independent trials Were
performed in Which [61-2) :10< (al-yi)2 ) :10< (bl-)2). In other
Words, the variance of the random component of forecast
error el- in Equation (27) Was made to be 10 times larger than
the variance of the signal. It Was found that MOS mse
exceeded FMOS mse for the slope parameter “a” by a factor
betWeen 3.5 and 57 .5. The mean of this superiority factor over
the 16 trials Was 15.3. In the same set of independent trials,
MOS mse exceeded FMOS mse for the intercept parameter
“b” by a factor betWeen 2.4 and 9.6. The mean of this supe
riority factor Was 6.1. Note that FMOS Was superior to MOS
in every one of the 16 trials for both parameters. The prob
ability of this happening by pure chance if FMOS Was no
better than MOS is 0.5l6:1.5><10_5. Hence, the null hypoth
esis that FMOS is not superior to MOS can be rejected With a
con?dence level of 99.999%.
[0073] An additional 16-trial test Was performed in Which
the variance of the random component of forecast error in
Equation (27) Was reduced by an order of magnitude so that
(61-2 ) :< (al-yi)2 > Ii (bl-)2). In this case, MOS mse exceeded
FMOS mse for the slope parameter “a” by a factor betWeen
3.8 and 16.7. The mean of this superiority factor Was 7.8. In
the same set of independent trials, MOS mse exceeded FMOS
mse for the intercept parameter “b” by a factor betWeen 1.5
and 5.8. The mean of this superiority factor Was 3 .7. Note that
FMOS Was superior to MOS in every one of the 16 trials.
Since the probability of this happening by pure chance if
FMOS Was no better than MOS is 0.5 1 6:1.5><10_5. Hence, the
null hypothesis that FMOS is not superior to MOS can again
be rejected With a con?dence level of 99.999%.
[0074] Comparison of the ?rst 16 trials With the second set
of 16 trials shoWs that the extent of superiority of FMOS over
MOS increases as the magnitude of the systematic part of
forecast error is reduced relative to the random part of forecast
error.
Dec. 2, 2010
[0075] Thus, the use of the FMOS method of the present
invention has numerous advantages. For example, its esti
mates of regression coef?cients are more accurate, particu
larly When the training data set is small. Moreover, if properly
con?gured, the results using the FMOS method of the present
invention Will be equivalent to those using MOS in the limit of
an in?nite data training set. This is a desirable feature
because, in the limit of a very large data set, MOS is guaran
teed to recover the true coe?icients.
[0076] It Will be appreciated by one skilled in the art that
one or more aspects of a method for providing stabiliZed and
spatially smooth regression coef?cients for Weather forecast
error correction from small training data sets as described
herein can be accomplished by one or more processors
executing one or more sequences of one or more computer
readable instructions read into a memory of one or more
computers from volatile or non-volatile computer-readable
media capable of storing and/or transferring computer pro
grams or computer-readable instructions for execution by one
or more computers. Volatile media can include a memory
such as a dynamic memory in a computer. Non-volatile com
puter readable media that can be used can include a compact
disk, hard disk, ?oppy disk, tape, magneto-optical disk,
PROM (EPROM, EEPROM, ?ash EPROM), SRAM,
SDRAM, or any other magnetic medium; punch card; paper
tape; or any other physical medium such as a chemical or
biological medium.
[0077] Although particular embodiments, aspects, and fea
tures have been described and illustrated, it should be noted
that the invention described herein is not limited to only those
embodiments, aspects, and features. It should be readily
appreciated that modi?cations may be made by persons
skilled in the art, and the present application contemplates
any and all modi?cations Within the spirit and scope of the
underlying invention described and claimed herein. Such
embodiments are also contemplated to be Within the scope
and spirit of the present disclosure.
What is claimed is:
1. A computer-implemented method for estimating a set of
regression coef?cients for use in estimating a model forecast
variable, comprising:
receiving, at a computer programmed With appropriate
softWare, data of an n-vector aMOS and an n-vector bMOS,
the elements of aMOS and bMOS comprising a set of mod
eled output statistics (MOS) estimates of the true values
of forecast correction coef?cients a and b;
receiving, at the computer, data of an n-vector ap’ior and an
n-vector bp'ior, the elements of ap’ior and bprior compris
ing a set of prior estimates of forecast correction coef?
cients a and b;
receiving, at the computer, data of a spatial covariance
matrix Ga for ap’ior and a spatial covariance matrix Gb
for bp'ior;
receiving, at the computer, data of a spatial error covari
ance matrix Ra for aMOS and a spatial error covariance
matrix Rb for bMOS; and
transforming the data of aMOS, bMOS, aprior, bp’ior, Ga, Gb,
Ra, and Rb into a set of estimated regression coef?cients
FMOS FMOS
a and b , wherein
aFMOS:aprior: Ga ( Ga :R6941 (aMOS_ aprior); and
US 2010/0305860 A1
2. The method according to claim 1, wherein the spatial
covariance matrix Ga has the form Ga: (aP'i°’—a)(aP”°’—
a)T> and the spatial covariance matrix Gb has the form G1,:
<(bP”°’—b)(bP”°’—b)T> Where a and b comprise n-vectors of
the true values of the regression coef?cients a and b.
3. The method according to claim 1, Wherein the matrix
elements {Rah-j and {Rbhj of the spatial error covariance
matrices Ra and Rh have the form {Rah-fl (aiMOS—ai)(ajMOS—
a)) and {Rb}lj:( (biMOS—bi)(bjMOS—bj) ), Where a, a]. and bl.
and bj are the 1th and jth true values of the regression coef?
cients a and b.
4. A computer-implemented method for estimating a set of
regression coe?icients for use in estimating a model forecast
variable, comprising:
receiving, at a computer programmed With appropriate
softWare, data of an n-vector aM and an n-vector bMOS,
the elements of aMOS and bMOS comprising a set of mod
eled output statistics (MOS) estimates of the true values
of forecast correction coef?cients a and b; _
receiving, at the computer, data of an n-vector am” and an
n-vector bp’ior, the elements of ap’ior and bp’ior compris
ing a set of prior estimates of forecast correction coef?
cients a and b; _
estimating a spatial covariance matrix Ga for apnor spatial
covariance matrix Gb for bp’ior;
estimating a spatial error covariance matrix Ra for aMOS
and a spatial error covariance matrix Rb for bMOS; and
transforming the data of aMOS, bMOS, aim”, bP’w’, Ga, G1,,
R , and Rb, into a set of estimated regression coef?cients
a OS and bFMOS, Wherein
QFMOSIQWWIGAGQIRJA (aM0s_aprior); and
5. The method according to claim 4, Wherein the spatial
covariance matrix Ga has the form Ga: (aP’i°’—a)(aP”°’—
a)T> and the spatial covariance matrix Gb has the form Gb:
<(bP”°’—b)(bP”°’—b)T> Where a and b comprise n-vectors of
the true values of the regression coef?cients a and b.
6. The method according to claim 4, Wherein the matrix
elements {Rah-j and {RbL-j of the spatial error covariance
matrices Ra and Rh have the form {Rah-fl (aiMOS—ai)(ajMOS—
a)) and {Rb}lj:( (biMOS—bi)(bjMOS—bj)), Where al, a]. and bi,
and the b]. are the ith and jth true values of the regression
coef?cients a and b.
7. A computer-implemented method for estimating a set of
regression coe?icients for use in estimating a model forecast
variable, comprising:
receiving, at a computer programmed With appropriate
softWare, data of values of a plurality of analysis/obser
vation variables yik, each value of yik representing a
value of an ith variable yl. during a kth event;
receiving, at the computer, data of values of a plurality of
corresponding forecast variables fl-k, each value of fl-k
being related to a corresponding value of yl-k by the
relation fik:aiyik+bi+eik, Where al. and bl. are regression
coef?cients and el-k is a random number associated With
the kth forecast;
receiving, at the computer, data of values of a plurality of
mean variables y, and fi, each of y, and comprising
average values of y, and If over a plurality of realiZa
tions;
Dec. 2, 2010
transforming the data of the yik, fik, i, and f. in to a ?rst set
of estimated regression coef?cients al-MO and bl-MOS, the
?rst set of estimated regression coe?icients comprising a
set of modeled output statistics (MOS) estimates of the
true values of regression coef?cients al. and bi;
receiving, at the computer, data of a ?rst set of second set of
estimated regression coef?cients al-P'w’ and bl-pnor, the
second set of estimated regression coe?icients repre
senting a prior estimate of regression coef?cients al. and
transforming the values of al-MOS into the n-vector aMOS and
the values of bl-MOS into the n-vector bMOS;
transforming the values of af '1'” into the n-vector aprior a
prior and the values of bl-P'w’ into an n-vector bP'_1°’;
estimating a spatial covariance matrix Ga for apnor and
spatial covariance matrix Gb for bp'wr;
estimating a spatial error covariance matrix Ra for aMOS
and a spatial error covariance matrix Rb for bMOS; and
transforming the data of aMOS, bMOS, ap'ior, Ga, G1,, Ra, and
Rb into a set of estimated regression coe?icients aFMO’
FMOS
Sand 12 , wherein
aFMOS:aprior: GAG‘, :Rayi (aMOS_ aprior); and
8. The method according to claim 7, Wherein the spatial
covariance matrix Ga has the form Ga (aP’i°’—a)(aP”°’—
a)T> and the spatial covariance matrix Gb has the form G1,:
<(bP”°’—b)(bP'1°’—b)T>, Where a and comprise n-vectors of
the true values of the regression coef?cients al. and bi.
9. The method according to claim 7, Wherein the matrix
elements {Rah-j and {RbL-j of the spatial error covariance
matrices Ra and Rb have the form {Ra}lj:( (al-MOS—al-)(a]-MOS—
a)) and {Rb}lj:( (bZ-MOS—bZ-)(bJ-MOS—bJ-)l , Where a,, a], and bi,
and b. are the ith and jth true values of the regression coef?
cients a and b.
10. A computer-implemented method of forecasting
Weather from a small data set, comprising:
receiving, at a computer programmed With appropriate
softWare, data of values of a set of observation variables
yl-k, each of yl-k representing a value of an observation
variable yl- at a kth event;
receiving, at the computer, data of n-vectors of ?ltered
model output coef?cients aFMOS and bFMOS, Wherein
aFMOS:aprior: GAG‘, :Rayi (aMOS_ aprior); and
and Wherein aMOS andb bMOS comprise n-vectors of a set
of MOS estimated regression coe?icients, am” and
bP'w’ are n-vectors of prior estimated coe?icients, Ga
is the spatial covariance matrix for ap'wr, Gb is the
spatial covariance matrix for bprior, Ra is the spatial
error covariance matrix for aMOS, and Rb is the spatial
error covariance matrix for bMOS; and
transforming each observation variable yl-k into a corre
sponding forecast variable fik, Where
fi :aiF osyik:biFMos ikeiks al-FMOS and b-FMOS are the:
ith elements of the n-vectors aFMOS and b MOS, respec
tively, and 61-1“ is a random number associated With the kth
forecast;
Wherein the transformation of yl-k into fl-k represents an
accurate description of relationship betWeen yl-k and fl-k.
11. The method according to claim 10, Wherein the spatial
covariance matrix Ga has the form Ga (aP’i°’—a)(aP”°’—
US 2010/0305860 A1 Dec. 2, 2010

a)T> and the spatial covariance matrix Gb has the form Gb mat'noes R“andR1’ have the fOrm{ R‘1}11..I ((al.MOS- alXaJ
. .MOS

a)) and {Rbhfl (bZ-MOS—bZ-)(bJ-MOS—bJ-)), Where ai, aj and bi,

<(bP”°’-b)(bP”°’-b)T>, Where a and b comprise n-vectors of and ‘DJ are the ith and jth true Values of the regression coef?
the true Values of the regression coef?cients al- and bi. cients a and b.
12. The method according to claim 10, Wherein the matrix
elements {Rah-j and {RbL-j of the spatial error covariance * * * * *