Professional Documents
Culture Documents
Dr Roland Young
Fall 2022
Theme 7
Introduction
Data processing is the general term for the sequence of steps used to convert
raw instrument measurements into useable data
What is required is very instrument- and spacecraft-specific, but there are a few
general processes that almost always occur:
• Data cleaning
• Navigation
• Calibration
Theme 7
Contents
1. Data levels
2. Navigation
3. Calibration
4. Data archives
Example instrument
L0
L1a
“Engineering”
L1b
L2
L3
“Science”
Data levels
Spacecraft missions typically use a set of data processing levels that identify
different stages of data processing.
Every mission will have its own details of exactly what each level means
But, in general:
Level 0 Spacecraft raw data packets delivered from the spacecraft at the Mission
Operations Center, including science and housekeeping data.
Level 2 Calibrated FITS radiance spectra in physical units (W cm-2 sr-1 (cm-1)-1) with
geometry information. Using reconstructed ephemeris.
Also includes associated quantities, e.g., brightness temperature.
Navigation
This is the step when each observation (e.g. a raw image) is combined with
information about where, when, and how it was taken. This is required to
pinpoint e.g. the location of each pixel in an image in terms of latitude and
longitude.
A major source of
information for navigation is
the NASA SPICE facility. The
main data products are
called SPICE kernels. These
contain navigation and other
ancillary information for
precision observational
geometry. Usually produced
by a spacecraft’s operations
team.
[https://naif.jpl.nasa.gov/naif/toolkit.html]
Theme 7.2
[https://naif.jpl.nasa.gov/naif/spiceconcept.html]
Theme 7.2
[https://naif.jpl.nasa.gov/naif/spiceconcept.html]
Types of SPICE kernels
• S kernel
Spacecraft ephemeris (position as a function of time, and orbital parameters)
• P kernel (SPK)
Ephemeris data for planets, moons, comets, asteroids, etc.
• I kernel (IK)
Instrument information: field-of-view, current status, orientation, status of
on/off switches, etc.
• C kernel (CK)
Orientation of various parts of the spacecraft: What direction the spacecraft
and instruments are pointing in
• E kernel (EK)
Spacecraft events, planned and unplanned
• Spacecraft clock kernel (SCLK)
Timestamp for other data so absolute and relative times can be compared
Theme 7.3
[Based on Hanel et al., pp.281-293]
Calibration
Precision
• How reproducible a measurement is
• Quantified by the random error
in the measurement
Accuracy
• How close the measurements are
to the true value
• Depends upon systematic errors
in the measurement [Copyright 2022, Aaron Keller, adapted by permission.]
Theme 7.3
The level of water linearly depends on the atmospheric pressure. E.g. when the
atmospheric pressure is 1010 mb, the water is halfway between the two marks
Calibration
Spacecraft instruments are calibrated while being designed and built, and again
regularly once they are in space, because
• The thermal and electrical environment change in space
• The launch may damage or otherwise affect the instrument
• Instrument response changes over time
• Continuous operation over a long period (years) affects the instrument
1. A radiometer/spectrometer is designed
to measure radiances over a range of values
2. Measurements are physically made by
photons hitting a detector. The actual
measurement will be some induced
voltage or photon count
3. We need to know which voltage / photon
count corresponds to which radiance [Edwards et al. 2021, Fig. 20]
Theme 7.3
Δ𝑉
𝑟 𝜐 =
Δ𝑊
EMM-EMIRS Instrument
Response Function 𝑟 𝜐
at three different
detector temperatures
(pre-flight)
The measurement on the detector (volts, or digital numbers – photon counts) while
facing the target (planet) is
𝜐2
𝐴1 = න 𝑟 𝜐 𝐼 𝜐 − 𝐵 𝜐, 𝑇eff 𝑑𝜐
𝜐1
It is essential for 𝑟 𝜐 and 𝐵 𝜐, 𝑇eff to be stable over the time period between
calibration(s) and measurement
Assuming:
1. The cold calibration target is deep space, i.e. 𝐵 𝜐, 𝑇3 = 0
2. Constant 𝑟 𝜐 between frequencies 𝜐1 and 𝜐2 , and 0 elsewhere
[Jones et al., 2021, Fig. 22]
𝐴1 −𝐴3
Then 𝐼 𝜐ҧ = 𝐵 𝜐,ҧ 𝑇2
𝐴2 −𝐴3
Where 𝜐ҧ = (𝜐1 + 𝜐2 )/2.
Recall: 𝐴1 = target, 𝐴2 = hot black body, 𝐴3 = cold black body
For a spectrometer, 𝐴𝑖 = 𝐴𝑖 (𝜐) so all frequencies are calibrated independently.
Theme 7.4
Data archives
The final destination for most spacecraft data is a data archive, either publicly or
commercially accessible.
Data standards for such archives are usually very high, with very strict formatting,
documentation, and storage hardware standards so data can continue to be used
for decades. For this reason, a lot of archived data is in ASCII (text) or custom binary
format, rather than commercially available formats (e.g. Excel spreadsheet).
[https://sdc.emiratesmarsmission.ae/]
Theme 7.4
[https://sdc.emiratesmarsmission.ae/]
Theme 7.5
Demonstration
L2 → L3 data processing
Sources
Hanel, Conrath, Jennings, and Samuelson (2003), Exploration of the Solar System by
Infrared Remote Sensing, CUP, 2nd Ed.
Edwards et al. (2021), Space Science Reviews, 217, 77, 10.1007/s11214-021-00848-1.
Dr Roland Young
Fall 2022
Theme 6
Contents
4. Remote sounding
5. Other planets
Theme 6.1
Weather Climate
Will it rain next week? How much will it rain next March?
Difficult-to-impossible to predict more than a Possible to predict decades into the future
week or so ahead
An initial value problem A boundary value problem
Highly sensitive to initial conditions (“chaotic”) Highly sensitive to external “forcings”
Dynamics more important Radiative balance more important
Influence of and on individual events very Influence of oceans, humans very important
important (e.g. storms, floods)
Theme 6.1
Temperature (oC)
Rainfall (mm)
https://www.timeanddate.com/weather/united-arab-emirates/al-ain/climate
Theme 6.1
Daily high
Daily high
– 30-day running mean
Daily low
Daily low
– 30-day running mean
Daily range
Advantages:
• Cheap
• Easily maintained / replaced / calibrated
• Basic instruments are easy to build
• Good vertical resolution
Disadvantages:
• Coverage limited to land + major air / shipping routes
• Enormous network required to collect data Radiosonde
• Urban heat island effect Argo float
Observation platforms:
• Ground station
• Balloons
• Ships
• Aircraft
• Buoys
• Floaters
[Metcheck, radiosondemuseum.org, argo.ucsd.edu]
Theme 6.2
My neighbour’s
house
(~100m away) Temperature offset of ~4 oC
and numerous outliers,
probably due to sunlight
falling on one of the
thermometers
My house
Theme 6.2
Satellite observations
Advantages:
• Systematic, regular, 3D, global coverage over short periods
• Can be operated from a small number of locations
• Same instrument is used everywhere
The “A-Train”
Disadvantages:
• Expensive
• Hard to maintain / calibrate
• Remote measurements
• Instruments are complicated
• Some lower atmosphere
measurements are difficult
Types of measurement:
• Visible imaging
• Infrared sounding
• Microwave sounding [NASA/JPL]
Ozone
Stratospheric Sounding Unit
Infrared sounders
Microwave sounders
All-sky microwave radiances
GPS radio occultation
Geostationary radiances
Atmospheric motion vectors
Scatterometers
WAVE intragrated parameters
Ground-based radar
Conventional (i.e. ground-based)
Hersbach+ 2019
Theme 6.2
Earth Radiation
Read+ 2016
Budget Satellite
(1984-2005)
[ESA]
Theme 6.2
Surface properties
Surface remote sensing is a multi-billion-$ industry (see GIS theme, SA1 course)
Advantages:
• Can monitor continuously
• Instrument design is (relatively) simple
Disadvantages:
• Relatively low vertical resolution (10 km)
• Generally only temperature can be
resolved vertically
• Composition measurements must
assume vertical distribution
Example:
• Emirates Mars InfraRed Spectrometer
on board the Emirates Mars Mission [Lee et al. 2009]
Theme 6.2
Instrument looks sideways at the atmosphere along the tangent to the planet
Advantages:
• High vertical resolution (<5 km)
• Long path through atmosphere
• No surface emission to complicate signal
• Aerosol profiles are possible
• Low concentrations can be measured
Disadvantages:
• Difficult in the troposphere
• Complex instruments / precise pointing
Example:
• Mars Climate Sounder on board
Mars Reconnaissance Orbiter [Lee et al. 2009]
Theme 6.2
Instrument observes a distance light source (e.g. Sun) through the atmosphere
Advantages:
• High vertical resolution (<5 km)
• Long path through atmosphere
• S/N ratio is high due to strong source with known spectrum
• Aerosol profiles are possible
• ppb concentrations can be measured
Disadvantages:
• Only possible at sunrise/sunset
• Same instrumentation challenges as limb
Example:
• Atmospheric Chemistry Suite on board
ExoMars Trace Gas Orbiter [Lee et al. 2009]
Theme 6.3
(2-4) are quantized – they can only occur for photons with discrete frequencies.
Each molecule and transition type has a characteristic set of energy levels.
Vibrational
modes of
two
important
atmospheric
molecules Carbon dioxide Ozone [kyoroskichannnel]
Theme 6.3
Once one of transitions (2-4) takes place, one of two things then happens:
The most important transitions for atmospheric monitoring are in the infrared and
microwave regions of the electromagnetic spectrum:
Consider light with radiance I moving a distance ds through an absorbing gas of density 𝜌.
The Beer-Lambert Law (or Bouguet’s Law) states that absorption is proportional to
1. The mass of the absorbing gas = 𝜌 𝑑𝑧 per unit area
2. The radiance I
k
The constant of proportionality is
called the absorption coefficient k.
𝑑𝐼 = −𝐼 𝑘 𝜌 𝑑𝑠
𝐼 = 𝐼0 exp − න 𝑘 𝜌 𝑑𝑧
Some definitions:
In general, 𝜒 = 𝑧𝑑 𝜌 𝑘 is called the optical path. 𝜒 > 0 always so I < I0 always.
When measured vertically downwards from the top of the atmosphere the optical path is
called the optical depth 𝜏:
𝑑𝜏 = −𝑘 𝜌 𝑑𝑧
(Note sign: optical depth = 0 at the top-of-atmosphere, integral is downwards, so 𝜏 > 0 still)
𝐼 = 𝐼0 𝒯 𝒯 = exp − 𝑧𝑑 𝜌 𝑘 𝒯 = 𝑒 −𝜒
Theme 6.3
𝑑𝐼 = 𝐵(𝑇) 𝑘 𝜌 𝑑𝑧
The atmosphere is not in true thermodynamic equilibrium, but in the lower atmosphere
conditions called Local Thermodynamic Equilibrium (LTE) apply, which are a sufficiently
good approximation for black-body emission to be used.
𝑘 = 𝑘𝜈 is a strong function of frequency 𝜈. For now, we will use the “grey” approximation: k
is constant, i.e. does not vary with position or frequency (think of it as an average over all
frequencies). This turns out to be surprisingly reasonable for now.
Theme 6.3
𝑑𝐼 = −𝐼 𝑘 𝜌 𝑑𝑧 + 𝐵 𝑘 𝜌 𝑑𝑧
𝑭↑ Upwards-propagating irradiance
𝑭↓ Downwards-propagating irradiance
Assuming:
• Infrared radiation is the only energy transfer mechanism
(OK for stratosphere, unrealistic for troposphere)
• “Grey” atmosphere – absorption coefficient is constant
𝜙 = 𝐹↑ − 𝐹↓
Theme 6.4
Remote sounding
Satellites measure radiative flux emerging from the top of the atmosphere. They
do not measure atmospheric (or surface) properties directly.
Remote sounding
• Rayleigh scattering
Ultraviolet/visible
<700 nm
1
Proportional to 4
𝜆
• Water vapour
5 – 8 𝜇m (strong)
20+ 𝜇m (strong)
• Carbon dioxide
4.3 𝜇m (narrow band, strong)
15 𝜇m (wide band, strong)
• Ozone and O2
9.5 𝜇m (narrow band, weak)
• Methane, N2O
We focus on thermal rather than solar radiation. The
8 𝜇m (narrow band, weak) principles are the same but infrared absorption is more
4.6 𝜇m (narrow band, strong) useful for deducing atmospheric structure and composition.
Theme 6.4
Weighting functions
A typical set of weighting functions By designing an instrument so that it
measures radiance over a range of
wavelengths where the absorption
coefficient changes from transparent to
opaque, the measured radiance is weighted
by different layers in the atmosphere.
Atmospheric retrievals
Sources
ESA/Roscosmos
ExoMars Trace Gas Orbiter
Atmospheric Chemistry Suite
Dust Surface
Thermal Infrared channel
(1090 cm-1) temperature
Nadir viewing, sensitive to
600-2500 cm-1 (1.7 – 17 𝜇m)
Temperature at
~40 km altitude
Water ice (820 cm-1)
Atmospheric temperature
above surface temperature [Korablev+ 2018]
The BSc Physics Space Science Track
Lecture-1
https://web.njit.edu/~gary/728/Lect
ure2.html
Topics in Radio Science:
Lecture-2
1. Radio Astronomy
2. Plasma Physics
3. Solar Physics
4. Atmospheric Studies
5. Earth Observation
Lecture-1
• Functions: coding/decoding,
compression, Telecom protocols.
SatCom Applications:
Types of Services:
Areas:
• Mobile Telephony.
• Radio/Television broadcast.
• Broadband internet service.
• Digital video/audio/data relay
• Access for mobility (transportation,
aviation, space exploration.
• Secure Government, Military, defence,
warfare.