CERTIFICATE

This  is  to certify that ​Raghvendra Mishra ​has completed the training in 

our  organization  India  Meteorological  Department,  Lodhi  Road,  New 
Delhi-110003  under  my  supervision  during  the  period  10  ​th  June,  2019 
​   July,  2019  and  submitted  the  project  on  ​“  ​To visualize the data
to  22  th
for soil moisture & correct the temperature bias​”​.  

During  her  training  period,  he  was found to be sincere and meticulous 

in her work. I wish her every success. 

​(Dr. D.R Patnayak)

 
 ACKNOWLEDGEMENT

We take this opportunity to express our gratitude and thanks to ​Dr. D.R Patnayak​,
Deputy Director General of Meteorology,Numerical Weather Prediction Division,
India Meteorological Department, New Delhi for assigning the topic and valuable
guidance.

We are especially thankful to ​Dr. D.R Patnayak​(Scientist - E) for his valuable

suggestions and help throughout the period of training.

I want to thank ​Ms. Dolly Sharma​, Head of Department(Computer Science),

KCC Institute of Technology and Management,Greater Noida, for allowing me to
join summer training program at IMD, New Delhi.

Special thanks to ​Dr. Sahil Kansal, Mr. Ravi Kant Nirala and Mr. Ghanshyam
Yadav ​who always proved as an excellent project guide & without his guidance
this training would not have been successful.

We are also thankful to the management and the staff of Numerical Weather
Prediction Division of India Meteorological Department, New Delhi for extending
all possible support and help without which it was not possible to complete the
assigned work.

​(Raghvendra Mishra) 
 ​TABLE OF CONTENT

Certificate……………………………………………………… i

Acknowledgement……………………………………………… ii

Table of contents……………………………………………….. iii

Introduction 1
● History…………………………………………………. 1.1
● Background…………………………………………… 1.2
● IMD Services………………………………………….. 1.3
● Numerical weather prediction​ Division 1.4
● Objective………………………………………………. 1.5

Data visualization 2
● Soil Moisture ……………………………….…. 2.1
● Programming Details………………………………... 2.2

Bias Correction of Temperature 3

● Bias Correction ……………………………….…. 3.1
● Programming Details………………………………... 3.2

● References………………………………………………
4
1.​ I​ ntroduction

1.1 ​History
The beginnings of meteorology in India can be traced to ancient times. Early philosophical
writings of the 3000 B.C. era, such as the Upanishadas, contain serious discussion about the
processes of cloud formation and rain and the seasonal cycles caused by the movement of earth
round the sun. Varahamihira’s classical work, the Brihatsamhita, written around 500 A.D.,
provides a clear evidence that a deep knowledge of atmospheric processes existed even in those
times. It was understood that rains come from the sun (​Adiyat Jayate Vrishti​) and that good
rainfall in the rainy season was the key to bountiful agriculture and food for the people. Kautilya’
sarthashastra contains records of scientific measurements of rainfall and its application to the
country’s revenue and relief work. Kalidasa in his epic, “Meghdoot”, written around the seventh
century, even mentions the date of onset of the monsoon over central India and traces the path of
the monsoon clouds.

1.2 ​Background

A disastrous tropical cyclone struck Calcutta in 1864 and this was followed by failures of the
monsoon rains in 1866 and 1871. In the year 1875, the Government of India established India
Meteorological Department, bringing all meteorological work in the country under a central
authority. Mr. H. F. Blanford was appointed Meteorological Reporter to the Government of
India. The first Director General of Observatories was Sir John Eliot who was appointed in May
1889 at Calcutta headquarters. The Headquarters of IMD were later shifted to Shimla (1916),
then to Poona (now Pune, 1928) and finally to New Delhi.
1.3 ​IMD Services
The India Meteorological Department (IMD) was established in 1875. It is the National
Meteorological Service of the country and the principal government agency in all matters
relating to meteorology, Seismology and allied subjects. IMD provides services to different
sectors.

At Present IMD is under the Ministry of Earth Sciences (MoES). One of the mandates of IMD is
to take meteorological observations on real time basis and to provide current and past
observations for operational and research use. Surface Observatories are located one in each
district (559 observatories) so as to meet the requirements of agricultural, transport and other
sectors. IMD’s present Upper air observational network comprises 39 radiosonde and 62 pilot
balloon, 39 RS/RW observatories, 15 ​Doppler Weather Radar, 711 AWS, 1205 ARG, ​55
Seismological observatories etc. spread all over the country.

For the convenience of administrative and technical control, there are 6 Regional Meteorological
Centres, each under a Deputy Director General with headquarters at Mumbai, Chennai, New
Delhi, Kolkata, Nagpur and Guwahati. Under the administrative control of Deputy Director
General, there are different types of operational units such as Meteorological Centres at State
Capitals, Forecasting offices, Agrometeorological Advisory Centres, Flood Meteorological
Offices, Area Cyclone Warning Centres and Cyclone Warning Centres.

In addition, there are separate Divisions to deal with specialised subjects. They are:-

● Agricultural Meteorology
● Aviation Meteorology
● Climatology
● Regional Specialized Meteorological Centre
 ● Hydrometeorology (Hydromet)
● Instrumentation
● Meteorological Telecommunication
● Positional Astronomy
● Satellite Meteorology
● Seismology

1.4 ​NUMERICAL WEATHER PREDICTION DIVISION

Numerical weather prediction​ (​NWP​) ​uses ​mathematical models​ of the atmosphere and oceans to
predict the weather​ based on current weather conditions. Though first attempted in the 1920s, it
was not until the advent of ​computer simulation​ in the 1950s that numerical weather predictions
produced realistic results. A number of global and regional forecast models are run in different
countries worldwide, using current weather observations relayed from ​radiosondes​, ​weather
satellites​ and other observing systems as inputs.
Mathematical models based on the same physical principles can be used to generate either
short-term weather forecasts or longer-term climate predictions; the latter are widely applied for
understanding and projecting ​climate change​. The improvements made to regional models have
allowed for significant improvements in ​tropical cyclone track​ and ​air quality​ forecasts;
however, atmospheric models perform poorly at handling processes that occur in a relatively
constricted area, such as ​wildfires​.

1.3 ​Objective

It is very difficult to handle such a huge data of rainfall and so as to retrieve that data.

2. ​Data visualization

2.1 ​Soil Moisture :​Soil moisture is important to forecasting. It affects both the ​temperature
forecast​ and ​precipitation forecast​. Let's first start with the temperature forecast. High soil
moisture will produce high evaporation, especially if temperatures warm significantly during the
day. This evaporation will produce evaporative cooling. Although the temperature warms during
the day, the evaporation does prevent the temperature from getting as warm as it otherwise
would have. A good example is comparing Mississippi to Arizona in the summertime (both
states are on about the same latitude). Mississippi generally has a high soil moisture content and
evapotranspiration while Arizona generally has dry soils and low values of evapotranspiration.
High temperatures in southern Arizona average over 100 degrees in the summer while
temperatures generally do not rise above 100 in Mississippi (unless a drought reduces soil
moisture). The ​humidity​ does make it feel more uncomfortable outside though. It is hotter in
Arizona, but the humidity in Mississippi can make it feel just as hot in the summer. High soil
moisture values will tend to increase the ​dew point​. This has a major consequence on forecasted
lows. The overnight low under uniform weather conditions will not drop by more than a couple
degrees below the evening dew point, especially if the dewpoint is above 60 F. Condensation (a
warming process) occurs when the temperature tries to drop below the dewpoint at night.
Therefore, high dew points limit the amount of overnight cooling. If dew points are low, such as
when a continental high pressure is in place or a location is located in a dry climate (or dry
weather pattern), the overnight low will be much cooler than the afternoon high. Since the
dewpoint is low, the temperature can continue falling at night without condensation warming the
air and limiting the cooling. Rule of thumb: If the ​dewpoint depression​ is large during the
afternoon, there will be a large temperature range between the high and low temperature.

Soil moisture is also important to precipitation forecasts. High soil moisture increases the
likelihood of ​moisture convergence​. A ​trigger mechanism​ such as a front or low pressure will not
produce precipitation unless there is moisture in place to lift. Moist air rising has a much better
chance of producing precipitation than ​dry air​ that is rising. High soil moisture continuously
evaporates moisture into the air, which helps to supply low level moisture. The best combination
is to have moist soils along with moisture being advected from a moisture source such as the
Gulf of Mexico into a trigger mechanism. Droughts and Floods can produce a positive feedback
loop that can continue the drought or flood. When the soils dry out, there is less moisture for
fronts and other trigger mechanism to lift and therefore there is a continuation of less rainfall.
When floods occur, the supply of evaporating moisture to the atmosphere is continuous and there
is always moisture in place for a trigger mechanism to lift. It takes a dramatic shift in the weather
pattern sometimes to end a drought or flood because of this positive feedback loop. There are
several ways to infer the soil moisture across a forecast region. One way is the study the 24-hour
precipitation charts each day. From these you can determine which locations have wet or dry
soils. Some states have mesonets that measure soil moisture directly. Keeping track of the soil
moisture can make you both a better temperature and precipitation forecaster.
2.2 ​Programming Details
Programming Language:
​We have used python as the programming language for visualisation of the data.
Python:
​Python is an easy to learn, powerful programming language. It has efficient high-level data
structures and a simple but effective approach to object-oriented programming. Python's elegant
syntax and dynamic typing, together with its interpreted nature, make it an ideal language for
scripting and rapid application development in many areas on most platforms.
Story behind the name
Guido van Rossum, the creator of the Python language, named the language after the BBC show
"Monty Python's Flying Circus". He doesn't particularly like snakes that kill animals for food by
winding their long bodies around them and crushing them.
Features of Python
Simple
Python is a simple and minimalistic language. Reading a good Python program feels almost like
reading English, although very strict English! This pseudocode nature of Python is one of its
greatest strengths. It allows you to concentrate on the solution to the problem rather than the
language itself.
Easy to Learn
As you will see, Python is extremely easy to get started with. Python has an extraordinarily
simple syntax, as already mentioned.
Free and Open Source
Python is an example of a FLOSS (Free/Libré and Open Source Software). In simple terms, you
can freely distribute copies of this software, read its source code, make changes to it, and use
pieces of it in new free programs. FLOSS is based on the concept of a community which shares
knowledge. This is one of the reasons why Python is so good - it has been created and is
constantly improved by a community who just want to see a better Python.
High-level Language
When you write programs in Python, you never need to bother about the low-level details such
as managing the memory used by your program, etc.
Portable
Due to its open-source nature, Python has been ported to (i.e. changed to make it work on) many
platforms. All your Python programs can work on any of these platforms without requiring any
changes at all if you are careful enough to avoid any system-dependent features.
You can use Python on GNU/Linux, Windows, FreeBSD, Macintosh, Solaris, OS/2, Amiga,
AROS, AS/400, BeOS, OS/390, z/OS, Palm OS, QNX, VMS, Psion, Acorn RISC OS, VxWorks,
PlayStation, Sharp Zaurus, Windows CE and PocketPC!
You can even use a platform like Kivy to create games for your computer and for iPhone, iPad,
and Android.
Interpreted
A program written in a compiled language like C or C++ is converted from the source language
i.e. C or C++ into a language that is spoken by your computer (binary code i.e. 0s and 1s) using a
compiler with various flags and options. When you run the program, the linker/loader software
copies the program from hard disk to memory and starts running it.
Python, on the other hand, does not need compilation to binary. You just run the program
directly from the source code. Internally, Python converts the source code into an intermediate
form called bytecodes and then translates this into the native language of your computer and then
runs it. All this, actually, makes using Python much easier since you don't have to worry about
compiling the program, making sure that the proper libraries are linked and loaded, etc. This also
makes your Python programs much more portable, since you can just copy your Python program
onto another computer and it just works!
Object Oriented
Python supports procedure-oriented programming as well as object-oriented programming. In
procedure-oriented languages, the program is built around procedures or functions which are
nothing but reusable pieces of programs. In object-oriented languages, the program is built
around objects which combine data and functionality. Python has a very powerful but simplistic
way of doing OOP, especially when compared to big languages like C++ or Java.
Extensible
If you need a critical piece of code to run very fast or want to have some piece of algorithm not
to be open, you can code that part of your program in C or C++ and then use it from your Python
program.
Embeddable
You can embed Python within your C/C++ programs to give scripting capabilities for your
program's users.
Extensive Libraries
The Python Standard Library is huge indeed. It can help you do various things involving regular
expressions ,documentation generation, unit testing, threading, databases, web browsers, CGI,
FTP, email, XML, XML-RPC, HTML, WAV files, cryptography, GUI (graphical user interface),
and other system-dependent stuff. Remember, all this is always available wherever Python is
installed. This is called the Batteries Included philosophy of Python.
Summary
Python is indeed an exciting and powerful language. It has the right combination of performance
and features that make writing programs in Python both fun and easy.

Libraries Used:
1. Pandas: ​Pandas​ is the most popular python library that is used for data analysis. It
provides highly optimized performance with back-end source code is purely written in ​C
or ​Python​.
We can analyze data in pandas with:
1. 2Series
2. Data Frames

2. Matplotlib:
Matplotlib is a Python 2D plotting library which produces publication quality figures in a variety of
hardcopy formats and interactive environments across platforms. Matplotlib can be used in Python scripts,
the Python and ​IPython​ shells, the ​Jupyter​ notebook, web application servers, and four graphical user
interface toolkits.
Matplotlib tries to make easy things easy and hard things possible. You can generate plots, histograms,
power spectra, bar charts, error charts, scatterplots, etc., with just a few lines of code. For examples, see
the ​sample plots​ and ​thumbnail gallery​.

For simple plotting the pyplot module provides a MATLAB-like interface, particularly when combined
with IPython. For the power user, you have full control of line styles, font properties, axes properties, etc,
via an object oriented interface or via a set of functions familiar to MATLAB users.

3. Numpy:
NumPy is the fundamental package for scientific computing with Python. It contains among other things:

● a powerful N-dimensional array object

● sophisticated (broadcasting) functions
● tools for integrating C/C++ and Fortran code
● useful linear algebra, Fourier transform, and random number capabilities

Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of
generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily
integrate with a wide variety of databases.

4. Geopandas:​ ​ ​GeoPandas is an open source project to make working with geospatial

data in python easier. GeoPandas extends the datatypes used by ​pandas​ to allow spatial
operations on geometric types. Geometric operations are performed by ​shapely​.
Geopandas further depends on ​fiona​ for file access and ​descartes​ and ​matplotlib​ for
plotting.
 The goal of GeoPandas is to make working with geospatial data in python easier. It
combines the capabilities of pandas and shapely, providing geospatial operations in
pandas and a high-level interface to multiple geometries to shapely. GeoPandas enables
you to easily do operations in python that would otherwise require a spatial database such
as PostGIS.
5. Tkinter:
The ​Tkinter module (“Tk interface”) is the standard Python interface to the Tk GUI toolkit. Both
Tk and ​Tkinter are available on most Unix platforms, as well as on Windows systems. (Tk itself
is not part of Python; it is maintained at ActiveState.)
Running ​python -m Tkinter from the command line should open a window demonstrating a
simple Tk interface, letting you know that ​Tkinter is properly installed on your system, and also
showing what version of Tcl/Tk is installed, so you can read the Tcl/Tk documentation specific to
that version.

6. Mplcursors:
mplcursors​ provides interactive data selection cursors for Matplotlib. It is inspired from
mpldatacursor, with a much simplified API.
mplcursors​ requires Python 3, and Matplotlib≥2.1.
Output:
 ​3.Bias Correction of Temperature
The Bias Correction (BC) approach corrects the projected raw daily GCM output using the differences in
the mean and variability between GCM and observations in a reference period (Figure 1).

Figure 1.
Schematic of the bias correction methodology. BC uses raw model output for the future period, and
corrects it using the differences (Δ) between historical reference data from the model and observations.
(OREF = observations in the historical reference period; TREF = GCM output from the historical
reference period; TRAW = raw GCM output for the historical or future period; TBC = bias-corrected
GCM output.)

If we assumed the variability as equal both for GCMs and observations, the daily data is simply shifted by
the mean bias in the reference period (Hawkins et al, 2013), thus:

However, it is possible to apply a more general form of this bias-correction method that corrects not only
the mean values but also the temporal variability of the model output in accordance with the observations
(Hawkins et al, 2013; Ho et al, 2012):
where σT,REF and σo,REF represent the standard deviation in the reference period of the daily GCM
output and observations, respectively. Note that this bias-correction procedure for the GCM output could
be applied to correct both the historical and future periods.

3.1 ​Programming Details:

Programming Language:
​We have used python as the programming language Bias Correction of the Temperature Data.
Libraries Used:
1. Pandas: P ​ andas​ is the most popular python library that is used for data analysis. It
provides highly optimized performance with back-end source code is purely written in ​C
or ​Python​.
2. Numpy:
NumPy is the fundamental package for scientific computing with Python. It contains among other things:

● a powerful N-dimensional array object

● sophisticated (broadcasting) functions
● tools for integrating C/C++ and Fortran code
● useful linear algebra, Fourier transform, and random number capabilities

Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of
generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily
integrate with a wide variety of databases.

3. ​Matplotlib:
Matplotlib is a Python 2D plotting library which produces publication quality figures in a variety of
hardcopy formats and interactive environments across platforms. Matplotlib can be used in Python scripts,
the Python and ​IPython​ shells, the ​Jupyter​ notebook, web application servers, and four graphical user
interface toolkits.
Matplotlib tries to make easy things easy and hard things possible. You can generate plots, histograms,
power spectra, bar charts, error charts, scatterplots, etc., with just a few lines of code. For examples, see
the ​sample plots​ and ​thumbnail gallery​.

For simple plotting the pyplot module provides a MATLAB-like interface, particularly when combined
with IPython. For the power user, you have full control of line styles, font properties, axes properties, etc,
via an object oriented interface or via a set of functions familiar to MATLAB users.

4.​ ​Pylab
Pylab is a programming environment, built on a set of unofficial python tools and libraries, that turns
Python into a high-performance scientific computing platform. The name pylab comes in part from the
resemblance of the resulting environment to MATLAB

The components of pylab have developed largely independently, so there's no unique or "official"
distribution. Nonetheless, there are at least four core components (in addition to the standard python
distribution) required to have an environment that can reasonably be considered a pylab environment.
These are:

NumPy​: this is a set of high-performance libraries (implemented in Fortran and C) that implement
contiguous-memory multidimensional arrays, BLAS and LAPACK linear algebra routines and many
other useful numerical tools. This is listed first because all other components depend on it.

Matplotlib:​ this is pylab plotting library. It is set up to seem familiar to users accustomed to Matlab's
plotting utilities, but it is in many ways much more powerful and flexible (It lets you choose between
many different backend renderers, for example, and allows you to build plots in an object-oriented
manner)

SciPy​: this is a set of mostly distinct modules implementing a variety of really useful scientific computing
tasks, including signal processing, FFTs, optimization, statistics, interpolation, numerical integration, etc.
It contains a lot of stuff that you would expect to see in any good scientific programming environment.
Though it depends strongly on Numpy, it is not as completely integrated into the pylab environment as
are the other components. As a result, you'll need to explicitly import most modules from scipy even if
you've already imported the pylab namespace.

IPython:​ last but certainly not least, ipython is an incredibly powerful alternative python interpreter that
actually defines the combined pylab module/environment and binds the other components of pylab
together.

5.​ ​Scikit-learn:

Scikit-learn provides a range of supervised and unsupervised learning algorithms via a consistent interface
in Python.It is licensed under a permissive simplified BSD license and is distributed under many Linux
distributions, encouraging academic and commercial use.The library is built upon the SciPy (Scientific
Python) that must be installed before you can use scikit-learn. This stack that includes:

● NumPy​: Base n-dimensional array package

● SciPy​: Fundamental library for scientific computing
● Matplotlib​: Comprehensive 2D/3D plotting
● IPython​: Enhanced interactive console
● Sympy​: Symbolic mathematics
● Pandas​: Data structures and analysis

Extensions or modules for SciPy care conventionally named ​SciKits​. As such, the module provides
learning algorithms and is named scikit-learn.The vision for the library is a level of robustness and
support required for use in production systems. This means a deep focus on concerns such as easy of use,
code quality, collaboration, documentation and performance.Although the interface is Python, c-libraries
are leverage for performance such as numpy for arrays and matrix operations, ​LAPACK​, ​LibSVM​ and the
careful use of cython.

Output:
Data Plot for a single Latitude and Longitude for 32 days

Regression line before modeling

 After Modeling Regression line on full Dataset

References:

● https://www.google.com/
● https://www.python.org/
● https://www.geeksforgeeks.org/python-programming-language/
● https://www.w3schools.com/python/