INDIAN INSTITUTE OF MANAGEMENT, AHMEDABAD

Big Data

Rise of Big Data for the field of Economics

Individual Assignment #3

Memo#3

By

Viral Mehta

ePGP B#2 (2018-20), SEM -IV

Student ID: 2238408

Submitted on: 24.04.2020

1|Page
Traditionally, economic science evolution based on empirical work by researchers which, with big
data, experiencing significant changes for better. Earlier the research based on sample data from
Government/specifically carried out sample surveys, now can use administrative data, private data
and may combine both. This transition from small sample to universal/near universal coverage of
population may advance research on several dimensions and lead to rigorous examination of
variations in different measures across population, track direct outcome from experiments
(natural/stimulated) and generate new research designs. This lead to analysis of better measurement
of economic effect with inclusion of broader range of economic variables and interactions.

Until 1980, most economic papers were in form of theoretical research, few based on government
data available with a lag. Data collection for an economic research was an uphill task. With big data,
scenario transformed, quality and quantity of data in economic activities are expanding rapidly. Now,
the administrative and private data both are available in real time, continuously updated, aid a great
value guiding economic policy. Data now available on previously unmeasured activity, e.g. personnel
communication, social networks, search and information gathering, geolocation etc., allows studying
issues that earlier thought to be difficult. Administrative data offer advantages compared to
traditional survey data, handles missing data well, sample selection and attrition no longer an issue,
helps causal inferences and policy evaluation.

Internet companies in every sector collect data and aggregate with primary purpose of business use,
there are potential research application in economics and other fields e.g. to create statistics on
aggregate economic activities that can be used to track the economy or as input to other research.
Collaboration with private firms also give rise to structured experiments with large-scale granular data
useful accessing the robustness of identifying assumption.

Big data using applied models of economics are creating exciting opportunities e.g. design of online
advertising auctions and exchanges by combine big data predictive model with sophisticated
economic market mechanisms. A dramatic expansion of this application is predicting the present with
google trends. Use of search engine data to forecast near term values of economic indicators e.g.
automobile sales, unemployment claims, travel destination, consumer confidence etc.

Economic research using large datasets have relied primarily on traditional econometric techniques.
With advent of machine – learning where data speaks for itself becomes more efficient and useful.
This new econometric tool with powerful capabilities solves different problems e.g., uncover
generalized patterns; identify unspecified structures, fits complex and very flexible functional forms
to the data without overfitting. ML is easy to uses as it seeks functions that predict well out of sample
and provides high dimensionality with flexible function forms.

Path is not smooth, challenges in terms of access, data management and programming capability,
creative approaches to summarize, describe and analyse, privacy and confidentiality concerns but, the
opportunities far exceeds challenges put together.

Big data statistical models can complement traditional econometric techniques, should extend the
frontiers of economics and pull previously unanswerable questions within reach.

No of words: 492

2|Page
References:

 ePGP class discussions

 Economics in the age of big data Liran Einav & Jonathan Levin
 Predicting the present with Google Trends – Choi & Varian
 Economists are prone to fads, and the latest is machine learning – Finance and Economics –
Nov 24 – 2016 edition
 Machine Learning – an applied econometric approach – Sendhil Mullainathan & Jann Spiess

3|Page