BUILD A DATA WAREHOUSE WITH SQLITE

Achmad Adyatma Ardi1

Programmer, Data Scientist, Engineer
*Corresponding author : achmad541997@gmail.com
Saturday, June 4, 2022

Abstract

Data science is a discipline where we try to answer questions that is needed instead of
using a mere assumption, we use data and logical thinking to answer that question
with a belief. In a business purposes, even though the value of uncertainty is always
there, it would be wise to make decisions based on the data you have. Data can be
your best friend in making important decisions. On this occasion, I will try to analyze
the data with the aim of answering important business-related questions using the
Python programming language with Pandas and Matplotlib libraries. Hope you enjoy
it!

Keyword : data science, python programming, sales analysis, python pandas, python
matplotlib, answer questions

Table of contents :
Ch. 1 Introduction to Data Warehousing........................1
…1
1
2
2
Ch. 2 SQL Lite...............................................................2

Additional attachment :
Attachment. 1 Example of a database...................3
Attachment. 2.......................................................3
Attachment. 3.......................................................3
Attachment. 4.......................................................3
A data warehouse is mainly a data
management system that’s designed to
Ch. 1 Introduction to Data enable and support business
Warehousing intelligence (BI) activities, particularly
analytics. Data warehouses are alleged

1 | Achmad Adyatma Ardi

to perform queries, cleaning,
manipulating, transforming and
analyzing the data and they also
contain large amounts of historical
data.
I. 1 Usage
Data warehouse allows users to
access critical data from the number of
sources in a single place. Therefore, it
saves user’s time of retrieving data
from multiple sources. Data warehouse
stores a large amount of historical
data. This helps users to analyze
different time periods and trends to
make future predictions. Below are 7
key benefits of data warehousing :
1. Saves time
2. Improves data quality
3. Improves business intelligence
4. Leads to data consistency
5. Enhances return on investment
(ROI)
6. Stores historical data
7. Increases data security
I. 2 Types of Data Warehouse
(DWH)
Three main types of data
warehouses (DWH) are :
1. Enterprise Data Warehouse
(EDW)
Enterprise Data Warehouse
(EDW) is a centralize
warehouse. It provides decision
support service across the
enterprise. It offers a unified
approach for organizing and
representing data. It also provide
the ability to classify data
according to the subject and give
access according to those
divisions
2. Operational Data Store (ODS)
Operational Data Store, which
is also called ODS, are nothing
but data store required when
neither data warehouse nor
OLTP systems support
organizations reporting needs. In
ODS, data warehouse is
refreshed in real time. Hence, it
is widely preferred for routine
activities like storing records of
the employees
3. Data Mart
A data mart is a subset of the
data warehouse. It specially
designed for a particular line of
business, such as sales, finance,
sales or finance. In an
independent data mart, data can
collect directly from sources.
I. 3 Warehousing Processes
Data warehouse generalize and
consolidate data in multi –
dimensional spaces. The construction
of data warehouses involves data
cleaning, data integration, and data
transformation. The data can be
viewed as an important step for data
mining. Data warehouse provides
architectures and tools for business
executives to systematically organize,
understand and use their data to make
strategic decisions. The information
gathered in a warehouse can be used in
any of the following domains (Figure
1):
1. Tuning production strategies
2. Customer analysis / customer
behaviour
3. Operation analysis
2 | Achmad Adyatma Ardi

Figure 1 Data Warehouse
There are four major processes that
contribute to a data warehouse :
1. Extract and load the data
2. Cleaning and transforming the
data
3. Backup and archive the data
4. Managing queries and directing
them to the appropriate sources
I. 4 Components of Data
Warehousing
A typical data warehouse has four
main components : (1) Central
database, (2) ETL (extract, transform,
load) tools, (3) metadata, (4) and
access tools. All of these components
are engineered for speed so that you
can get results quickly and analyze
data on the fly.
1. Central database
A database serves as the
foundation of your data
warehouse. Traditionally these
have been standard relational
database running on premise or
in the cloud. But because of big
data, the need for true, real –
time performance, and a drastic
reduction in the cost of RAM, in
– memory databases are rapidly
gaining in popularity.
2. Data integration (ETL tools)
Data is pulled from source
systems and modified to align
the information for rapid
analytical consumption using a
variety of data integration
approaches such as ETL (extract,
transform, load) and ETL as well
as real – time data replication,
bulk – load processing, data
transformation, and data quality
and enrichment services
3. Metadata
4. Data warehouse access tools
https://www.sap.com/
insights/what-is-a-data-
warehouse.html#:~:text=A
%20typical%20data%20warehouse
%20has,analyze%20data%20on
%20the%20fly.
Figure 2 The components of a data warehouse
I. 5 Data Warehousing Tools
Ch. 2 SQL Lite
In machine learning (and in
mathematics) there are often three
values that interests us :
1. Mean – the average value
2. Median – the mid point value
3. Mode – the most common value
3 | Achmad Adyatma Ardi
References

https://pynative.com/python-mysql-
database-connection/

https://articles.cyzerg.com/warehouse-
processes-how-to-optimize-
them#:~:text=11%20MIN
%20READ-,The%20six
%20fundamental%20warehouse
%20processes%20comprise
%20receiving%2C%20putaway%2C
%20storage%2C,a%20higher
%20perfect%20order%20rate.

4 | Achmad Adyatma Ardi

Attachment. 1 Example of a database

Carname Color Age Speed AutoPass

BMW red 5 99 Y
Volvo black 7 86 Y
VW gray 8 87 N
VW white 7 88 Y
Ford white 2 111 Y
VW white 17 86 Y
Tesla red 2 103 Y
BMW black 9 87 Y
Volvo gray 4 94 N
Ford white 11 78 N
Toyota gray 12 77 N
VW white 9 85 N
Toyota blue 6 86 Y

Attachment. 2
Attachment. 3
Attachment. 4

5 | Achmad Adyatma Ardi