01 - Pandas - Getting Started With Data Analysis

9/13/21, 9:27 AM 01_Pandas_Getting Started with Data Analysis
Corey Schafer Tutorial - Youtube
(1) Python Pandas Tutorial (Part 1): Getting Started

with Data Analysis - Installation and Loading Data
In [1]:
import pandas as pd
In [3]:
df = pd.read_csv('../data/survey_results_public.csv')
In [5]:
# Col & Row count : Shape attribute
df.shape
Out[5]:
(83439, 48)
file:///C:/Users/dasan/Desktop/SORTED/PROJECT/06_TCS/PYTHON_LEARNING/Code/01_Pandas_Getting Started with Data Analysis.html 1/15

In [6]:
df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 83439 entries, 0 to 83438
Data columns (total 48 columns):
ResponseId 83439 non-null int64
MainBranch 83439 non-null object
Employment 83323 non-null object
Country 83439 non-null object
US_State 14920 non-null object
UK_Country 4418 non-null object
EdLevel 83126 non-null object
Age1stCode 83243 non-null object
LearnCode 82963 non-null object
YearsCode 81641 non-null object
YearsCodePro 61216 non-null object
DevType 66484 non-null object
OrgSize 60726 non-null object
Currency 61080 non-null object
CompTotal 47183 non-null float64
CompFreq 52150 non-null object
LanguageHaveWorkedWith 82357 non-null object
LanguageWantToWorkWith 76821 non-null object
DatabaseHaveWorkedWith 69546 non-null object
DatabaseWantToWorkWith 58299 non-null object
PlatformHaveWorkedWith 52135 non-null object
PlatformWantToWorkWith 41619 non-null object
WebframeHaveWorkedWith 61707 non-null object
WebframeWantToWorkWith 52095 non-null object
MiscTechHaveWorkedWith 47055 non-null object
MiscTechWantToWorkWith 38021 non-null object
ToolsTechHaveWorkedWith 72537 non-null object
ToolsTechWantToWorkWith 65480 non-null object
NEWCollabToolsHaveWorkedWith 81234 non-null object
NEWCollabToolsWantToWorkWith 73022 non-null object
OpSys 83294 non-null object
NEWStuck 83052 non-null object
NEWSOSites 83171 non-null object
SOVisitFreq 82413 non-null object
SOAccount 82525 non-null object
SOPartFreq 67553 non-null object
SOComm 82319 non-null object
NEWOtherComms 82828 non-null object
Age 82407 non-null object
Gender 82286 non-null object
Trans 80678 non-null object
Sexuality 73366 non-null object
Ethnicity 79464 non-null object
Accessibility 77603 non-null object
MentalHealth 76920 non-null object
SurveyLength 81711 non-null object
SurveyEase 81948 non-null object
ConvertedCompYearly 46844 non-null float64
dtypes: float64(2), int64(1), object(45)
memory usage: 30.6+ MB

In [7]:
pd.set_option('display.max_columns', 100)

In [8]:
df

Out[8]:
ResponseId MainBranch Employment Country US_State UK_Country EdLeve
Seconda
Independent
I am a school (e.g
contractor,
0 1 developer by Slovakia NaN NaN America
freelancer, or
profession high schoo
self-em...
G
I am a Bachelor
student who Student, full- degre
1 2 Netherlands NaN NaN
is learning to time (B.A., B.S
code B.Eng., etc
I am not
Bachelor
primarily a
Student, full- Russian degre
2 3 developer, NaN NaN
time Federation (B.A., B.S
but I write
B.Eng., etc
co...
Master
I am a degre
Employed
3 4 developer by Austria NaN NaN (M.A., M.S
full-time
profession M.Eng
MBA, etc
United Master
Independent
I am a Kingdom of degre
contractor,
4 5 developer by Great NaN England (M.A., M.S
freelancer, or
profession Britain and M.Eng
self-em...
Northern I... MBA, etc
... ... ... ... ... ... ...
Bachelor
I am a United
Employed degre
83434 83435 developer by States of Texas NaN
full-time (B.A., B.S
profession America
B.Eng., etc
Independent Bachelor
I am a
contractor, degre
83435 83436 developer by Benin NaN NaN
freelancer, or (B.A., B.S
profession
self-em... B.Eng., etc
Seconda
I am a United school (e.g
Employed New
83436 83437 developer by States of NaN America
full-time Jersey
profession America high schoo
G
Bachelor
I am a
Employed degre
83437 83438 developer by Canada NaN NaN
full-time (B.A., B.S
profession
B.Eng., etc
I am a Profession
Employed
83438 83439 developer by Brazil NaN NaN degree (JD
full-time
profession MD, etc
83439 rows × 48 columns

In [9]:
df_schema = pd.read_csv('../data/survey_results_schema.csv')
pd.set_option('display.max_rows', 100)

In [10]:
df_schema

Out[10]:
qid qname question force_resp type selector
<div>Hel...
1 QID12 MetaInfo Browser Meta Info False Meta Browser
<span style="font-size:22px;
font-family: aria...
Which of the following

3 QID2 MainBranch True MC SAVR
options best describes ...
Which of the following best

4 QID24 Employment False MC MAVR
describes your cur...
Where do you live? In which state or territory

6 QID7 US_State False MC DL
of the USA do y...
In which part of the United

7 QID9 UK_Country False MC DL
Kingdom do you liv...
Which of the following best

9 QID25 EdLevel False MC SAVR
describes the high...
At what age did you write

10 QID149 Age1stCode False MC MAVR
your first line of c...
How did you learn to code?

11 QID276 LearnCode False MC MAVR
Select all that apply.
Including any education, how

12 QID32 YearsCode False MC DL
many years have y...
NOT including education,

13 QID34 YearsCodePro False MC DL
how many years have y...

14 QID31 DevType False MC MAVR
describes your current ...
Approximately how many

15 QID29 OrgSize False MC MAVR
people are employed by ...
Which currency do you use

16 QID50 Currency True MC SB
day-to-day? If your ...
What is your current total

17 QID51 CompTotal False TE SL
compensation (salar...
Is that compensation weekly,

18 QID52 CompFreq False MC MAVR
monthly, or yearly?
Which programming,
20 QID233 Language False Matrix Likert
scripting, and markup la...
Which database
21 QID262 Database environments have you False Matrix Likert
do...
Which cloud
22 QID263 Platform platforms have you done False Matrix Likert
ext...

Which web
23 QID264 Webframe frameworks other frameworks

24 QID265 MiscTech False Matrix Likert
and libraries ha...
Which
25 QID275 ToolsTech tools have False Matrix Likert
you done ext...
Which development
26 QID274 NEWCollabTools False Matrix Likert
environments</strong...
What is the primary

27 QID71 OpSys False MC SAVR
operating system in which ...
What do you do when you

28 QID243 NEWStuck False MC MAVR
get stuck on a problem...
Which of the following Stack

30 QID266 NEWSOSites False MC MAVR
Overflow sites ha...
How frequently would you

31 QID100 SOVisitFreq False MC MAVR
say you visit Stack O...
Do you have a Stack

32 QID101 SOAccount False MC MAVR
Overflow account?
How frequently would you

33 QID102 SOPartFreq False MC MAVR
say you participate i...
Do you consider yourself a

34 QID106 SOComm False MC MAVR
member of the Stack...
Are you a member of any

35 QID267 NEWOtherComms False MC MAVR
other online developer...
Please name up to 5 other

36 QID268 NEWOtherCommsNames False Matrix Likert
online developer com...
38 QID127 Age What is your age? False MC MAVR

39 QID122 Gender False MC MAVR
describe you, if any? P...
Do you identify as
40 QID153 Trans False MC MAVR
transgender?

41 QID136 Sexuality False MC MAVR

42 QID126 Ethnicity False MC MAVR

43 QID124 Accessibility False MC MAVR

44 QID125 MentalHealth False MC MAVR
Final False DB TB
Qu...
How do you feel about the

46 QID132 SurveyLength False MC MAVR
length of the survey...

How easy or difficult was this

47 QID133 SurveyEase False MC MAVR
survey to compl...

In [11]:
df.head(40)

Out[11]:
ResponseId MainBranch Employment Country US_State UK_Country E
Independent Se
I am a
contractor, sch
0 1 developer by Slovakia NaN NaN
freelancer, or Americ
profession
self-em... sch
I am a Ba
1 2 Netherlands NaN NaN
is learning to time B.S.,
code
I am not
Ba
primarily a
Student, full- Russian degre
2 3 developer, NaN NaN
time Federation B.S.,
but I write
co...
Master’s
I am a
Employed (M.A
3 4 developer by Austria NaN NaN
full-time M.Eng
profession
United
Independent Master’s
I am a Kingdom of
contractor, (M.A
4 5 developer by Great NaN England
freelancer, or M.Eng
profession Britain and
self-em...
Northern I...
I am a Ba
United
student who Student, part- degre
5 6 States of Georgia NaN
America
code
Se
I code United
I prefer not to New sch
6 7 primarily as States of NaN
say Hampshire Americ
a hobby America
sch
I am a Ba
7 8 Malaysia NaN NaN
code
Ba
I am a
Employed degre
8 9 developer by India NaN NaN
part-time B.S.,
profession
Master’s
I am a
Employed (M.A
9 10 developer by Sweden NaN NaN
full-time M.Eng
profession
United
Ba
I am a Kingdom of
Employed degre
full-time B.S.,
Northern I...
Ba
I am a
Employed degre
11 12 developer by Spain NaN NaN
full-time B.S.,
profession

Master’s
I am a
Employed (M.A
12 13 developer by Germany NaN NaN
full-time M.Eng
profession
I am a Master’s
student who Employed (M.A
13 14 Peru NaN NaN
is learning to full-time M.Eng
code
I used to be United
Not Ba
a developer Kingdom of
employed, degre
14 15 by Great NaN England
but looking B.S.,
profession, Britain and
for work
but no... Northern I...
I am a Se
United
student who Student, full- sch
15 16 States of Missouri NaN
is learning to time Americ
America
code sch
Ba
I am a
Employed degre
16 17 developer by Turkey NaN NaN
full-time B.S.,
profession
Ba
I am a
Employed degre
17 18 developer by Canada NaN NaN
full-time B.S.,
profession
I am not
primarily a Other
I prefer not to
18 19 developer, Singapore NaN NaN degree
say
but I write Ed.
co...
I used to be
Ba
a developer
Employed degre
19 20 by Brazil NaN NaN
full-time B.S.,
profession,
but no...
Ba
I am a
Student, full- degre
time B.S.,
profession
I am a Ba
21 22 Sweden NaN NaN
code
I am a
student who Student, full- college/u
22 23 India NaN NaN
is learning to time study
code ea
Master’s
I am a
Employed (M.A
23 24 developer by France NaN NaN
full-time M.Eng
profession
Master’s
I am a
Employed (M.A
full-time M.Eng
profession

I am not
primarily a Other
Employed
25 26 developer, Germany NaN NaN degree
full-time
but I write Ed.
co...
I am not
Master’s
primarily a
Employed (M.A
26 27 developer, Switzerland NaN NaN
full-time M.Eng
but I write
co...
United
Ba
I am a Kingdom of
Employed degre
full-time B.S.,
Northern I...
United
Independent Se
I am a Kingdom of
contractor, sch
freelancer, or Americ
self-em... sch
Northern I...
Master’s
I am a
Employed Russian (M.A
29 30 developer by NaN NaN
full-time Federation M.Eng
profession
I code
I prefer not to
30 31 primarily as Malawi NaN NaN Someth
say
a hobby
Ba
I am a
Employed degre
31 32 developer by Israel NaN NaN
full-time B.S.,
profession
Not Master’s
I am a
employed, (M.A
32 33 developer by Poland NaN NaN
but looking M.Eng
profession
for work
Ba
I am a
Employed degre
33 34 developer by Turkey NaN NaN
full-time B.S.,
profession
Master’s
I am a
Employed (M.A
34 35 developer by Ukraine NaN NaN
full-time M.Eng
profession
Master’s
I am a
Employed (M.A
full-time M.Eng
profession
Ba
I am a United
Employed District of degre
36 37 developer by States of NaN
full-time Columbia B.S.,
profession America

I am a United
Employed college/u
37 38 developer by States of Massachusetts NaN
full-time study
profession America
ea
I am a Not Ba
student who employed, degre
38 39 Viet Nam NaN NaN
is learning to but looking B.S.,
code for work
I am a Ba
student who Student, part- degre
39 40 India NaN NaN
code

01 - Pandas - Getting Started With Data Analysis

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

01 - Pandas - Getting Started With Data Analysis

Uploaded by

Copyright:

Available Formats

9/13/21, 9:27 AM 01_Pandas_Getting Started with Data Analysis

Corey Schafer Tutorial - Youtube

(1) Python Pandas Tutorial (Part 1): Getting Started

# Col & Row count : Shape attribute

file:///C:/Users/dasan/Desktop/SORTED/PROJECT/06_TCS/PYTHON_LEARNING/Code/01_Pandas_Getting Started with Data Analysis.html 1/15

RangeIndex: 83439 entries, 0 to 83438

Data columns (total 48 columns):

ResponseId 83439 non-null int64

MainBranch 83439 non-null object

Employment 83323 non-null object

Country 83439 non-null object

US_State 14920 non-null object

UK_Country 4418 non-null object

EdLevel 83126 non-null object

Age1stCode 83243 non-null object

LearnCode 82963 non-null object

YearsCode 81641 non-null object

YearsCodePro 61216 non-null object

DevType 66484 non-null object

OrgSize 60726 non-null object

Currency 61080 non-null object

CompTotal 47183 non-null float64

CompFreq 52150 non-null object

LanguageHaveWorkedWith 82357 non-null object

LanguageWantToWorkWith 76821 non-null object

DatabaseHaveWorkedWith 69546 non-null object

DatabaseWantToWorkWith 58299 non-null object

PlatformHaveWorkedWith 52135 non-null object

PlatformWantToWorkWith 41619 non-null object

WebframeHaveWorkedWith 61707 non-null object

WebframeWantToWorkWith 52095 non-null object

MiscTechHaveWorkedWith 47055 non-null object

MiscTechWantToWorkWith 38021 non-null object

ToolsTechHaveWorkedWith 72537 non-null object

ToolsTechWantToWorkWith 65480 non-null object

NEWCollabToolsHaveWorkedWith 81234 non-null object

NEWCollabToolsWantToWorkWith 73022 non-null object

OpSys 83294 non-null object

NEWStuck 83052 non-null object

NEWSOSites 83171 non-null object

SOVisitFreq 82413 non-null object

SOAccount 82525 non-null object

SOPartFreq 67553 non-null object

SOComm 82319 non-null object

NEWOtherComms 82828 non-null object

Age 82407 non-null object

Gender 82286 non-null object

Trans 80678 non-null object

Sexuality 73366 non-null object

Ethnicity 79464 non-null object

Accessibility 77603 non-null object

MentalHealth 76920 non-null object

SurveyLength 81711 non-null object

SurveyEase 81948 non-null object

ConvertedCompYearly 46844 non-null float64

dtypes: float64(2), int64(1), object(45)

memory usage: 30.6+ MB

file:///C:/Users/dasan/Desktop/SORTED/PROJECT/06_TCS/PYTHON_LEARNING/Code/01_Pandas_Getting Started with Data Analysis.html 2/15

file:///C:/Users/dasan/Desktop/SORTED/PROJECT/06_TCS/PYTHON_LEARNING/Code/01_Pandas_Getting Started with Data Analysis.html 3/15

file:///C:/Users/dasan/Desktop/SORTED/PROJECT/06_TCS/PYTHON_LEARNING/Code/01_Pandas_Getting Started with Data Analysis.html 4/15

ResponseId MainBranch Employment Country US_State UK_Country EdLeve

... ... ... ... ... ... ...