Professional Documents
Culture Documents
- Most of us have rented cars while on vacation or on a business trip. When you arrive at
your destination airport, you go to the rental car facility. The lines are long, but no
worries, because you're a gold preferred customer. So you head straight to the marquee
board, expecting to find your name and car listed there. Surprise, surprise, your name is
nowhere to be found. So now, you have to go into the rental car office, hoping to speak
to someone at the gold preferred counter. But that counter is closed for the night. So
now, you have to stand in that long line with the regular customers. But wait, your
nightmare is not over yet. When you finally get to the counter after that long wait, you
are told that they ran out of cars and the only vehicles available are luxury SUVs for a
higher rate. What's wrong with this picture? You had a reservation. You're a VIP, a gold
preferred customer. But all that went out the window. You had a difficult and bad
experience, not better. It took longer and it cost more. Operationally, it was far from
excellent. What does it mean to be operationally excellent? In general, what is operation
excellence? Simply put, it is better, faster, and cheaper. Better. Better quality of products
and services, better processes, and better user experience, and better value. Better in
whatever's relevant and important. For example, if safety and reliability are
important, then better safety and better reliability. Better means improved performance
on metrics in the quality dimension. Quality in its broadest sense. Faster. Faster service,
faster response, faster processing, and on-time or faster delivery, if customers
prefer. Faster means improved performance on metrics in the time or temporal
dimension. Cheaper. Cheaper to operate, cheaper to process, and cheaper for
customers. Cheaper means being more efficient in the use of resources and improved
performance on metrics in the cost dimension. To achieve operational
excellence, organizations needs to have processes that are effective and efficient in
delivering value. Processes need to be well-designed, capable, and consistent. Where
people perform the work are capable and have the means of knowing what's important,
what to do, when to take action, and what actions to take. They have the necessary
process authority, responsibility, and accountability. Organizations also need to have the
tools and techniques for design, improvement, and control. For example, for design,
there's Design for Six Sigma. For improvement, there's Six Sigma and Lean. For control,
there's Value Stream or Process Management. In addition, operational excellence means
having the mindset and behaviors where everybody wants to and is able to be
operationally excellent. The right mindset is embraced and the right behaviors are
encouraged by the leadership, enabled by targets and metrics, and encouraged
by performance goals, rewards, and recognition. Finally, an organization cannot
achieve operational excellence without four things. Enterprise-wide alignment of
strategies, priorities, policies, and decisions. This has to be across different functions and
up and down all levels of the organization to consistently drive the right behaviors and
results. These four elements are essential to achieve operational excellence. To be
better, faster, and cheaper. You want to get to the point when everyone in the
organization is willing and is able to do well, to be operationally excellent
everyday. Everyone is engaged, enabled, and empowered to achieve operational
excellence and the desired results for competitive advantage. So the next time you rent
a car, hopefully you have a much better experience, receive faster service at a cheaper
price, and drive off in a car you reserved.
Variation
- A customer places an order and is told it will take two to 10 days for delivery. You
make a request for a transaction in your company and you're told it will happen in five
to 10 days. Why is there so much variation? A more important question is what
happens when you try to promise, budget, staff, schedule, and plan on getting things
done correctly and on time when there is so much variation? The answer is with a lot of
difficulty, uncertainty, increased costs, and mixed results and performance such as not
meeting deadlines, not meeting surface level agreements, and other contractual
obligations. Not to mention, upsetting customers and potential loss of business. Let's
understand variation. Let's say there are five activities or processing steps needed to
complete a transaction. These steps could be mapped onto many different kinds of
processes. So, think about a process that's relevant to your work. It can be
manufacturing, services, or healthcare. But no matter what the process is, there is
variation in each step. In step one, it may take anywhere from one to four hours. Step
two, takes one to eight hours. Step three, one to three hours. Step four, one to six, and
step five, one to three hours. So the entire process can take anywhere from five to 24
hours. Mathematically speaking, averages or means add up, but standard deviations do
not. It is a square of the standard deviations. In other words, it is the variances that add
up. The variance of the total processing time is the sum of the variances of each of the
five steps. That is why you see a very wide processing time curve for the whole
process. Variation does not just impact time performance. Variation also impacts how
well specifications are met or not met. In other words, variation impacts quality. There
are specifications for services and products. These specifications are expressed as
specification limits. Here's the curve showing an example of the distribution compared
to the specification limits. The performance is off target and has wide variation. Because
large areas of the curve are outside the lower and upper specification limits, defects
occur frequently. While companies make changes to adjust the mean or average to be
on target, as shown here, the wide variation still results in performance outside the
specification limits. In other words, defects still occur. How often do companies report
only means or averages? And when the mean hits a target, they celebrate, or should
they? For example, we did very well last month. Our average or mean delivery time was
33 hours and that is less than our 34 hour guarantee. But then, why are there customer
complaints on late deliveries? This diagram shows why. It is the variation. Variation has
to be reduced to improve performance. Section marked projects are excellent for
reducing variation. Here's the result. With performance on target and the variation
reduced, there is a huge reduction in defects, quality improves. Let me conclude by
sharing an everyday example. How many of us have experienced shopping online and
buying products which states assembly required? What if there is variation in the size of
each of the parts or components beyond their specification limits? Well, as we learned
earlier, variances do add up. So the cabinet that you order online might not fit properly
together when you try to assemble it. Doors hang unevenly, screws don't fit in
holes. This is because of variation. That is why, where possible, I avoid buying any
product that requires assembly. So the next time anyone brags about average
performance being on target, ask them about variation.
Error-proofing or poka-yoke
- When you shop online, the credit card number you input has to be accurate, otherwise
you cannot complete your order. That's an example of error proofing, or mistake
proofing. In lean, it's often called poka-yoke from the Japanese terms (speaks in foreign
language) meaning mistake, and (speaks in foreign language) meaning to avoid. To
avoid inadvertent error or human error. Wouldn't it be useful to error proof or mistake
proof your processes, products, and services? Let's discuss the basic principles of error
proofing and how they can be applied. Error proofing is best when it prevents error
from occurring. If that's not possible, the next best thing is to facilitate the work so that
errors are minimized. Lastly, if errors do take place, then detection should be
made obvious and immediate, or be automated. There are basically three levels of error
proofing. Here they are in order of preference. One, prevention. Two, facilitation. Three,
detection. The first and most preferred is error proofing by prevention. An example is
the traction control system in cars. It works actively full-time to prevent wheels from
over spinning on slippery roads or low friction surfaces. Regardless of how good or bad
the weather is, or the driver is. Prevention-based error proofing is also possible in
processes. For example, in-store pickup of a prepaid online order. In addition to
showing a driver's license, the same credit card must be scanned for the system to
authorize and process the pick up. If prevention is not possible, then the next choice is
error proofing by facilitation. An example of this is anti-lock brakes, in which the car
helps facilitate your application of brakes in an emergency, so it is done efficiently,
effectively, and the brakes don't lock up. Similarly, in processing transactions, dedicated
pre-programmed buttons such as those on point of sales systems, for cheeseburger or
large fries are used. Also, you can connect your point of sales system to inventory
management. It can help you track inventory and prompt you when and how much to
order. Or as I've experienced recently, many easy to assemble furniture are not easy to
assemble. However, I've come across some well-designed truly easy to assemble
furniture where parts to be assembled to each other are labeled by the same matching
numbers and the parts fit perfectly. Facilitation-based error proofing makes it easier and
it minimizes errors, but it does not prevent non-compliance due to forgetfulness or
human error such as pressing the wrong button by mistake. If prevention or facilitation
is not possible, then employ the third choice, error proofing by detection, where
detection of errors is immediate, either by being made obvious or by automated
detection. An example is the annoying beeping sound and warning light in your car to
alert you when you forget to put on your seatbelts. Another example, is the fuel
gauge where the warning light comes on when the gas tank is low with a range of less
than 30 miles left. The only trouble is, for me, when I finally notice the light, I'm never
quite sure if it just came on or if it came on 25 miles ago. To summarize, if prevention is
not possible, use a combination of facilitation and detection. An example is a
combination of anti-lock brakes, seatbelts, and airbags. So prevent errors if possible. If
not, minimize errors or at the very least, detect and mitigate these effects immediately.
Lean principles
- Remember the last time you were delayed, or when you made multiple trips to a
customer work site? Or when you had to rework or redo a task? Or spend time, effort,
and money to hurry up only to wait at the next step? Bottom line, let's just say what
you've experienced is simply not lean. Lean can be viewed as a management
philosophy, a mindset, a methodology, a tool set, or an approach to daily work. At it's
core, it's all about minimizing wastes and maximizing value to customers. Value is what
customers need and expect and are willing to pay for. This includes receiving the right
products and services at a specified price, time, and place. For example, if you see an
extra charge in your invoice because warehouse personnel spent five hours to look at
your item, would you want to pay for that? Of course not! It is of no value to you. Waste
is anything that is of no value or adds no value for customers. Waste comes in many
forms. Eight types of waste have been identified. I know a guy named Tim Woods
who can help us remember it all. TIMWOODS is an acronym. T for transport, that's
movement of people, materials, products, or documents between activities or
locations. I for inventory. Just in case additional inventory whether it is raw materials,
work in process, or finished goods inventory. M for motion, movement within an
activity that does not add value for the customer. W for waiting, time wasted between
activities, waiting for required resources, materials, parts, people, or information. O for
overproduction, producing more than what's needed. For example, making 20 copies
when only two were needed. O for overprocessing, inappropriate, excessive
processing and unnecessary duplication of work that adds no value. For example,
polishing your presentation slides repeatedly after multiple rehearsals before presenting
to senior management. D for defects, defective work, defective items, or any undesired
outcome that adds no value. S for skills underutilized. This refers to underutilization of
employee skills and intellect such as when a highly qualified scientist or engineer is
assigned to administrative work. So that's TIMWOODS, the eight types of waste. Lean
targets the elimination of waste to improve efficiency, flow, and speed. Here are some
key concepts of lean. Value stream, a value stream is all those processes, activities, and
resources, including information, used to transform inputs into outputs that are sealable
to customers. The elimination of waste will improve efficiency, speed, and flow of the
value stream. Theory of constraints, the output of a value stream is only as fast as the
slowest processing step, bottleneck, or constraint. To improve the rate of output or
throughput, focus on improving the constraint until it is no longer a limiting factor. That
is the crux of the theory of constraints, developed by Eli Goldratt. Pull instead of push,
customer demand pulls the order, product, or transaction throughout the value stream
from suppliers to customers. Traditionally, products have been pushed to customers by
suppliers or producers, regardless of whether they want it or not or if they're ready to
receive it or not. That is why you see your end clearance sales. Pull, on the other hand, is
about customer demand pulling and authorizing work and delivery as and when it is
needed. Just-in-time flow, items or transactions should be produced, processed, or
delivered just-in-time, as it is needed, at the same rate as customer demand. If goods or
services are produced to the drum beat or rate of customer demand, where a demand
for an item signals a production and delivery of that item from the next upstream
workstation then there's no need for inventory. When you find yourself redoing
something, making unnecessary trips, waiting in line, or stuck with items you don't
need, fix the issue by applying lean concepts.
Process mapping
- They say that a picture is worth a thousand words. Yes, it is much easier if you are able
to visualize what you're trying to understand, manage or improve. A process map is a
diagram that provides a visual representation of the process flow, or a sequence of
activities or steps that take place in a process from start to finish. There are different
types of process maps. At the highest level is the high-level process map. This provides
a view of the process at 10,000 feet high. A high-level process map displays the main
activities of major steps in the process. Usually showing a whole process in 10 or fewer
major steps. If any of these major process steps needs more granularity to be better
understood, utilize process decomposition. This is when you drill down or decompose
those specific steps into more detail using detailed process maps. A detailed process
map provides sufficient granularity to enable the project team to understand what is
going on, not going on, and display where the decisions, re-work loops, delays,
bottlenecks, and walk arounds occur. If multiple groups are involved in a process, then a
swimlane process map will be useful to map across functional process. Think of a
swimlane process map as a detailed process map that has been allocated in the
respective lanes where the activities are performed. Visualize an Olympic swimming
pool where you have multiple lanes. One for each swimmer. A lane for each group
function or department involved in the cross functional process. A swimlane process
map is also called a deployment map because it shows where the work is deployed. You
may ask, so what, why bother with a swimlane process map? A swimlane map shows
which group or department is performing each process step. And where the handoffs
are. Handoffs are the weak links in a process. Where things can fall through the
cracks due to miscommunication between departments resulting in delays, mistakes and
defects. Being able to see these opportunities for failure is very useful. So if you have a
cross functional process, map it using a swimlane process map. Another type of process
map that has gained popularity with lean and operational excellence is the value stream
map. A value stream map is a diagram that shows the major steps involved in getting a
product or service from supplier to customer. It shows the material and information
flows from order to delivery. It is basically a high-level process map with additional
information, such as customer data, processing data, and information flows pertinent to
the value stream. At a glance, we can see the end-to-end process. From order to
delivery. Or from check-in to check-out. It shows the flow of information and
material including process steps, processing time, cycle time and the number of
servers. We can see the backlog or work in process, in front of each process step. The
timeline at the bottom of the map shows the lead time and actual processing times. The
value stream map provides a snapshot of entire value stream and its performance. To
summarize, the different types of process maps discussed are: high-level process
map, detailed process map, swimlane process map or deployment map, and value
stream map. Using these process maps, you can create a picture of the process that is
really worth a thousand words.
Implementation challenges
- There are many challenges in implementing and sustaining operational
excellence. Here are some common implementation challenges. Fear of headcount
reduction and job losses. Many employees view operational excellence, especially lean,
as a means by management to lean out the organization, resulting in job cuts. Rightly or
wrongly, that's the view and fear. Senior management has to address this fear head
on. Not through pronouncements, but through actions. When processes of value
streams become more efficient, any displaced workers should be reassigned to fill useful
roles elsewhere within the company. And as the company grows, these folks can fill new
roles, first removing any need for new hires. Lack of buy-in and acceptance. Most people
don't like change. To improve buy-in, and acceptance of changes and improvements,
ensure that the right metrics and performance targets are implemented. Metrics and
rewards that motivate the right mindsets, behaviors, and results much support the
changes. For example, traditionally incentive pay is based on piece count or number of
transactions. Instead, reward based on the percentage of defect-free pieces produced or
the percentage of correct and accurate transactions processed. Reverting back to old
habits and the good old days. It's just human nature to take the easiest path or the path
of least resistance. Ensure that the new way of doing the work is easier than the old
way. If it is more cumbersome and more involved, it will not be sustainable. Employees
should love the new improved way of getting things done because it is now easier and
more user-friendly than ever. Keep it simple. Too many initiatives and projects. Most
organizations are already stretched thin with very lean staffing levels. It is not
uncommon to here that everyone's busy with their regular jobs and to top it off, there
are too many projects in progress, tying up whatever little time and resources there are
left. A common challenge is that there's no time and no resources available for
projects. To address this, take inventory of all existing projects and initiatives. Map them
to the company's annual goals and strategies to see if there is any connection. If there is
no connection, let alone alignment, then either kill those projects, put a hold, or
postpone them. For the remaining projects, prioritize them to gather the new
operational excellence projects. Such a mapping and alignment exercise will reduce the
number of ongoing projects. And to keep it streamlined, senior management can
proactively select and prioritize the list of projects. Too many metrics. This is another
common challenge when in the rush and temptation to measure and be data-driven, the
list of metrics to be monitored and controlled keeps growing. As the list grows, so
grows the frustration of employees. To address this, evaluate the reason for every
metric by checking to see how well the metric maps through CTQs and all the
company's business priorities. And any KPIs or key performance indicators should be
limited to just the key ones. Imagine the stress and nightmare if you drive a car with a
dashboard showing more than a dozen indicators on digital displays. You don't need to
measure everything, just what matters. These common challenges should be addressed
head on. Management must follow through, actively demonstrate their own buy-in and
walk the talk. It is management's responsibility to make the new operationally excellent
way of working a routine. Everyone in the organization will then be engaged, and able,
and empowered to achieve the desired results.
Next steps
- Congratulations. You have made it to the end of this course. Operational excellence is all about
getting work done better, faster, cheaper, while delivering superior value to customers. Just a
review. To achieve operational excellence, you need to have processes that are effective and
efficient at delivering value, tools and techniques for design, improvement, and control, the right
mindset and behaviors where everybody wants to and is able to be operationally excellent. And
enterprise wide alignment of strategies, priorities, and decisions. If you are interested in learning
more, I recommend watching my courses on Six Sigma Foundations, Six Sigma: Green Belt, and
Six Sigma: Black Belt. These courses provide a deeper dive into many of the concepts, tools, and
techniques we have discussed. For a closer look into graphs, charts, and data analysis for
operational excellence, I recommend my introductory course called Learning
Minitab. Remember, operational excellence is thought, and should not be thought of, nor should
it ever be perceived as a one time initiative or project. The pursuit of operational excellence
never ends.
Give fee
Statistics Foundations: 1
Welcome
- In our modern society we've all become addicted to numbers, even those that say they
hate math. What's their first move every morning? They grab their phone. Check the
time and temperature, numbers. Numbers tell us how fast to move, how warm to
dress. They help us decide where to invest our money. Numbers can motivate us to
act, and that's just when the numbers are given to us. If you have the power to organize
large pools of data, you have the ability to discover trends, prove yourself right, or
maybe prove others wrong.
This is the power of statistics. Whether you're a manager or a designer, whether you're
in business, science, sports or education, whether you're trying to save time, money or
any other valuable resource, understanding statistics is vital if you wanna be more
effective and efficient.
Hi there, my name is Eddie Davila and I'm a university instructor with degrees in
business and engineering. I write ebooks and of course I develop other online
educational content. I'm a huge sports fan, I love to follow the entertainment
industry and I'm passionate about science and health, and I can tell you that in every
important facet of my life having a better understanding of statistics allows me to
improve my performance and often to find a greater level of satisfaction whether I'm
working or playing.
This course, Statistics Fundamentals, is the first of a three part series that I'm hoping will
empower you to better understand the numbers you will encounter in your life. In this
course we'll discuss basic terms like mean, median and standard deviation. We'll look at
many different forms of probability. We'll explore the power of the bell shaped normal
distribution curve. We'll discuss issues like false positives, and expected monetary value,
and I'll tell you even if you know what all these things are, I think you'll walk away with a
new prospective. Actually I'm hoping you'll never look at these basic concepts the same
way again. You won't just understand what these numbers are and how they're
calculated, you'll know their inherent weakness too. So, welcome to Statistics
Fundamentals.
Well, let's say we have data relating to a number of adults. In a table, we can display all
the weights of these adults in pounds from heaviest to lightest. That's sort of
interesting. We can also create a table that reports the frequency of each weight. In
other words, how many adults weighed 170 pounds? How many weighed 130
pounds? Charts are interesting, but how about if we create a dot plot?
This is very similar to our table that reported the frequency of each weight. But, for
some reason this is just more appealing and easier to consume for our eyes and
brains. Still, there are so many possible values that both the chart and table can still be a
bit overwhelming. How about if we grouped this data by creating 10-pound
intervals? Here's what our table would look like.
But we could now also use a bar graph called a histogram. As you can see, it's pretty
much the same thing as our table, but again, the histogram is a bit more appealing. One
more time, back to our interval table.
Let's add another column. Based on our 50 observations, we can now also report in a
third column what was the relative frequency in each weight interval? As you can see,
according to our chart, 10 adults weighed between 140 and 149 pounds. 10 adults of 50
adults would be 20% of all the people measured.
And if you'd like, we can turn this into a relative frequency histogram.
Weight intervals define each separate bar. The height of the bar indicates the relative
frequency. How about if we aren't concerned with weights?
Actually, how about if we aren't measuring based on numbers? For example, let's
consider the color of each person's hair. Here's the data in table form.
Notice, the categories here are qualitative, different colors of hair, they are not
quantitative like the weight ranges we saw before. Here's a histogram for that hair color
data. Here's one with relative frequencies, but we can also use pie charts to display this
data. A circle or the whole pie represents all possible data. Each slice gives us an
indication of just how many adults are in each category. So if I provide the data in this
manner, an astute audience might be able to see both the frequency and the relative
frequency of each hair color for the population measured. These tables and charts are
merely the tip of the iceberg. While these are some of the most common tables and
charts, they are by no means the only available options. In the last decade, a group of
talented and creative statisticians have started to use their skills to create incredible
infographics that are interesting, colorful and sometimes even funny.
So, next time you look at a chart or table, don't get intimidated, don't look for the right
answer, read it like a story. Think about what it means to you and embrace questions as
an opportunity to open a discussion about the quality of the data and how the data
might help you make good decisions.
This is the sum of all the data points divided by the total number of
observations. Looking at data set one, our test scores, we can add our 25 test scores and
then divide by 25. Our average is 65%. That doesn't look like a very good average, does
it? The students in the class might complain that the exam was too difficult. That would
be one way to look at those results. Another common way to find the center of the data
though is by finding the midpoint. We call this the median.
For this, we organize our 25 exam scores from top to bottom. With 25 values, the 13th
value is our midpoint or median. Why? The 13th value has 12 values above and 12
values below it. When we look at our exam data this way, we find that the median
student's score is a 76. This would seem to indicate that the exam might have been
quite fair for those that studied. The problem might have been for those that did not
study. They were doomed to get horrible test scores and when we look at the lowest
test scores, we can see that just a few students really brought down the course average.
Remember though, there could be so many different explanations for these
outcomes. Perhaps some students were not qualified to take this course. Maybe a
majority of the class had heard the exams were going to be extremely difficult and hired
tutors. Perhaps the lowest scores were earned by students that could not afford
tutors. How about if many of the lowest performing students were exchange students
that struggled with language and reading?
As you can see, the mean and median help us identify two different types of
centers. When we investigate the exam scores, the median and average scores each tell
us a different story. Neither is complete, but together they have helped write the
beginning of our story.
Governments, businesses, and the media love to provide people with means and
medians. Often they are intended to tell you a story. Next time you are given a mean or
a median, don't look at those numbers as the end of a story. Look at them as the
beginning of an adventure or mystery. Start to figure out how they might provide
clues. Start to imagine how the individual data points in your data set might look. Come
up with questions you'd like to ask. You never know, it's possible that their mean and
medians aren't telling a true story. Perhaps they are just part of someone's fantasy.
For example, let's say we have 10 data points. The fifth data point has four data points
below it and five above it. The sixth data point has five data points below it and four
above it. Neither is the true median. What to do? Simple. In this case, we just take the
average of the two middle points. Here, we can see that the fifth data point is 20 and
the sixth data point is 30. So, our median point for this data set is 25. So, now you're
ready to find the median of any data set.
Weighted mean
- A mean, or an average, is good when all data points are created equal. But sometimes,
some data points are more important than others. Let's consider an academic course as
our example.
If the professor had four exams, and these were your four exam scores, calculating your
average would be easy, 80%. But in many courses, instructors will also have
quizzes, homework, and term papers. In those cases, exams are usually worth more than
quizzes, and homework, and term papers, may also have their own values. How do we
figure out a student's average in a class like this?
For this, we will use a weighted mean. Let's figure out how to calculate a weighted
mean, and then discuss issues that should be considered in calculating and interpreting
a weighted mean. So, let's look at how weighted means are calculated. First, let's
consider the weights of each category. Let's say our class has two exams, each worth
30% of the total grade. The quizzes are worth 10% of the grade, as is the
homework. The term paper is worth 20% of the grade. Notice, the weights add up to
100%. Next, let's look at each student score in each category. 90% on exam one, 80% on
exam two, the student's average on their quizzes was 75%, they got 100% on all of their
homework, and they got an 85% on their term paper.
Now all we do, is multiply the score times the weight. 90% on exam one, with a weight
of 30%, 0.30, that gives us a weight score of 27.0. We then do this for each
category, multiply the score in each category by the weight for that category. Now that
we have the weighted score for each category, we can add up all of the weighted
scores, to get the weight mean. This student, has an average of 85.5% for the course.
This doesn't just work for students in a class, we could use weighted means to rate
employees, or suppliers, we could use it to pick a home, or a school.
And while we're presently looking at data that has already been collected, we can use
weighted means to signal to others what we value. When a professor tells you that each
exam is worth 30% of the grade, students get signals about the potential difficulty of
exams, and also, how students should best use their time. Which of course means, if you
are going to use a weighted average, the categories and the weights of each
category, are very important to consider.
In general, there are no rules about which categories are chosen, there are certainly no
rules about how weights for each category are determined. So, it's important, that when
someone provides you with a weighted mean, that you question how the categories and
weights were chosen. Equally as important, if you are the person in charge of
developing a weighted mean, you need to carefully consider your choices in category
and weight, it is likely your audience will ask you very specific questions about your
calculations.
Finally, if you plan to use your weighted mean to motivate people, whether they be
employees, students or suppliers, consider the messages they will receive when they see
your rubric. If someone gave you this rubric,
how would you react? How about if they gave you this one? As you can see, the
weighted mean is simple, it's flexible, so, it's a very popular tool, but, it is also fairly
arbitrary. If you are using a weighted mean, be sure to take the time and effort to
maximize the value of the data it will generate.
median 60, mean 70, mode 80. Here's what this might look like. As you can see, mode,
median, mean, none of these on their own provide us a complete picture, but when you
use mode, median, mean in a chart together you get a nice glimpse of your complete
data set, and perhaps now you can start to ask the deeper question about the data
set and how it might help you make better decisions.
The range
- Let's take a look at these two small data sets.
Notice both have the same average, both have the same median values. Still, it's
obvious that the data sets are vastly different. These data sets are small so we can
quickly view all of the data and see the differences. What happens when the data sets
are enormous? How can we measure the differences in data sets that might have very
similar medians and means? Better yet, how can we get a better idea of what kind of
data makes up this data set? When we measured mean and median, we were looking for
the middle. Let's now measure how far out from the means and averages the farthest
data points lie. The simplest measure of variability is the range. Finding the range is
easy. You just take the largest number in the data set and the smallest number in the
data set. The difference between these two numbers is the range.
When you look at this data set, our range is 50.
Suppose these are exam scores. It's possible one student didn't study at all and got a
15%, 45% less than the average, and the highest grade might've been 85%. Again, I
would recommend using a histogram to help you better understand the makeup of your
mean, median, and range.
Here, we can see that one student really opened up the range. Perhaps the range isn't as
helpful as we thought.
Then again, if the data set looks like this, mean 60, median 58, range 10, we know that
this data would seem to be fairly centralized. Not only are the mean and median
similar, the difference between the biggest and smallest values is only 10 units. Range is
a nice, simple tool for our statistics toolbox, but we need to remember that it's not
always indicative of the overall data set. It only takes one rogue data point to
exaggerate the size of your data set's range.
It's a fairly common term in the world of statistics. But still, the concept can be
intimidating to many. So what exactly is the standard deviation? Well it's sort of the
average distance from the mean. Let's look at these three data sets.
All are very different. Still, all three have an average of 18. The first data set's range
indicates that this is clearly a data set of similar numbers, but data sets two and three
have identical means and ranges. Here, standard deviation is useful. While both have
large and small data points, the standard deviation of data set three tells us the
numbers in data set three are more similar to each other than those in data set
two. Notice we keep seeing a similar pattern. Often it takes a collection of basic statistics
tools to get a sense of what a data set contains. So how do we calculate a standard
deviation for a data set? Remember when I said it was sort of an average distance from
the mean? Well I meant it. It's sort of the average distance from the mean, but not
quite. It's the average square distance from the mean. Here's the formula.
Yeah kind of ugly, but don't worry, most calculators and spreadsheets are capable of
doing all the work for you. Plug in the values of the data set and the machine does all
the work. So finding a standard deviation doesn't need to be so difficult. Still for the
curious, let's go ahead and show you how to use this formula. This can get ugly so let's
start out with a small data set.
So for the first data point, take two minus eight. Negative six squared is 36. The sigma
tells us we need to do this for every data point and then add all of those values.
Next, we divide by three. This value is our variance for the data set.
the weights of 10 men that visited a doctor's office today. The mean weight is 189
pounds. The standard deviation is about 90 pounds. One standard deviation from the
mean would be 90 pounds lower than 189 all the way up to 90 pounds heavier than 189
pounds. Roughly from 99 pounds to 279 pounds. When we look at our individual data
points, we can see that the first nine data points are within one standard deviation from
the mean. How about that last data point? Let's try 1.5 standard deviations. So again,
about 135 pounds in either direction from the mean. 54 to 324. Nope, 425 is not within
1.5 standard deviations from the mean. Here's two standard deviations. How about 2.5
standard deviations? No, 425 is still not within that range. And yes, we now have gotten
below zero on the low end, about negative 36 pounds. Only when we get to three
standard deviations does the last data point fall within our limits. I guess the next
question is, what does this mean? Is any of this significant? Well, for data that would be
considered symmetrical, which means that we have a nice bell-like distribution centered
at the mean, it is estimated that about 68% of your data points should fall within one
standard deviation of the mean. We had 90% fall within one standard deviation. Better
than expected. Most of our patients are probably somewhat similar. With our small data
set, we can see that this is true. If we had a huge data set and 90% of our data was
within one standard deviation from the mean, we'd probably feel pretty good. How
about that last data point? Well, as I said, 68% of the data points within one standard
deviation is considered normal. How about for two standard deviations? Well, we would
expect about 95% of our data points to be within two standard deviations of the
mean. And for three standard deviations of the mean, 99.7%. And that is where our last
data point lies. That would seem to be fairly extreme. So, that last data point is
definitely what we would consider an outlier. As you can see, standard deviation can be
a very helpful tool in understanding data sets and their individual data points. More
importantly, standard deviation will help us generate interesting questions about the
data collection methods, the entire pool of data, and even the individual data points.
Outliers
- Nowadays we hear the term outlier quite a bit. Perhaps a certain athlete is incredibly
talented and productive, much more so than any other competitor in the league. Their
statistics far exceed those of any other single player. They might be labeled an
outlier. Maybe one 10 year old in your city is taking a course in calculus. Or perhaps the
opposite occurs, another 10 year old child struggles with basic addition. In an effort to
center the discussion on the masses, educators may exclude these children as
outliers. But how about a child that scores a perfect score on a nationally standardized
test for 10 year olds? Are they an outlier? Or how about our athlete? If they average 45
points per game in basketball, at what point is our star athlete an outlier? If no one else
is above 40 points per game, is our star an outlier? If no one else is above 35 points per
game? How about if our star player averages 45, the second best player scores 40 points
per game, and the next best player is at 32 points per game? Do we have two outliers or
do we not have any outliers? So what exactly is an outlier? The most common answer
you'll get is that an outlier is a data point that is an abnormal distance from the other
values in the data set. This brings about a few questions. First, what's abnormal? There is
no set definition, but I think it's important to understand that the term outlier is not a
very specific term. So, it's less about absolutely identifying outliers, rather it's about
motivating discussions of what is normal, about what is possible. Perhaps talking about
outliers will help important issues surface. So, how can we identify outliers? Tables and
charts can be useful. Sometimes they make outliers stand out. Perhaps we use standard
deviation. Maybe we say that anything more than two standard deviations from the
mean is a statistical outlier. Sometimes an outlier is just something new, something that
we've never even considered, which brings us to the next question. What should we do
with outliers? Should we just throw them out, not consider them at all? In general, I'd
say no. Most consider outliers as freaks or freakish events that are not likely to be seen
again, as a result they're ignored, not considered worth investigating since they're so
odd. Instead though, they should be considered as opportunities. Are they the
beginning of a new trend? Does this person know something that we don't? Is it
possible others will learn from this outlier, and we might see a massive change in
behavior? Was there a special circumstance for that particular person? Why did they do
so poorly? Perhaps this person got very ill and had to leave the program. Or why did
they perform so well? Did they get extra help? Did they have additional training? Should
we consider extra training for everyone? As you encounter outliers in your data, at the
office, or even in your daily life, ask good questions. Is this really an outlier? How did this
happen? What can we learn? What needs to change? A mass of closely distributed data
points can be very instructive, but sometimes the lone outlier can provide us with a
brand new perspective.
For this reason, it is also sometimes referred to as three sigma rule, where sigma stands
for standard deviation. The rule goes further though. It explains that about 68% of all
the data points will lie within one standard deviation of the mean.
Notice how this is illustrated on our bell-shaped curve. The empirical rule then goes on
to say that 95% of all data points fall within two standard deviations of the mean. Again,
notice how this is illustrated in our bell-shaped curve. Finally, the empirical rule tells
us that when you have the bell-shaped curve, often referred to as a normal
distribution, 99.7% of the data points in the data set will fall within three standard
deviations.
So, as you can see, now almost all of the area under that bell-shaped curve has been
accounted for. One very important note, this works when we have the well-
centered, symmetrical bell-shaped curve. The rule begins to lose value the farther our
data set strays from the classic normal distribution. That said, in most cases, the 68-95-
99.7 rule, which we call the empirical rule, holds up pretty well. With this knowledge, you
can, hopefully, better evaluate data points. If someone says that a data point has a Z-
score of 1.8, you know it's within two standard deviations of the mean, and thus it is
likely among 95% of all the data points in the data set. That said, if something had a Z-
score above 3.0, we could be pretty confident that this data point was a true
outlier, since it is likely not among 99.7% of all the data in our data set. In terms of our
bell-shaped curve, this data point would likely be out of this region. No matter if you call
it the empirical rule, the three sigma rule, or the 68-95-99.7 rule, we now better
understand the normal distribution of data in a way that allows us to better understand
our data set and the data points that lie within that data set.
Our student got an 85% on the exam, but that was the third lowest score in the
class. Conversely, we can look at this set of 10 exam scores. Here, our student also
scored an 85%, but now, they had the highest score in the class. Which then begs the
question, how do we calculate the student's percentile rank? To calculate percentile
rank, we use this formula.
As you can see, we need to count up the total number of values in our data set. For our
most recent example of 10 exam scores, we have 10 values. Next, we count up how
many values are below the score of 85%.
For this data set, our student had the third lowest score. Only two exam scores were
below this student's score. So this is how we set up our formula. According to this, our
student scored an 85% on the exam, but she is in the 25th percentile. Let's do the
calculation again for other data set. Here, our student had the highest grade in the
class. Nine other scores were below her score. In this data set, our student is in the 95th
percentile. Did you notice what just happened there? We had 10 exam scores, yet the
highest percentile that could be achieved was 95. Why? Well, a 100th percentile is not
possible, since that would be like saying that you are in the top zero percent of your
class. The best that one person could say is that they are in the top 1% or the 99th
percentile. Still, why not put the student in the 99th percentile? And if you use a
spreadsheet like Excel, they have their own internal formula for calculating
percentiles. In any case, the numbers are often close enough that it should not present a
significant problem, and often, as the size of the data set gets bigger, these differences
become negligible. So for the competitive folks out there, you now have two ways to
frame your goal. You can either tell people that you are shooting to be in the top 1% of
your organization or instead, you can tell them that one day, you hope to be in the
organization's 99th percentile.
Defining probabilty
- What are the chances I can flip a coin and get heads three times in a row? What's the
likelihood a certain basketball player will make a free throw? What are the odds that it
will rain in Berlin tomorrow? What are the chances a child born today will live to be 90
years old? Everyone is interested in probability. Science, sports, business, gambling, all
of them rely on probability in an effort to make informed decisions. But, what is
probability? I guess the most basic definition would be the likelihood that some event
will occur. Typically it's measured via a ratio. The desired outcome divided by all possible
outcomes.
So let's go back to my very first question. What are the chances I can flip a coin and get
heads three times in a row? So, we have a random experiment: tossing a coin. The
sample space, which is a list of all the possible outcomes is this.
As you can see, we have eight possible outcomes. Only one of those is the desired
outcome. So, you can see the probability of getting heads three times in a row is one in
eight, or 12.5%. Let's try one more. This time, we can roll a pair of dice. What are the
odds I will roll two sixes? Our random experiment, rolling a pair of dice. The sample
space is given here.
So this time we have 36 possible outcomes. Only one of those is the desired
outcome. Our probability of rolling double sixes is one in 36, or about 2.8%. Dice and
coins are easy though. They tend to be fair, and thus, fairly predictable. Most of life
doesn't work that way, though. It's not always easy quantify all the possible
outcomes. For example, what are the odds your boss will wear a black dress to work
tomorrow? It would depend on who your boss is, how many black dresses they
have, how many other outfits they might own. Perhaps, we also need to understand the
events of that work day. Some of the things we might know. Some we would not
know. Maybe only some of those things are important. Perhaps, there are other
factors we have not even considered. The formula for basic probability may be
simple, but that doesn't mean calculating probability is easy. Let's also consider this
scenario. Let's say, you and I bet.
I tell you, that so long as you do not roll a double six with two dice, you will win the
bet. You have a 97.2% chance of winning. Suppose you roll the dice, and you roll a
double six, does this mean the probability was wrong? No, it simply means you were
every unlucky. Probability does not guarantee an outcome. It simply tries to inform you
on the possibilities. Next time someone provides you with a probability consider how it
was calculated, whether or not you trust the probability, and what the number actually
means. Understanding probability, both its strengths and its weaknesses, definitely
increases your odds of making good decisions.
Examples of probability
Some probabilities are easy to understand and calculate. Even odds across the
board. What are the odds of flipping a coin and getting heads? One in two, 50%. What
are the odds of rolling a six sided die and getting a four? The odds are one in six. The
nice thing about these two examples is that each possible outcome is equally as
likely. By that I mean, the odds of getting heads is 50%, the odds of getting tails is 50%.
The odds of rolling any of these outcomes with the six sided die are equal. The odds of
getting any one of these outcomes is one in six. It's not quite as easy to calculate the
probability of a rainy day in Los Angeles tomorrow. It's not like we can say that the odds
of rain tomorrow is 50% and the odds of no rain are 50%.
In Los Angeles, typically there are only about 20 to 30 days per year when it rains. And in
London, there are typically over 100 days per year when it rains. In these cases we say
that the odds are weighted.
The odds that it might rain on any given day in Los Angeles is about 7%, and about 29%
in London. Then again, we also need to remember that these probabilities are stated on
the basis of an entire year. If I instead say, what are the odds that it will rain on
December 10th in London? The probability of rain on that particular day may be much
higher since December is traditionally London's wettest month. That said, the sum of the
probabilities of all possible outcomes must add up to 100%. So for our coin, 50% heads,
50% tails, 100% total. For our six sided die, each of our six outcomes has a probability of
one in six. When we add up all six outcomes our probability is 1.0 or 100%
So if we define each day as either a day with rain or a day without rain, in London the
probability of rain on any given day might be stated as 29%. So the probability of a dry
day must be 71%. That again, sums up to 100%. Let's consider a scenario where I
put two, red ping pong balls in a container. That's it, there is nothing else in the
container. What are the odds that if you take one ball out of the container that it will be
red? Obviously, since both balls in the container are red, the ball you take out of the
container must be red. So the probability is 100%. On the other hand, what are the odds
that the ball you take out of the container will be white? There are no white balls in the
container, so the odds are 0%. This outcome is not possible. These are both simple
scenarios, but they help us understand a few basic things. The highest probability for
any scenario is 100%. The lowest probability for any scenario is 0%. So, to recap, the
probability of all possible outcomes must sum to 100%. Sometimes the probability of
every possible outcome is equally as likely. Sometimes some outcomes are more likely
than others. But no matter what, the probability of an outcome can never be less than
0%, nor can it be greater than 100%.
Types of probability
- There's a 50% chance that the result of a coin flip will be heads. There's an 80%
chance that the best basketball player on your team will make a free throw. There's a
75% chance that the unemployment rate in the United States will drop next year. Not
only are these three probabilities about three very different events, these are also three
different categories of probabilities. The coin flip is an example of classical
probability. The free throw example is an example of empirical probability. Both of these
are objective probabilities, meaning they are based on calculations. The unemployment
example is an example of subjective probability. Here, there are no calculations. So
what's the difference between these three different types of probabilities, and when is
each appropriate? Let's begin with the coin flip, an example of classical
probability. Assuming this is a fair coin, we have two possible outcomes, heads and tails,
both equally as likely.
Let's say you win the coin flip if heads is the result. What's the probability of
victory? How do we calculate this probability? One outcome is a winner, and we divide it
by two, the total number of outcomes. Your chance of winning is 50%. We could do this
with dice also.
We have a six-sided die, six possible outcomes. You win if you roll a one or a two. Two
winning outcomes divided by six possible outcomes. Your probability of winning here is
33%. As you can see, classical probability works well when you know all possible
outcomes and all the possible outcomes are equally likely to occur. When things are fair
and equal, when we understand every possible outcome, classical probability works
well. But what happens when not everything is fair and equal? For this, we turn to
empirical probability.
We can get heads on flip one, tails flip two. Heads flip one, heads flip two. Tails flip one,
tails flip two. And finally, tails flip one, heads flip two. We can see here that two different
events will win this contest for you. Event one, heads of flip one, two out of the four
scenarios provide that result. Event two, heads on flip two, same here, two out of the
four scenarios provide that result, but we also see that there is an overlap here.
but we subtract the overlap, which is when both event one and event two occur. We can
see that our probability of getting heads on at least one of two coin flips is 75%. This is
what is called the addition rule. Let's up the difficulty level just a tiny bit. Let's do this
with a pair of six-sided dice. You win if you roll either a six with roll one or roll two. Here
are all the 36 possible outcomes.
There are six outcomes where we roll six on the first die. There are six outcomes where
we roll six on the second die, but one of those outcomes overlaps. So to calculate the
total probability we add 6/36ths plus 6/36ths and subtract the overlap, 1/36ths, thus the
probability is 11/36ths or 30.56%. Sometimes there are no overlapping scenarios, for
example, what are the odds that the sum of two rolled dice will sum to either seven or
11? Here are the scenarios where the dice add up to seven, here are the two scenarios
where the dice add up to 11, notice there is no overlap. So here there is no need to
subtract anything, just add the odds of rolling a seven, 6/36ths, and add the odds of
rolling 11, 2/36ths. Here the probability is 8/36ths or 22.2%. We can also flip this
around. What are the odds of not rolling a seven or 11? Again, here are the eight
scenarios where the sum of the two dice would be seven or 11. We can count all the
other scenarios, but since we know eight of 36 scenarios sum to seven or 11, we know
that 28 of 36 scenarios do not sum up to seven or 11. The probability is 77.8%. These
were all relatively simple scenarios, but hopefully as you go forward you'll remember
you can add probabilities and you can also subtract probabilities.
I'm gonna pick two cards at random and award each person whose names are on those two cards
$100. Let's say Jose and Sally are two of the six people on my list, what are the odds I will pick
both of their names? Well, here are the 15 possible outcomes,
only one of which contains both Jose and Sally. So, the probability that both will win together is
1/15 or 6.67%. Let's say I change the way we play the game. Let's say that instead of picking
both cards at the same time, I will pick one name and then pick the second name a few minutes
later. What are the odds of both Sally and Jose winning if the first card I pick has the name
Audrey? For this, we use the concept of conditional probability. This helps us answer questions
like this one. In this case, now that Audrey is one of the winners, it is impossible for both Sally
and Jose to win. The probability that both Sally and Jose are the two winners has dropped from
6.67% before we picked Audrey's name to zero after Audrey's name was picked. But, what about
if Sally's name was on the first card? What are the odds that both Sally and Jose win
now? Remember these were all of the possible outcomes. But now that Sally is out of the pool of
names, here are the five possible outcomes that remain.
As we can see, initially the odds that Sally and Jose both win were 6.67%. Once Sally's name is
chosen, the probability went up to 20% since Sally and Jose were one of five possible
outcomes. Often it helps to draw probability trees to visualize what's happening. Here's a
probability tree for three coin flips.
As you can see, the odds of getting tails three times in a row is initially 12.5%. We get that by
multiplying the probabilities along this branch.
Once we flip, the first tails, though, the odds of getting tails three times in a row increases to
25%, only two parts of that branch remain.
Here's a set of health-related data: 1,000 people, how long they lived, and whether or not they
exercised at least three days per week. Consider these two events: event A, people that lived
more than 85 years, event B, people that exercised at least 30 minutes three or more days per
week. How would we find the probability that someone lived more than 85 years given that they
exercised at least three days per week? Let's build the tree for this scenario.
The given event is that this person exercised three days per week. So this is like the first coin
flip. 240 out of 1,000 people exercised three days per week. 760 of 1,000 people did not
exercise at least three days per week. Now comes the second event, how long did they live? 40 of
the 240 exercisers lived less than 75 years, 16.6%. 70 of the 240 exercisers lived 75 to 85 years,
29.2%. And 130 of the 240 exercisers lived more than 85 years, 54.2%. Here's what the other
branch on that tree would look like. So, from all the data we can see that only 13%, 130 of the
1,000 people lived more than 85 years and exercised at least three days per week. But once we
are given the fact that the person in question worked out at least three days per week, the
probability that this person lived more than 85 years is 54.2%. According to this data, it looks
like exercising three days per week might have its advantages. I guess we can say that working
out is an enormous factor in living past 85. Not so fast. Don't go overboard. People who are
committed to working out three days a week likely have other good habits. So, the exercise alone
may not be the only contributing factor. Understanding how to calculate conditional
probabilities is very important in the world of statistics. But understanding what those numbers
might mean and knowing which questions to ask, that might even be more important.
Welcome
- As a person that loves statistics, or perhaps as someone who just appreciates
statistics, you're probably comfortable with the basics: means, medians, standard
deviations, probabilities, and normal distributions. They're all part of your stats
vocabulary. But perhaps for you, stats appreciation is not enough. You want to collect
your own data. You want to make reasonable predictions. You'd like to test statistical
assumptions. You've come to the right place because that is what this course is all
about. My name is Eddie Davila, and I'm a university instructor with degrees in business
and engineering. I write ebooks, and of course I develop online educational content. I'm
a huge sports fan. I love to follow the entertainment industry. And I'm passionate about
science and health. And I can tell you that in every important facet of my life, having a
better understanding of statistics allows me to improve my performance and often to
find a greater level of satisfaction whether I'm working or playing. This course, Statistics
Fundamentals Part Two, is the second of a three-part series that I'm hoping will
empower you better to understand the numbers you will encounter in your life. In this
course, you'll discuss the collection of data and the importance of the simple random
sample. You'll look at confidence intervals. We'll explore what margins of error
mean. We'll discover the importance of hypothesis testing in the fields of science,
business, and beyond. And I'll tell you, even if you know what many of these things are, I
think you'll walk away with a new perspective. Actually, I'm hoping you'll never look at
these concepts the same way again. You won't just understand the power of data and
statistics. You'll know their inherent weaknesses too. Welcome to Statistics
Fundamentals Part Two. Improved performance and increased satisfaction are just
around the corner.