3.1.

1 Module 3 Roadmap

From Deciding to Doing

Module 2 gave you a portfolio of decision-making techniques. But it’s just as important to
know how to implement decisions as how to make them. And that’s a whole different
process.

Q: How do you define “implementation?” 

VP: The implementation is the implementation of the decision taken.

Hendrina: I'd define implementation as converting a certain plan or decision into action.

As we saw in Module 1, implementation is the process by which managers and

organizations translate intentions into actual results. As a manager, it’s your job to get
the job done. While you may not always have a say in what choices your organization
makes, you have to know how to implement them. That’s the heart of good
management.

In this module, we’ll stick with our process lens. We’ll study cases of spectacularly
effective and ineffective implementation. We’ll conclude next week by examining the
close relationship between implementation and learning.

Implementation is perhaps the most important process for successful management. As a

result, this module will be slightly longer than the other three. We hope you’ll agree it’s
worth the effort.

Learning Objectives
By the end of this module, you should be able to:

 Provide a grounded, granular definition of effective implementation

 Detect and diagnose common causes of poor implementation 
 Identify the seven stages of the implementation process and the manager’s role
in shaping them
 Recognize that implementation is focused on the present and that long-term
success requires a focus on the future as well (and therefore, attention to
learning and improvement)
 Define learning as a three-step process of acquiring, interpreting, and applying
information
 Realize the important roles of learning from past experience, learning from
others, and experimentation in an organization’s ability to improve and innovate
  Use a range of managerial levers to improve organizational learning

3.1.2 Ineffective Implementation

An Obsession with Big Ideas

Often, companies allow themselves to get distracted by big strategic goals and
disruptive innovation. The hard truth is that the success of every organization rests on its
capacity to implement its decisions and execute its key processes efficiently, effectively,
and consistently. 

Here’s Kevin Sharer on the importance of implementation in management:

Video

Q: Describe an experience you’ve had that involved ineffective implementation.  This

could involve a project that failed to deliver at all, or a project that delivered late, or a
project that failed to meet its original design. This is a scratchpad Reflection and will not
be shared with your classmates.

VP: As a project selection consultant for financing, I had a case in which in the almost
final phase of obtaining approval for an interim loan, the client did not present a
breakdown of its debt at book value due to inefficiency and most likely reluctance on the
part of management.

Ineffective implementation is pretty easy to recognize. Simply examine a project that’s

significantly over-budget, a new product that’s delayed, a team in the midst of a mean-
spirited fight, or a service that’s low-quality and unreliable. In most cases, poor
management of implementation is the cause. Effective implementation can be harder to
spot because there are no problems. 

3.1.3 Effective Implementation

A Definition of Effective Implementation

Here’s my own take on effective implementation:

Video
Q: Consider the definition of variability. This is one that people don’t immediately grasp.
How does variability undermine implementation and are there particular circumstances
in which minimizing variability is especially important?

VP: Variability as part of the process should be foreseen and avoided if possible, because it
leads to non-compliance with deadlines, budget and sometimes - to failure to achieve the end
result to the desired extent.

Anissa: Variability can bring errors and brings unexpected events or unexpected results
that can slow down the process and the journey towards success. 
In scientific research for example, minimizing the variability in an experimental protocol
is extremely important to reduce errors and get to the expected result. If a step is
changed, then it can change the entire process.
Hendrina: Variability undermines implementation because it forms a dependence on
uncertain factors. Particular circumstances in which minimizing variability is important: I
feel it's always important to minimize variability. Couldn't think of any particular ones
(next to new products f.e.) to be honest.

I like these criteria because they eliminate many common excuses for unsatisfactory
performance. Such gems include:

 “The plan was too ambitious.” 

 “It was the right idea for the wrong time.” 
 “The economy took a bad turn.”
 “The marketing department really blew it.”
 “There’s no way we could have predicted that the supplier would fold.” 

In a word: “It wasn’t my fault!”

VP: As a project selection consultant for financing, I had a case in which in the almost final phase
of obtaining approval for an interim loan, the client did not present a breakdown of its debt at
book value due to inefficiency and most likely reluctance on the part of management.

SUBMITTED ON APRIL 17, 2022 04:59 AM

Q: In your ineffective implementation experience, which of these four goals did it fail to meet?
Delivering what is planned or promised
Delivering on time, on budget and at quality
Delivering with a minimum of variability
Delivering in the face of unexpected events

Results
 38.7 percent selected Planned or promised
 30.7 percent selected Time, budget, and quality
 17.599999999999998 percent selected Unexpected events
 13.00000000000001 percent selected Minimum variability
-3-2-1123-1138.7%Planned or promised30.7%Time, budget, and quality17.6%Unexpected
events13.0%Minimum variability
323 of 554 participants have responded. You may return at any time for the latest
results.

Anticipation + Resilience
Don’t get the wrong idea. My definition of effective implementation does not include or
imply managerial omniscience or clairvoyance. What it does require, as we saw in
the Columbia case, is a blend of anticipation and resilience—the ability to sniff out
potential problems while they are still correctable and the ability to bounce back should
problems occur anyway. These skills, in turn, require a firm understanding of
implementation as a process that needs to be managed well.

Here’s Prof. Len Schlesinger on the importance of taking a process perspective on

implementation:

Video

As with the decision-making process, mastering the implementation process requires

disabusing yourself of a passel of myths.

3.1.4 Myths About Getting the Job

Done

Implementation: Myth #1
Let’s look once more at a list of myths, this time about implementation.

Implementation works best with broad, open-ended goals and deliverables.

Ineffective or naive managers tend to think of implementation as “tactical”—something

that they can delegate while focusing on “more important” issues, like crafting high-level
strategies. Best to set loose goals and let your employees figure out the details, right?
Other managers are too wary of micro-managing and feel they should empower
employees by leaving them alone. 
Either way, managers often fail to communicate to others what tasks need to get done
and why they are essential to company strategy. This leaves employees either (a)
missing the big picture and going off in the wrong direction or (b) missing essential
details necessary for the plan to succeed. In fact, these problems are two sides of the
same coin.

In the first case, managers fail to provide their team with information on context (industry
and competitive backdrop, department politics, functional loyalties, key stakeholders,
nonfinancial concerns, etc.). The idea is that too much strategic or contextual information
will distract employees from their primary focus—execution.  

In the second case, when employees lack understanding of granular details, confusion
and delay will arise because people will not understand what they are actually supposed
to do. The objectives of the process are not communicated or vaguely defined. As Henry
Mintzberg has argued, strategies must be “programmed" so that lofty goals and
objectives (e.g., “improve customer satisfaction”) are translated into concrete activities
and deliverables (e.g., “raise net promoter scores 20% on product XYZ by increasing the
speed of email responses to customer complaints”). 

In short, implementation works best with crisp goals and deliverables which can be
measured. Progress and milestones need to be measured so that teams can adjust what
they are doing, as will almost always be necessary to get the job done. 

3.1.4 Митове за свършената работа

Изпълнение: Мит №1

Нека разгледаме още веднъж списък с митове, този път за прилагането.

Внедряването работи най-добре с широки, отворени цели и резултати.

Неефективните или наивни мениджъри са склонни да мислят за внедряването

като за „тактическо“ – нещо, което могат да делегират, докато се фокусират върху
„по-важни“ въпроси, като изработването на стратегии на високо ниво. Най-добре е
да си поставите свободни цели и да оставите служителите си да разберат
детайлите, нали? Други мениджъри са твърде предпазливи по отношение на
микроуправлението и смятат, че трябва да овластяват служителите, като ги
оставят на мира.

Така или иначе мениджърите често не успяват да съобщят на другите какви

задачи трябва да бъдат свършени и защо са от съществено значение за
стратегията на компанията. Това оставя служителите или (а) да пропускат общата
картина и да вървят в грешна посока или (б) да липсват съществени детайли,
необходими за успеха на плана. Всъщност тези проблеми са двете страни на една
и съща монета.

В първия случай мениджърите не успяват да предоставят на екипа си информация

за контекста (индустрия и конкурентен фон, политика на отделите, функционална
лоялност, ключови заинтересовани страни, нефинансови проблеми и т.н.). Идеята
е, че твърде много стратегическа или контекстуална информация ще отклони
служителите от техния основен фокус - изпълнението.

Във втория случай, когато служителите нямат разбиране за детайлни детайли, ще

възникнат объркване и забавяне, защото хората няма да разберат какво всъщност
трябва да правят. Целите на процеса не са съобщени или неясно дефинирани.
Както Хенри Минцбърг твърди, стратегиите трябва да бъдат „програмирани“ така,
че високите цели и задачи (например „подобряване на удовлетвореността на
клиентите“) да бъдат превърнати в конкретни дейности и резултати (напр. скорост
на отговорите по имейл на оплаквания на клиенти“).

Накратко, изпълнението работи най-добре с ясни цели и резултати, които могат да

бъдат измерени. Напредъкът и основните етапи трябва да бъдат измерени, за да
могат екипите да коригират това, което правят, тъй като почти винаги ще е
необходимо, за да се свърши работата.
Implementation Myth #2
Once responsibility for a deliverable has been assigned, the manager’s job is
largely done. 

An effective manager transfers responsibility for an activity—often described as

delegation—while retaining accountability for the ultimate outcome. They must therefore
take steps to make sure the subordinates are successful. These steps include deciding
what to delegate and to whom, establishing a timeline, providing essential support,
checking in, providing oversight and feedback, and more. The manager’s job is not done
until the job is done.

Мит за внедряване № 2

След като отговорността за даден резултат е възложена, работата на мениджъра

до голяма степен е свършена.

Ефективният мениджър прехвърля отговорността за дейност – често описвана

като делегиране – като същевременно запазва отговорността за крайния резултат.
Следователно те трябва да предприемат стъпки, за да се уверят, че подчинените
са успешни. Тези стъпки включват решение какво да делегирате и на кого,
установяване на график, предоставяне на основна подкрепа, проверка,
осигуряване на надзор и обратна връзка и др. Работата на мениджъра не е
свършена, докато работата не бъде свършена.

Implementation Myth #3
Managers should only intervene in the implementation if something goes wrong.

Plans are bound to go off track, if only because of changes to the context. Managers
should certainly intervene under such circumstances. Returning to earlier steps of the
implementation process or even revising plans altogether is to be expected.

More importantly, managers should never be afraid to get involved before things go
south. As implementation proceeds, managers should be monitoring progress, keeping
an eye out for warning signals, providing helpful feedback, removing roadblocks, and
otherwise proactively helping the process to proceed smoothly. Good managers identify
a number of leading indicators of progress. When they detect that the process is not
proceeding as expected, they treat that as a red flag that things may be off track. With
regular monitoring and pre-established check-ins, a manager may be in the dark for a
few days, but not—as is often the case—for weeks or months. 

Мит за прилагане №3

Мениджърите трябва да се намесват в изпълнението само ако нещо се обърка.

Плановете непременно ще излязат от релси, макар и само поради промени в

контекста. Мениджърите със сигурност трябва да се намесят при такива
обстоятелства. Може да се очаква връщане към по-ранни стъпки от процеса на
изпълнение или дори цялостно преразглеждане на плановете.

По-важното е, че мениджърите никога не трябва да се страхуват да се включат,

преди нещата да тръгнат на юг. С напредването на изпълнението мениджърите
трябва да наблюдават напредъка, да следят за предупредителни сигнали, да
предоставят полезна обратна връзка, да премахват пречките и по друг начин
проактивно да помагат на процеса да протича гладко. Добрите мениджъри
идентифицират редица водещи индикатори за напредък. Когато открият, че
процесът не протича според очакванията, те третират това като червен флаг, че
нещата може да не са на път. С редовен мониторинг и предварително установени
проверки, мениджърът може да бъде на тъмно за няколко дни, но не – както често
се случва – в продължение на седмици или месеци.

Implementation Myth #4
Projects conclude with a defined end and an acknowledgement by all involved
that they are completed. 

Sadly, this all too rarely happens. As projects wrap up, many managers and their teams
understandably run out of gas. People want the project to be done so they can move on.
They often fail to agree on a shared assessment of whether they actually got the job
done—as defined when it was begun. Few managers hold post-game reviews to
compare what has been delivered to what was promised at the start. Next steps are
rarely discussed. 

Мит за внедряване № 4

Проектите завършват с определен край и потвърждение от всички участници, че са

завършени.

За съжаление, това се случва твърде рядко. С приключването на проектите много

мениджъри и техните екипи разбираемо остават без бензин. Хората искат
проектът да бъде изпълнен, за да могат да продължат напред. Те често не успяват
да се споразумеят за споделена оценка дали наистина са свършили работата –
както е определено, когато е започнала. Малко мениджъри правят прегледи след
игра, за да сравнят това, което е доставено с обещаното в началото. Следващите
стъпки рядко се обсъждат.

Let's return to your example of ineffective implementation.

As a project selection consultant for financing, I had a case in which in the almost final phase of obtaining approval for an
interim loan, the client did not present a breakdown of its debt at book value due to inefficiency and most likely reluctance
on the part of management.
SUBMITTED ON APRIL 17, 2022 04:59 AM
Q: Do you think its failure was rooted in any of these four myths?  This is a scratchpad
reflection and will not be shared with your classmates.

VP: It was rooted to Myths 1. - Implementation works best with broad, open-ended
goals and deliverables.

While we’ve identified some principles as to what contributes to effective

implementation, we have not clearly defined a process for implementation. What
concrete steps are involved in what order? We will now introduce a model of
implementation that addresses these question. Its main goals are to describe (a) how to
get work done today and (b) how to create the conditions to do better work tomorrow.

Once we’ve done that, we’ll present you with two highly-publicized cases: one
miraculously effective implementation, and one catastrophically ineffective
implementation.

3.1.5 The Implementation Process

Implementation: An Art, Practice, and

Process
Why are there so few systematic models of implementation? Here’s Prof. Schlesinger to
describe why he believes implementation is more an art than a science:

Video

VP: For me it is both - as they are motivation and goals in the same time.

Faith: Implementation is more of an art than a science. This is because implementation

sometimes requires creativity and to constantly be checking-in and resetting things. It
does not just get set and go.
Hendrina: More of art - it's not a mere bunch of logic, it's like a pile of tasks (or
problems); organization and work processes need to be tapped into to analyze,
organize, and prioritize, so the whole crew can move smoothly from a certain step to the
next one.

A Basic Model of Implementation

If you were to ask a manager, “How did you get your last project done?” you’d be likely
to get a range of unhelpful responses. “What do you mean ‘get it done?’” she might say.
So many conversations, emails, meetings, and presentations went into her last project
that she can’t give you a succinct answer. Managers usually feel that they have
improved over time, but lack a comprehensive, orderly framework with which to improve
the odds of success. 

Such a model would certainly be helpful. What should go in it?

Here’s Kevin Sharer’s answer:

Video

All effective implementation processes share common attributes which I have captured
in a seven step process for implementation.

Picture

Access image details

Flowchart describing the 7 stages of the implementation process. Stage 1 is to set goals
and define deliverables. Stage 2 is to determine roles and responsibilities, and to clarify
relationships. Stage 3 is to delegate the work. Stage 4 is to execute the plan, monitor
progress and performance and provide support. Stage 5 is to take corrective action by
making minor adjustments and major revisions. Stage 6 is to get closure on the project
and get agreement on the output. Stage 7 is to conduct a retrospective on the
implementation process. At different points arrows circle back to indicate how a stage
might be repeated. At Stage 5, the corrective action might be to return to Stage 1 to re-
set goals or re-define deliverables. Also at Stage 5, the corrective action might be to
return to Stage 4 to re-execute the plan, more closely monitor progress and
performance, or provide more support. At Stage 6, trying to get agreement on output
might prompt returning to Stage 4 to re-execute the plan, more closely monitor progress
and performance, or provide more support.

1. Set goals  and define deliverables.

2. Determine roles, responsibilities, and relationships.
3. Delegate the work.
4. Execute the plan, monitor progress and performance, and provide continued
support.
5. Take corrective action (either minor adjustments or major revisions).
6. Get closure on the project and agreement on the output.
7. Conduct a retrospective or review of how the process went.

Two caveats before we dive into each of these stages. First, note that this process
assumes that managers or relevant leaders have already made a number of strategic
decisions: what markets the company will play in, how it plans to gain a competitive
advantage in those markets, what projects to pursue to carry out its strategy, and so on.
The implementation process we've outlined thus describes how a manager should go
about executing an already established plan, but does not involve assessing whether
that plan is worth pursuing. 

Second, these stages do not always unfold as neatly or rigidly as our list suggests. You’ll
almost always have to jump around or circle back in response to new information. In
fact, if you make it through the process without revisiting earlier stages, you probably
didn’t implement very well. Why not?
Because implementation is an iterative process, not a linear one. The deliverable
defined in stage one might need to be redefined in stage five. A responsibility assigned
in stage two may need to expand to meet unforeseen demands discovered in stage
four. 

In sum, think of this implementation process not as a formula but as a set of

guidelines: a checklist of managerial processes—like goal-setting, delegating,
monitoring, and reviewing—each of which you must learn to shape.

Поставете цели и дефинирайте резултатите.

Определете роли, отговорности и взаимоотношения.
Делегирайте работата.
Изпълнете плана, наблюдавайте напредъка и представянето и осигурявайте
непрекъсната поддръжка.
Вземете коригиращи действия (или малки корекции или големи ревизии).
Получете приключване на проекта и споразумение за изхода.
Направете ретроспекция или преглед на това как е протекъл процесът.
Две предупреждения, преди да се потопим във всеки от тези етапи. Първо, имайте
предвид, че този процес предполага, че мениджърите или съответните лидери вече са
взели редица стратегически решения: на какви пазари ще играе компанията, как
планира да спечели конкурентно предимство на тези пазари, какви проекти да
преследва, за да изпълни своята стратегия , и така нататък. По този начин процесът на
изпълнение, който очертахме, описва как мениджърът трябва да се заеме с
изпълнението на вече установен план, но не включва оценка дали този план си струва
да се следва.

Второ, тези етапи не винаги се развиват толкова спретнато или строго, както предлага
нашият списък. Почти винаги ще трябва да подскачате или да се връщате назад в
отговор на нова информация. Всъщност, ако преминете през процеса, без да
преглеждате по-ранните етапи, вероятно не сте приложили много добре. Защо не?

Тъй като изпълнението е итеративен процес, а не линеен. Резултатът, дефиниран в етап

първи, може да се наложи да бъде предефиниран на етап пет. Отговорността,
възложена на етап втори, може да се наложи да се разшири, за да отговори на
непредвидени изисквания, открити в четвърти етап.

Накратко, мислете за този процес на изпълнение не като за формула, а като за набор от

насоки: контролен списък от управленски процеси – като поставяне на цели,
делегиране, наблюдение и преглед – всеки от които трябва да се научите да оформяте.

3.1.6 The Seven Stages of

Implementation

Stage 1: Set Goals and Define Deliverables

In any project, the manager must first articulate one or more goals. Goals should be
SMART. That means: 

 Specific
 Measurable
 Achievable
 Results-oriented
 Time-bound 

These goals should be set in consultation with all relevant stakeholders—leaders,

employees, other departments, and so on—to increase the likelihood of success. 

Okay, you’ve set a SMART goal to build a product with features A, B, and C with help
from team D, by date E, for under F dollars, for market segment G. But that alone
doesn’t make your goal actionable. (How would a team even start building such a
product?). After all, declaring that you will lose 10 pounds in 3 months is nice and
specific, but it doesn’t guarantee that you’ll deliver on eating better or exercising
regularly.

So now the manager has to define deliverables—actionable and “fail-able” tasks that
must be completed in order for a team to hit its targets. Here, the manager needs to
focus on behaviors. What needs to happen, by when, and how will it be measured—in
effect, how will this test be graded? To lose 10 pounds in 3 months, for example, you
might commit to cutting out all sugar and doing 30 minutes of cardio 4 times a week.

Even so, remember that your plan is only a rough draft. There will be many more
iterations to come.

Q: What accounts for projects being launched with unclear goals?

3.1.6 Седемте етапа на изпълнение

Етап 1: Поставяне на цели и дефиниране на резултатите

Във всеки проект мениджърът трябва първо да формулира една или повече цели.
Целите трябва да са УМНИ. Това означава:
Специфичен

Измерими

Постижимо

Ориентирани към резултати

Обвързани във времето

Тези цели трябва да бъдат поставени след консултация с всички съответни

заинтересовани страни – лидери, служители, други отдели и т.н. – за да се
увеличи вероятността за успех.

Добре, поставили сте си SMART цел да създадете продукт с функции A, B и C с

помощта на екип D, по дата E, за под F долара, за пазарен сегмент G. Но това
само по себе си не прави целта ви действащ. (Как изобщо един екип би започнал
да изгражда такъв продукт?). В крайна сметка, декларирането, че ще отслабнете с
10 паунда за 3 месеца е хубаво и конкретно, но не гарантира, че ще се храните по-
добре или ще спортувате редовно.

Така че сега мениджърът трябва да дефинира резултати – приложими и

„неуспешни“ задачи, които трябва да бъдат изпълнени, за да може екипът да
постигне целите си. Тук мениджърът трябва да се съсредоточи върху
поведението. Какво трябва да се случи, до кога и как ще бъде измерено –
всъщност как ще бъде оценен този тест? За да отслабнете с 10 паунда за 3
месеца, например, може да се ангажирате да изключите цялата захар и да
правите 30 минути кардио 4 пъти седмично.

Въпреки това не забравяйте, че вашият план е само груб чернова. Ще има още
много итерации.
Q: Какво обяснява стартирането на проекти с неясни цели?

VP: Projects that start with unclear goals are very risky and most likely unsuccessful.
Experienced managers must have an extremely important reason or emergency in order
to decide to start such a project.

Hendrina: Because they are missing one of the SMART components, thus missing one
of the pillars of the foundation.
Faith: Many things may account for projects being launched with unclear goals.
-Not fully understanding the project.
-Being in a rush to complete the project. 
-Inexperience by the management team.
-Not consulting all parties that are affected or included in the project.
Allen: Teams may be excited about getting started on the work and do not stop to
consider what success looks like and how it can be measured. It may also be that the
team is doing something for the first time ever and there are no clear indicators of
success.

Stage 2: Determine Roles, Responsibilities,

and Relationships
Now that we know—for the moment, anyway—what needs to be done, we need to know
who will do it. That is, what roles are needed and what responsibilities will each one
have. You also have to clarify relationships: exactly how these different players—
people, teams, departments, vendors—will interact. Who will report to whom? Who will
need to collaborate to get the job done?

But this is more than “sketching out the org chart.” You need to spell out how people will
interact in practice: decision rights, accountabilities, rules of communication, and so on.
For example, who gets to make the call on a certain piece of the project? Who is
ultimately accountable for the success (or failure) of a deliverable? Who will provide
updates, write up status reports, and conduct check-ins to make sure everyone is on the
same page? 

Teams might use a RACI matrix or a similar tool to make these distinctions clear. A
RACI matrix, for example, specifies who is responsible for what deliverable, who is
accountable for the quality and timeliness of that deliverable, who needs to
be consulted on key decisions concerning the deliverable, and who needs to
be informed as the deliverable becomes due and the project continues. Being upfront
about roles, responsibilities, and relationships early on is absolutely crucial because it
prevents confusion and turf wars down the line. 

Stage 3: Delegate the Work

Delegation brings the first two stages together and into practice by formally assigning the
deliverables defined in stage 1 to the people in the roles defined in stage 2.

But you can’t just hand out work and send people on their way. To delegate effectively,
you have to give team members enough resources, background, context, authority,
latitude, support, and guidance. So delegation is itself a multi-step process that unfolds
throughout the implementation—and, as we’ll see, even after. 

Q: Clarity in roles is essential for effective implementation. Why do organizations so

often fail to define and communicate roles and responsibilities of the individuals involved
in implementation processes?

VP: Employees must have a thorough understanding of their roles and responsibilities
during and after the process. If a manager underestimates this element of the
implementation process, the result will not be what is needed or expected.

Faith: Just as sometimes goals are unclear in a process, the roles and responsibilities of
the individuals involved can often times not be properly defined or communicated. Like it
is stated above, delegating the work is connecting stage 1 of setting goals and stage 2 of
determining the roles together. If one of these elements is not properly established then
it becomes impossible to define the roles and responsibilities of individuals. On top of
that communication is one of the biggest challenges in an organization and many times
things fall through the cracks if an implementation process is not well managed.
Hendrina: Speaking from my own experience, I'd say they fail often because:
- there are multi-role tasks and there's no clarity about who's accountable;
- there's not enough support and guidance on-the-job;
- there's a topdown approach, not leaving space for the accountability of others.

Stage 4: Execute the Plan, Monitor Progress

and Performance, and Provide Continued
Support
As an experienced manager once told me, only partly in jest, “This is where the magic
happens.” No matter how well planned, a project will hit road bumps during execution.
So a manager must consistently monitor progress and performance. As people
are doing, the manager needs to be viewing and comparing what’s been done to
agreed-upon milestones and quality standards. This isn’t micro-managing; that’s telling
people how to do what they already know how to do or can figure out for themselves.
This is passing on crucial information—from your manager’s viewpoint at a somewhat
higher altitude—on whether people need to change course (which they can do in stage
five). 

Monitoring is a lot more work than it sounds, but you can make it easier by making the
process more systematic; for example, by planning daily, weekly, or monthly check-ins,
establishing due dates and milestones, and sending out status reports. Such methods
can serve as early warning systems that bring problems to the surface before they
become serious.

Monitoring, however, should not be a one-way street. Use the information you gather to
offer resources, remove roadblocks, give feedback, and generally provide continued
support. Monitoring will always be something of a burden to employees, but if that isn’t
outweighed by the help it provides, then you’re doing something wrong.

So you can think of this stage as a cycle of “doing, viewing, and helping.” 

Stage 5: Take Corrective Action (Adjust or

Revise)
No matter how well you manage the first four steps, mid-course corrections will be
necessary. 

In some cases, managers have to make minor tweaks and adjustments as new
information becomes available. Maybe another coder is needed or maybe market
research needs more time to pursue what’s turning up in the focus groups. In other
cases, perhaps because of unexpected competitive moves, technological
breakthroughs, or the loss of a key performer, managers will have to significantly
revise their plans and essentially start over—completely redefining the deliverables, the
roles and responsibilities, and so on. It may even mean going back to the drawing board
and formulating a new strategy altogether.

The key decision at this stage is: “Do we need to adjust?” or “Do we need to start over?”
It’s not always obvious.

As we said, implementation is invariably an iterative process. And for more experimental

projects, like making a new product or trying out a new business model, starting over or
even failing regularly should be the norm.

Q: Earlier, we said that effective managers identify and monitor red flags when
supervising the implementation process.  Nonetheless, many managers ignore early
signs that the process is off track or that one of the key assumptions underlying the
process are invalid. What do you think explains that?

VP: Assumptions are usually not discussed because they are considered "known" by
everyone, so there is no expectation of risk and a warning sign in this direction.
Faith: I think managers are hopeful that in the end the plan will work out. Many
resources such as time, money and energy, have been invested in getting to this point.
To admit that it does not work is seen as too much of a failure to some. By making small
adjustments along the way, they could be saving themselves from starting over
completely, but this too to some shows defeat. Many times managers suffer from
confirmation bias and are unwilling to recognize the errors that appear in the process.
Hendrina: Possible explanations:
- having a reputation to live up to;
- "we've always done it this way and it's a proven method";
- possible failure in the past, laying too much emphasis on the final outcome instead of
on the process to actually get there;
- rooster behavior.

Stage 6: Get Closure on the Project and

Agreement on the Output 
At the end of a project, everyone on the team should be in agreement on what “done”
looks like. But, in fact, they often have widely varying perceptions of what was produced
and how well. So managers must meet with the team to acknowledge the varying
opinions and try to get everyone aligned on what they have completed and how it
compares to what was originally envisioned. They should also communicate the results
of the project widely and give credit to individuals as well as to the team. Having a
launch party does not substitute for any of this.

Stage 7: Conduct the Retrospective Review

While stage six concerns the final outcomes, the seventh stage of implementation
involves conducting a review of the implementation process itself. Reviews can be very
formal or informal, but need to get at a few basic questions. What did we set out to do?
How did it go? What went well? What went poorly? How can we do better next time? A
manager should seek feedback from individual team members on how he or she
managed the process and should provide feedback and coaching for next time. An
organization that hopes to learn must also articulate a set of lessons learned after each
project. This is extra work, but not as much extra work as repeating your mistakes.

Q: As you consider this seven stage process, what do you think your organization does
well? Why? This is a shared reflection, so you may wish to be vague with details.

VP: In the culture of our company, the mentioned parts of the implementation process
are deeply integrated and are strictly implemented.
Projects and tasks are different in type, deadline and commitment of human resources.
For each of them it is necessary to use some or all parts of the implementation process
described so far.

The obligatory parts are: setting the goal and the expected result; determine the
participants in the team, their responsibilities and the relationship between them, as well
as the delegation of parts of the task - if necessary; in the process of implementation, the
manager responsible for the implementation of the task and the team, monitors the
progress in the implementation of the task and takes corrective action in case of
omissions; after the completion of the task the produced result is either handed over to
the next team for use or further development, according to their competence, or provided
to the client as execution of the order. In case of project related to the preparation of
tender documentation and participation in a tender and in case of a positive result, we
congratulate the team for successful result than we provide the contract to the
competent team for execution. In case of a negative result - we review our performance
and analyze the reasons for it, assess the objectivity and the existence of arguments for
appeal, etc. Adherence to the steps in the implementation process is a proven tool in
our professional history.

Faith: I think my organization does well on the overall evaluation of the process but
tends to fail more when it comes to the individual evaluation. Sometimes we tend to think
that the overall success or failure is representative of the success or failure of the
individuals. We tend to think there is always a direct correlation and in that we miss
some opportunities to recognize strengths or weaknesses of our colleagues. I believe we
are trying to get better at this but it is a very large process to change how we think and
will take some time.
Hendrina: What our organization does well: learning from past mistakes. I'm confident to
say we're conducting the retrospective review and are truly benefitting from it.
Allen: Our organization does well with Stage 1 (Set Goals and Define Deliverables),
Stage 4 (Execute the Plan, Monitor Progress and Performance, and Provide Continued
Support), and Stage 7 (Conduct the Retrospective Review). We are actively making
investments in clarifying Stage 2 (Determine Roles, Responsibilities, and Relationships)
since we have many new people on the team and a recent reorg has shifted
responsibilities.

Stage 1: Set Goals and Define Deliverables

In any project, the manager must first articulate one or more goals. Goals should be
SMART. That means: 

 Specific
  Measurable
 Achievable
 Results-oriented
 Time-bound 

These goals should be set in consultation with all relevant stakeholders—leaders,

employees, other departments, and so on—to increase the likelihood of success. 

So now the manager has to define deliverables—actionable and “fail-able” tasks that
must be completed in order for a team to hit its targets. Here, the manager needs to
focus on behaviors. What needs to happen, by when, and how will it be measured—in
effect, how will this test be graded? To lose 10 pounds in 3 months, for example, you
might commit to cutting out all sugar and doing 30 minutes of cardio 4 times a week.

Stage 2: Determine Roles, Responsibilities,

and Relationships
Now that we know—for the moment, anyway—what needs to be done, we need to know
who will do it. That is, what roles are needed and what responsibilities will each one
have. You also have to clarify relationships: exactly how these different players—
people, teams, departments, vendors—will interact. Who will report to whom? Who will
need to collaborate to get the job done?

https://www.novavizia.com/matritsa-raci/

Матрица RACI на отговорностите

Матрицата RACI на отговорностите е акроним от четири
думи:

 R (Responsible) – Реализиращ. Това е човекът, който

реално работи по задачата и я реализира на практика;
 A (Accountable) – Отговорен. Това е човекът с най-
висока отговорност и правомощия относно задачата;
 C (Consulted) – Консултиращ. Това е човекът, с
когото следва да се консултира задачата;
 I (Informed) – Информиран. Това е човекът, който
трябва да е информиран относно задачата.

Приложение на матрицата RACI

За да се използва матрицата RACI, е необходимо да се прилага
следната последователност от стъпки:

 Изяснява се заданието – проект, крайна цел, решение.

 на едната ос на матрицата се описват всички задачи и
дейности, които трябва да се извършат.
 на другата ос на матрицата се описват всички хора в
екипа.
 Получава се матрица, в която с помощта на буквите
R, A, C и I се отбелязва какво е отношението (ролята)
на всеки човек по отношение на задачите/дейностите,
които следва да се извършат. За всяка отделна
задача/дейност се посочва ясно кой ще работи, кой е
отговорен, кой ще консултира и кой ще е
информиран.
 На всеки ред от матрицата RACI следва да има само
един отговорен (A) и поне един Реализиращ (R).
Консултиращи (C) и Информирани (I) са само опции,
които не са задължителни.
Когато матрицата RACI е попълнена, е добре да се огледа с
критично око. Полезно е да се прецени:

 Дали за всички дейности и задачи има балансирано

разпределение на хора?
 Дали хората притежават нужната висока
компетентност, за да се справят с отговорностите си? 
След като всичко е изяснено, трябва да се запознаят всички
членове на екипа, за да са наясно с ролите и отговорностите си.
Тези роли и отговорности могат да бъдат обсъдени и
предварително с хората, разбира се.

HBS
Again, our seven-stage model is a highly idealized version of how plans are
implemented. Real implementation is messier, which is why I call it more of an art than a
science. But artists, too, must learn, practice, and improve.

Q: As you reflect on the seven step process, do you think that one or more of these
steps would be very difficult to implement in your organization? Why? This is a shared
reflection, so you may wish to be vague with details.

VP: As an improvement or weakness, I would suggest that "Conduct the Retrospective

Review" should be discussed in more detail with the team, and not just "tick" the final
result as a successfully completed project and continue with the next one, because there
are always moments on the path to a "successful project" from which it is worth drawing
conclusions for better performance next time.

Faith: An area that may be very difficult to implement in my organization is delegating

the work. There tend to be two extremes that occur. Either work is "over" delegated
where individuals are often micro-managed or leaders and managers have issues
delegating and essentially "giving" up work. There are many individuals in my
organization that are so invested and so connected that they want to be involved in
every little thing. This is essentially a downfall as this is not possible to effectively get
work done. I believe that as leaders and managers we need to trust our people to take
on roles and responsibilities. Their work is truly a direct reflection of us.
Hendrina: Stage 3: although it's very important to do so. I think difficulties will arise
because of:
- an investment of time (training and coaching others);
- an investment of resources (getting them equipped with the right knowledge and skills);
- the hybrid workplace we're having now. A lot of employees are working from home.
Allen: None of the seven steps would be very difficult to implement, but the one that is
most often overlooked as important is Stage 7 (Conduct the Retrospective Review). This
is a required step in our process, but after putting a substantial amount of time and effort
into saving a customer, there is little desire to conduct a churn post-mortem to revisit the
reasons why the customer chose to use another solution. We already do thorough
reviews at the leadership level on a regular basis, but we could be more diligent in
reviewing at the manager/individual contributor level.

In the next two lessons, we’ll study two cases in which leaders attempted to manage the
implementation of a major undertaking. One failed, the other succeeded. Both outcomes
will show how important it is to treat implementation as a process.

3.2.1 Failures and Miracle of

Implementation
As we’ve seen, neither the failure nor the success of a project is likely to be due to a
single cause or a single person. Almost any plan, project, initiative, or mission—certainly
in business—involves multiple people and processes and it’s only when they are aligned
that implementation goes well. That’s where managers come in.

For the remainder of this week’s content, we’re going to walk through two cases of
implementation—the first, the launch of Healthcare.gov, will serve as an example of
ineffective implementation. The other example will be one of effective implementation—
the rescue of Chilean miners trapped deep in a mine.

For each case, we’ll walk through the steps of implementation and discuss what they did
wrong and what they did right. But first, let’s introduce you to what happened.

3.2.1 Провали и чудо на внедряването

Както видяхме, нито провалът, нито успехът на даден проект вероятно ще се дължат на
една кауза или на един човек. Почти всеки план, проект, инициатива или мисия – със
сигурност в бизнеса – включва множество хора и процеси и само когато те са
съгласувани, изпълнението върви добре. Тук влизат мениджърите.

За останалата част от съдържанието от тази седмица ще преминем през два случая на

внедряване – първият, стартирането на Healthcare.gov, ще служи като пример за
неефективно прилагане. Другият пример ще бъде един за ефективно прилагане -
спасяването на чилийски миньори, хванати дълбоко в мина.

За всеки случай ще преминем през стъпките на прилагане и ще обсъдим какво са

направили нередно и какво са направили правилно. Но първо, нека ви запознаем със
случилото се.

Video

Healthcare.gov was widely viewed as a technical, operational, and managerial

catastrophe. President Obama himself called it an “unmitigated disaster.” 

Chilean Mining Rescue

In 2010, Chilean President Sebastian Pinera, the Ministry of Mining, and other officials
were facing a national crisis. The dangerous San Jose mine had collapsed, trapping 33
miners. Rescuing them wouldn’t be easy, if it were possible at all. But, as many of you
will remember, the whole world was watching. 

Video

This “miracle” was a prime example of effective implementation. One sign of that—
contradictory as it may seem—is how many failures and iterations it took to get the men
out.

3.2.2 Healthcare.gov: What Not to Do

To understand the failure of Healthcare.gov, let’s begin by recalling our definition of

effective implementation:

Effective implementation is delivering what is planned or promised; on time, on budget,

at quality; and with a minimum of variability; even in the face of unexpected events and
contingencies. 

The website launched on October 1st did not meet any of these criteria:

 Delivering what is planned or promised: The goal of the website was to

provide a seamless, high-quality shopping experience to Americans
seeking to buy health insurance coverage via the Affordable Care Act
(ACA). To accomplish this goal, the government would need to build an e-
commerce exchange in which individuals could compare insurance plans,
apply eligible subsidies, and buy plans for themselves and their families.
But most visitors to the site could not accomplish this goal when the site
launched.  They encountered technical difficulties, could not log in, and
had trouble navigating the site.
 Delivering on time, at budget, at quality: Lack of technical skills, poor
management, major communication issues, and policy delays caused the
user testing window for the site to shrink significantly. The result was poor
quality. As a result, the government would need to spend more time and
money fixing it after it crashed.
 Delivering with a minimum of variability: The site could handle far
fewer users than anticipated and would frequently crash. Load times were
slow. People would frequently—and often unpredictably—be logged out.
Experiences with the site were highly uneven.
 Delivering despite unexpected events: Those tasked with building the
site failed to account for a number of contingencies: that this was not like
any project the government had done before; that a number of Republican
governors would refuse to build state-level insurance exchanges,
dramatically increasing reliance on the federal exchange; that CMS would
lose crucial funding from Congress to implement the law; that the White
 House would face delays in writing insurance regulations; and others.
Given the law’s immensity and its toxic reputation on the political right,
failing to plan for these very real contingencies was a classic case of
overlooking predictable surprises.

Let’s look more closely into this flawed implementation process, guided by Professor Len
Schlesinger, the author of the Healthcare.gov case here at Harvard Business School.
We will also hear from Mina Hsiang, who was involved in the fix of the website and who
is now part of a new initiative to prevent such failures from happening again.

Video

3.2.3 The Chilean Mining Rescue:

Implementation Done Right

On October 13th, 2010, the last of 33 trapped miners was extracted from the San Jose
mine. The search and rescue had taken 69 days, cost about US$20 million, and required
the cooperation of hundreds of experts, disciplines, government agencies, industries,
and organizations. Yet despite so many contributions, the success was anything but a
sure bet. On several occasions, experts figured the chances were almost nil. In many
ways, then, the team had pulled off a miracle. But they did it by managing an effective
implementation process:

 Delivering what is planned or promised: Upon hearing of the cave-in,

President Pinera publicly committed the government to rescuing the 33 miners.
This was accomplished.
 Delivering on time, at budget, at quality: Experts estimated that the miners
could last a maximum of 30-40 days without food. Given this estimate, the team
assessed all rescue options and quickly determined that drilling was the only
option. They designed a drilling plan based not on quality or cost, but on reaching
the miners before they died of hunger. On the 17th day, the drillers found the
men, in time to provide them with food and supplies through small holes until they
could get them out. So there were no budget or quality requirements, but the
deadline—get to them before they died—was met.
 Delivering with a minimum of variability: Team members worked in multiple
shifts to drill around the clock. Rescuers carefully managed handoffs from one
shift to the next to prevent miscommunications and reduce variability. They also
met frequently to discuss progress and performance and solve daily problems.
Leaders kept everyone focused by reminding them of their one common purpose:
finding and rescuing the miners. 
 Delivering despite unexpected events: There were near misses, broken drill
bits, and technological surprises. One important factor was that the team,
 knowing how little they could be sure of anything they were doing, didn’t put all its
eggs in one basket. There were, in fact, multiple rescue plans, so that if one plan
hit a setback, they could keep going on another. 

Here’s Andre Sougarret, the lead engineer of the rescue, to frame the immense
challenge and recount how they got the job done:

Video

ANDRÉ SOUGARRET: I currently work at Antofagasta Minerals mining group. My

previous job, during the time of the San José mine accident, I was working at Codelco,
at the El Teniente Division.

The mining industry is one of Chile's most important economic activities. We are the
world's leading copper producer. The San José mine was 100 years old. After the
collapse, all the companies and the society turned into a supporting network in order to
rescue the miners. The first attempt was unsuccessful. The mine was active. There were
continuous rockslides. So it didn't succeed. That's when the government steps in and
asks for the help of the large mining companies in the country.

Eventually, Codelco was hired for this and I was sent to the mine to see what could be
done. We didn't have much information at the beginning. The geological data and plans
of the mine were pretty useless and we couldn't give an accurate report at first.
Therefore we had to check further, on the ground, to understand what the real situation
of the mine was. The mine had one access. The access had been blocked at 4 km depth
by a large mass of rock.

The challenge was figuring out whether the miners were alive. The first thing they told us
was that the miners were working in an area outside where the collapse happened. As
we started checking the records and the conditions the miners had been working in, and
talking to people who worked there who knew well what and how the tunnels were, we
realized there was a chance of finding some alive To be honest, I never thought that the
33 of them would be alive and well. We expected to bump into at least one tragedy
among the trapped miners. In the first stage, we ruled out any chance of getting through
the underground mine, so we started drilling from the surface which would allow us to
contact the miners. Bear in mind that these drillings were 10 cm wide. It was clear that
the miners couldn't be pulled through those drillings.

However, when we were able to contact them, we would be able to know their situation
and provide food, which was the most important priority that they needed. Therefore we
put 7 drills at the same time allowing us to be in contact with them from different spots,
to the place that we assumed they were, at the bottom of the mine. That went on for 17
days.

On the 17th day, we were able to contact them. I'd say that was the most important
moment, emotionally speaking, of the rescue. Knowing that they were alive was so
important, it would also allow me to keep a promise that I had made to the families, that I
would give them an answer, either good or bad, of what was going on with their beloved
ones.

Video
ANDRÉ SOUGARRET: People were crying. Everyone hugged us. Many families who
did not believe that we were working to dig them out came up to me to apologize, They
were crying of happiness, because we had found their relatives alive.

In the second stage of the rescue, we had to decide how to dig them out and in order to
do that, the length of the drilling must be between 600 and 700 meters. Then we created
the first team and named it Plan A. The aim of Plan A was to use a raise borer to
basically dig a central drilling shaft. Then, with a reamer, we would widen the initial
drilling, and complete a ventilation shaft that we could use to dig them out and rescue
the miners. That was Plan A. That plan would take approximately 4 months. That was
the plan. We knew the plan would be risky so we began planning "Plan B".

Basically, the aim of "Plan B" was to take one of the original three drillings we had made
to keep in contact with the miners, and use different-sized drillings to widen it. And the
third plan, just in case the other two failed, would use a drilling like the ones used for oil.
These types of drillings are big and take time to install properly.

Once the raise boring was already drilled, we had to make a decision about which
capsule or cage was going to pull the miners out and we had to prevent any slide or
collapse. To address that, we already had a pipe of the same size of the hole which we
drilled, and this would allow us to maintain the width of the shaft through the 600 meters.
The fortification had to be strong enough to withstand any collapses. That was one of the
risks we had. However, putting in a pipe 600 meters deep was complicated. One of the
risks of putting in the pipe was that it might fall and block the hole. Finally, at the meeting
we had, we decided which spot would need the pipe the most. We decided that only the
first 100 meters from the surface had the greatest risk of rockslides. In that zone only,
we inserted a pipe in a way that it couldn't collapse and the cage could run throughout
the hole without a problem.

Luis Urzúa, who was the deputy chief of the trapped group said that he had finished his
shift. That was his expression. He said he had handed over the whole team safe and
sound and, of course, he thanked me and the team for having helped them out and
reunited them with their families.

3.2.4 Differences in Implementation

Why did the American government fail in the construction of an online exchange—
something that had already been done well by any number of companies and
organizations—while the Chilean government succeeded with a one-of-a-kind rescue
that all the experts considered a desperate long shot? 

One factor—among many, of course—is that the two groups had very different mindsets.
One group simply assumed implementation would occur, while the other assured it.

Here’s Prof. Schlesinger to describe the difference:

Video

Stanford University management scholar James March once said that, “leadership
involves plumbing as well as poetry.” That is certainly true of effective implementation.
Unfortunately, many managers know how to inspire and motivate others (the poetry), but
not how to design, install, shape, and direct processes, manage people and
relationships, and fix organizational problems (the plumbing). 

Q: Describe a time in which you assumed that something—a strategy, a plan, a decision
—would take care of itself when you should have assured it would happen. What was
the result? What should you have done instead? This is a shared reflection, so you may
wish to be vague with the details.

VP: A common case of such weaknesses is, for example, when hiring a new employee
for a position in the company. It is assumed that the relevant managers will present it to
everyone they will work with; will acquaint him with the rules and requirements of the
workplace; will provide him with the necessary equipment and environment; they will
guide him for a while - at least until they make sure that the employee is able to work
independently, understand and apply the processes.

When the necessary steps in the integration of the employee are not performed, various
risks can occur. Therefore, the role and responsibilities of managers in this case is to do
exactly their job and "launch the ship".

Faith: While assisting in project managing the implementation of a new software system
I was assigned with making each department aware of when trainings they needed to
take occurred. I assumed that they would take the trainings they needed in order to use
the new system. This did not occur with everyone. Instead I should have set weekly
checks for each department and employee to assure they attended the trainings and
take note of any concerns. This led to the need of additional training for many.
Allen: I had designed a process by which we could more effectively solve a common
problem at our company. I assumed that by staffing the process with people and
explaining the steps and importance of the process, it would be taken care of. This did
not take place. Rather than simply explain the process, I should have taken time to
validate the steps with the ones participating, encouraged dissent and disagreement to
identify any flaws in the process, and also clearly defined SMART goals for measuring
success for the team.

When implementation is merely assumed, the process is likely to unfold in

unforeseeable (and often undesirable) directions. When managers do their best
to assure implementation by managing it as a process—knowing how easily it can go off
the rails and taking care to foresee and prevent that—they have a much better chance
(sorry folks, never a guarantee) of getting the job done.
Here’s Prof. Amy Edmondson to describe how the Chilean mining rescue team took the
latter approach:

Video

Q: As Amy pointed out, receiving feedback (early and often) and iterating based on this
new information is a huge part of effective implementation. What is one concrete way
that you could incorporate this mindset into your own organization going forward? This is
a shared reflection, so again - you may wish to be vague with specific details.

VP: I'll try to ensure that everyone is united to achieve a common goal, to work as a
team to achieve it, to communicate in a timely and adequate manner, to discuss their
ideas and reach common decisions and implementation, to monitor progress and correct
when needed.

Hendrina: To be honest, we're doing a good job on this already. But one way to
incorporate this is: to plan regular moments for reflection/reviewing, in different phases.
Make this an actual part of the process. Be sure you're not skipping this before going to
the next step.
Faith: I think simply sharing this idea as a main principle to follow is important for my
organization. Even being in a field where people depend on you for their life, with the
need of food, it is ok to fail and try again and improve. In the end it only helps us do even
better. I think we believe we cannot fail because people depend on us, but looking at the
Chilean mine rescue is a perfect example of how in the end failures can produce the
best result.

3.3.1 Set Goals and Define

Deliverables

If people don’t know what their goal looks like, they’re not likely to get there. That’s why
the first stage of implementation is to set goals and define deliverables. The
“deliverables” are small, manageable, actionable tasks and activities. For example, if the
project’s goal is to “increase our sales volume by 10% in 6 months,” deliverables might
include “create a sales brochure by March with help from the creative department” or
“design two advertisements for release in GQ and Rolling Stone.” You can’t know all the
necessary deliverables at the outset, but you certainly need to know some.

Some key questions for managers to consider during this first stage of implementation
are:
  What’s the ultimate goal of the project? What’s our target?
 What needs to be delivered to accomplish that goal?
 Who is it for? What is the target audience?
 When does it need to be delivered? What is our timeline? 
 What resources, support, and budget do I have?
 What is the scope of the project? 
 Have we done this before?
 Who needs to be involved? What knowledge or skills are necessary to get the job
done?
 What standard of quality is acceptable? What does success look like?
 How much leeway do I have in terms of resources, costs, time, etc.?
 With what frequency must the end product or service be delivered (one-off,
recurring, etc.)

Note: for each stage of implementation, we’re going to provide you with a list of key
questions for that stage. They will all be included in the takeaway document provided at
the end of this course.

3.3.1 Поставяне на цели и дефиниране на резултатите

Ако хората не знаят как изглежда целта им, няма вероятност да стигнат дотам. Ето
защо първият етап на изпълнение е да се поставят цели и да се определят
резултатите. „Резултатите“ са малки, управляеми, приложими задачи и дейности.
Например, ако целта на проекта е „да увеличи обема на продажбите си с 10% за 6
месеца“, резултатите могат да включват „създаване на брошура за продажби до
март с помощта на творческия отдел“ или „проектиране на две реклами за пускане
в GQ и Rolling камък." Не можете да знаете всички необходими резултати в самото
начало, но със сигурност трябва да знаете някои.

Някои ключови въпроси, които мениджърите трябва да обмислят по време на този

първи етап на изпълнение, са:

Каква е крайната цел на проекта? Каква е нашата цел?

Какво трябва да бъде предоставено, за да се постигне тази цел?

За кого е? Каква е целевата аудитория?

Кога трябва да бъде доставен? Каква е нашата времева линия?

С какви ресурси, подкрепа и бюджет разполагам?

Какъв е обхватът на проекта?

Правили ли сме това преди?

Кой трябва да участва? Какви знания или умения са необходими, за да се свърши

работата?

Какъв стандарт за качество е приемлив? Как изглежда успехът?

Колко свобода на действие имам по отношение на ресурси, разходи, време и т.н.?

С каква честота трябва да се доставя крайният продукт или услуга (еднократно,

повтарящо се и т.н.)

Забележка: за всеки етап на внедряване ще ви предоставим списък с ключови

въпроси за този етап. Всички те ще бъдат включени в документа за вкъщи,
предоставен в края на този курс.

Before comparing the differences between Healthcare.gov and the Chilean mining
rescue, we’d like you to meet someone from an organization that sets goals and defines
deliverables quite well: the US Army. 

Colonel Paul Reese is former Director of the Center for Army Lessons Learned (CALL),
which collects and analyzes data from Army training events and deployments and
translates it into best practices and lessons learned.

A key principle for all Army missions is “Commander’s Intent.” According to Army
doctrine:

“The commander's intent describes the desired end state. It is a concise expression of
the purpose of the operation and must be understood two echelons below the issuing
commander. . . It is the single unifying focus for all subordinate elements… Its purpose
is to focus subordinates on the desired end state. Its utility is to focus subordinates on
what has to be accomplished in order to achieve success, even when the plan and
concept of operations no longer apply, and to discipline their efforts toward that end.”

In simple terms, the Commander has to state the purpose of the operation so the
soldiers understand why they are doing what they are doing and what will make sense to
do instead if the circumstances suddenly change.
Here’s Col. Reese explaining how important that is:

Video

Healthcare.gov
Unfortunately, this is what President Obama and other leaders didn’t do with
Healthcare.gov. The goal was vague and there were misunderstandings about the
deliverables.

At the very top, President Obama and other White House policymakers conflated
passage of the ACA bill (their main deliverable as politicians) with the provision of health
insurance (the broader goal of the law, which obviously involved many other parties
besides politicians). Having formulated the policy, they paid far too little attention to the
technical deliverables—the regulations for insurance companies, the e-commerce
platform, the systems integration, and so on—that were needed to achieve the goal of
providing health insurance to the public. For them, delivery of the website was in many
ways an afterthought. But, in reality, it was the key operational deliverable on the way to
making their goal of affordable health care real. It was a deliverable without which they
couldn’t get the job done.

Q: Returning to the list of questions presented at the beginning of this page, which
questions do you feel the Obama administration failed to adequately investigate?

VP: At least the following:

What needs to be delivered to accomplish that goal?

What resources, support, and budget do I have?

Have we done this before?

Who needs to be involved? What knowledge or skills are necessary to get the job done?

What standard of quality is acceptable? What does success look like?

How much leeway do I have in terms of resources, costs, time, etc.?

With what frequency must the end product or service be delivered (one-off, recurring,
etc.)
Hendrina: I think the overseen questions are:
- What needs to be delivered to accomplish that goal?
- What is the scope of the project? 
- Who needs to be involved? What knowledge or skills are necessary to get the job
done?
- What standard of quality is acceptable? What does success look like?
Faith: After getting the bill passed to allow for affordable healthcare there was no
consideration of what it would look like to make this a reality. None of the below
questions were adequately investigated:
What needs to be delivered to accomplish that goal?
When does it need to be delivered? What is our timeline? 
What resources, support, and budget do I have?
What is the scope of the project? 
Have we done this before?
Who needs to be involved? What knowledge or skills are necessary to get the job done?
What standard of quality is acceptable? What does success look like?
How much leeway do I have in terms of resources, costs, time, etc.?
With what frequency must the end product or service be delivered (one-off, recurring,
etc.)

The second misunderstanding occurred at the departmental level, in particular within the
Center for Medicare and Medicaid Services (CMS).  Housed within the Department of
Health and Human Services (HHS), CMS would ultimately become responsible for
implementing the law. 

Faced with what for them was a novel task—creating a web platform—CMS nonetheless
believed it could use its traditional work processes. For example, CMS managers
assumed that they could contract out various pieces of the site to their regular stable of
IT contractors. But these were not vendors who really knew how to build an e-commerce
site.

Furthermore, CMS managers believed that they could integrate the various pieces of the
site itself; that is, that they could be the project’s general contractor. Numerous people
within government, including several key players within CMS itself, doubted whether the
Center had the ability to coordinate the activities of 55 contractors. CMS’s software
engineering department was too small, its managers too inexperienced, and its IT
systems too antiquated for such a complicated job. US Chief Technology Officer (CTO)
Anish Chopra and HHS CTO Todd Park raised these issues with HHS Secretary
Kathleen Sebelius, but were rebuffed.

Here’s Len Schlesinger on how CMS failed to understand the nature, size, and scope of
the work before it:
Video

One way to think about the HHS’s misunderstanding of its goals and deliverables is that
the government made a “flawed analogy.” It compared Healthcare.gov to projects it had
done before, minimizing the differences. 

Chilean Mining Rescue

In contrast, those involved in the Chilean mining rescue had a very clear idea of their
objective—to find and then extract the miners. As a result, they could link deliverables to
that goal and keep refining their approach in order to deliver. 

For one thing, President Pinera was not just a figurehead. He and his aides stayed in
constant contact with the specialists on the ground and provided whatever resources
and support they needed. With a clear, shared goal, Sougarret and the rescue team
were thus able to set priorities, which in turn enabled them to discern what they needed
to deliver each day.

Video

ANDRÉ SOUGARRET: I think the most important thing was that we all had the same
common objective, which we repeated to ourselves with certain frequency, which was to
give a response, in the first stage, to the relatives, if the miners were alive or dead. That
was like the first thing, but once we had the miners alive, that we knew about them, the
truth is that it went becoming easier because we all have the same goal. Which was to
take out the miners, that they reunite with their relatives.

The conviction and common objective among all the participants I think is vital. In
particular, in this mission that we had, the bonding we had between the different people,
many of us which did not know each other, was really that we had a common goal.

These two cases have already delivered some valuable lessons: 

 Don’t mistake planning for execution. Assure implementation by managing the

work as a process.
 Avoid flawed analogies to past work. Test all comparisons to past work to
ensure their relevance. (Have we done something like this before? In what ways
can we use old processes? Might we need to establish new ones?)
 Goals that are not well-articulated, well-communicated, or widely shared will not
be met. Deliverables that are not actionable or not clearly linked to the project’s
goals will not be delivered. 
 Reiterate goals and deliverables at every chance. Remember: if it can’t be
measured, it can’t be managed.

  Тези два случая вече дадоха някои ценни уроци: Не бъркайте
планирането с изпълнението. Осигурете изпълнение, като управлявате
работата като процес. Избягвайте погрешни аналогии с минала работа.
Тествайте всички сравнения с минала работа, за да се уверите, че са
уместни. (Правили ли сме нещо подобно преди? По какви начини
можем да използваме стари процеси? Може ли да се наложи да
установим нови?) Целите, които не са добре формулирани, добре
комуникирани или широко споделени, няма да бъдат постигнати.
Резултатите, които не подлежат на действие или не са ясно свързани с
целите на проекта, няма да бъдат доставени. Повтаряйте целите и
резултатите при всяка възможност. Запомнете: ако не може да се
измери, не може да се управлява.

Q: Comparing the two cases, what do you think the most important differences in the
decision-makers approaches were in setting goals and defining deliverables?

VP: In my opinion, the approach in both cases is political. The two Presidents set goals
to achieve results that are important to society.

There are huge differences in their approach to implementation. In the case of the site -
there is an underestimation of the technological, legislative and educational complexity
of the project. In the case of miners, the responsibility is given to top experts, and
unreserved support and resources are provided by politicians.

Faith: I think the most important difference in decision-makers approaches were in

setting goals and defining deliverables is that from the beginning in the case of the
Chilean mine rescue the goal was incredibly clear. Nothing was assumed of how this
goal was going to be reached and the connecting deliverables. When it came to the
Healthcare.gov decision-makers this was the complete opposite. It was just assumed
that things would happen and because of this deliverables were not defined, setting the
process of implementation up for failure.
Allen: In the case of Healthcare.gov, the goal was less clear - to create some sort of
website where people could sign up for health insurance. This ambiguity left a lot to the
imagination and did not have clear milestones for development and iterative testing,
which are best practices in software development. Also, it was assumed that the
agencies and contractors who were already in place would have the skills and people to
take on such a large and complex project. Because of this inexperience, deliverables
were unlikely to be oriented toward ensuring overall success - simply what they thought
might work.

In the case of the Chilean Miner Rescue, the goal was very clear - first to confirm if the
miners were dead or alive, and once they were confirmed to be alive, to bring the miners
back to their families safely. There were no assumptions about anything working. In fact,
they assumed that everything would fail, so they made multiple contingency plans. The
case study has not mentioned it, but there clearly must have been some devil's
advocacy or critical thinking involved, since the teams were able to anticipate failures
and plan for them - such as the example of including the support pipe to prevent
rockslides at the highest 100m of the new hold drilled.

3.3.2 Determine Roles,

Responsibilities, and Relationships

This second stage will overlap somewhat with the first, as the most realistic deliverables
are typically defined in consultation with the people actually charged with delivering
them. If you don’t know much about a particular coding language it will be hard to fully
define the deliverables (stage 1) until you’ve chosen a developer (stage 2) who can help
you do that. 

In determining roles, a manager should consider the following questions:

 What types of knowledge, skill, and prior experience will each role require?
 Whom do I know who fits the bill?
 Who will be the leader?
 Who should not be involved in this project?

In determining responsibilities, a manager should consider the following questions:

 Who will be ultimately accountable  for project success?

 Who will own and be accountable for each deliverable?
 What needs to be delivered? When? With what quality? With what budget? With
what variance?
 If key deliverables require multiple people, which pieces should each person work
on?

In determining relationships, a manager should consider the following questions:

 Who will report to whom?

 Who will have access to what information?
 Who will have decision rights on key choices that we know must be made down
the road?
 Who needs to provide status updates and progress reports?
 Who needs to communicate regularly with whom?
 Who gets to provide information, input, and recommendations on key decisions?
 Whom should people go to if they need help, come into conflict, or have trouble?
Healthcare.gov
Lacking a firm understanding of goals and deliverables, officials continued to mismanage
implementation of the healthcare law by poorly determining roles and responsibilities
down the line.

At the top, President Obama had to choose which White House team should be
responsible for managing the law’s implementation and, by extension, the
Healthcare.gov website. 

Video

Obama assigned the project to Nancy-Ann DeParle, director of the White House Office
of Health Reform and Healthcare Policy, seeing her as a policy expert who was familiar
with the agencies that would need to get involved. But she had few of the technical skills
required to run a healthcare enterprise or oversee the construction of a website.

Q: Knowing, in hindsight, that Nancy-Ann lacked technical experience, what might you
suggest to ensure her success in the role, and a smooth technical roll out?

VP: To determine participants and responsible for the technical side of the project, their
roles and levels in the decision-making process, communication levels with other
stakeholders, support and control of implementation.

Faith: Knowing Nancy-Ann's lack of technical experience, I would suggest that the first
two questions about roles be examined further: What types of knowledge, skill, and prior
experience will each role require? and Whom do I know who fits the bill? If these
questions had been properly considered then it would have been identified that someone
would need to be found that had the technical skills required to run a healthcare
enterprise or oversee the construction of a website. They would have properly prepared
for the implementation of this program and either found someone in another department
or outsourced this necessary work.
Hendrina: Suggestions:
- proper guidance;
- training on-the-job by experienced employees;
- regular 1-on-1 reflection and feedback sessions;
- tapping into the knowledge and experience of other team members: you don't have to
reinvent the wheel.
Allen: "Surround yourself with experts in the technology, who have built large-scale
systems like this before, and don't be afraid to leverage outside consultants. You've
never done this before - no one in the United States Government has - so be bold, but
stay focused on building a system that makes it easy and fast for people with limited
means and familiarity with technology to sign up for healthcare."
Here's what happened. DeParle was soon promoted to White House deputy chief of staff
for policy. She gave oversight of the implementation process to her White House
successor, Jeanne Lambrew, while assigning the day-to-day job of implementing the law
and, by extension, the website, to HHS Secretary Kathleen Sebelius. (If this
arrangement sounds confusing, that’s because it was.) 

Sebelius had to assign the project to the Center for Medicare and Medicaid Services,
one of the few divisions of HHS with its own revenue streams. However, CMS was a
massive organization with over 4,000 employees nationwide and yet, at the time, had
only a temporary leader. It was also struggling with issues of morale and turnover.

In May 2010, just six weeks after the law had passed, Harvard healthcare economist
David Cutler wrote a memo on the status of implementation at the invitation of Obama’s
economic policy team. He listed a number of factors that he felt would doom the law’s
implementation, including the fact that it was being run by CMS:

“A good deal of reform implementation needs to occur at…CMS. You were dealt a bad
hand here. The agency is demoralized, the best people have left, IT services are
antiquated…You have an agency where the philosophy of health system reform is not
widely shared, where there is no experience running a health care organization, and
where the desire to move rapidly is lacking.”

Video

An organization that’s been thrown into the deep end of the pool can still bring in people
who do have the missing knowledge and skills, as long as it’s open to new ways of
implementing. Unfortunately, CMS didn’t and wasn’t.

In fact, when Chopra and Park, both of whom could bring technical expertise to the
project, went to meetings on Healthcare.gov, Lambrew told them that they wouldn’t be
needed. When Chopra later offered to help with the project, he received a one-word
email response: “STOP.” 

Here’s Prof. Schlesinger on the political resistance that kept CMS from trying something
new:

Video

Q: Considering the list of questions presented at the beginning of this page, which
questions do you feel the Obama administration failed to adequately investigate?

VP: Almost all of them:

What types of knowledge, skill, and prior experience will each role require?
Whom do I know who fits the bill?
Who will be the leader?
Who should not be involved in this project?
Who will be ultimately accountable for project success?
Who will own and be accountable for each deliverable?
What needs to be delivered? When? With what quality? With what budget? With what
variance?
Which pieces should each person work on?
Who will report to whom?
Who will have access to what information?
Who will have decision rights on key choices that we know must be made down the
road?
Who needs to provide status updates and progress reports?
Who needs to communicate regularly with whom?
Who gets to provide information, input, and recommendations on key decisions?
Whom should people go to if they need help, come into conflict, or have trouble?
Faith: I feel like overall the Obama administration failed to adequately investigate all of
the questions presented above but most importantly was the types of roles and
qualifications needed and who would be accountable for which part of the project. They
were essentially unwilling to determine the correct people with the correct skills needed
to make this job a success. And when this started to show there was no accountability
on anyone. The process kept going down a black hole of failure.

Allen: As it pertains to ROLES, these questions were not adequately investigated:

• What types of knowledge, skill, and prior experience will each role require?
• Who should not be involved in this project?

As it pertains to RESPONSIBILITIES, these questions were not adequately investigated:

• Who will own and be accountable for each deliverable?
• What needs to be delivered? When? With what quality? With what budget? With what
variance?
• If key deliverables require multiple people, which pieces should each person work on?

As it pertains to RELATIONSHIPS, none of the questions (included below for reference)

were adequately investigated:
• Who will report to whom?
• Who will have access to what information?
• Who will have decision rights on key choices that we know must be made down the
road?
• Who needs to provide status updates and progress reports?
• Who needs to communicate regularly with whom?
• Who gets to provide information, input, and recommendations on key decisions?
• Whom should people go to if they need help, come into conflict, or have trouble?

In short, other than handing off the project to HHS and assuming that President Obama
would ultimately held accountable for the success or failure, very few of the critical
questions needed for a successful implementation were asked or answered.

Another big problem was lack of accountability. In order to implement the law, multiple
offices inside CMS had to work together with numerous contractors. Yet, because CMS
did not have a leader, there was no single authority who was really in charge. At one
point, there were three separate “command centers” within CMS for building the site,
and HHS had one of its own, too.

With no one really accountable, the project experienced total confusion over roles,

responsibilities, and relationships. That’s when teams enter a dreaded “doom loop,”
which typically proceeds as follows:

Picture
Circular flowchart showing the relation between unclear accountabilities, multiple
command centers, confusion over decision rights, inaction or turf wars, and delays,
blown budgets and/or quality problems.

Multiple groups begin thinking that they are in charge, but aren’t sure. Eventually, people
begin to realize that they need help from other groups, but find they have little authority
to get them to deliver key work according to their desired timelines.  The result is
typically either abdication or political infighting and turf wars. In the end, the job doesn’t
get done, or only very poorly, and instead of lessons learned there are fingers pointed.

Chilean Mining Rescue

In contrast, the leaders of the Chilean mining rescue clearly defined who was to do what.

At the top, President Pinera consulted with Codelco executives to identify and select
Andre Sougarret as leader of the rescue. Pinera introduced Sougarret to the public and
let them know that he was the one in charge of this rescue. The appointment of
Sougarret as the clear technical leader immediately prevented a doom loop scenario.
The many contributors could focus on doing their part rather than struggling over what
part they would do. 

Here’s Dean Nohria on the importance of having a clear leader of the implementation
process and how everything else stems from that choice:

Video

Sougarret, for his part, quickly defined other roles and responsibilities. 

Video

ANDRÉ SOUGARRET: Mmm...Well, the first thing we did was build a work team that
would allow us to organize the large number of people and resources at our disposal.
We need to understand that we weren't the only ones, there was the regional
government, there was many of the miners who worked in the Copiapó area. There was
the Armed Forces. There were emergency service organizations. Therefore, there was a
great number of people, and the first thing we did was sort out those things that would
be useful and those that wouldn't. Try and have less people, basically. And focus on
understanding what the problem was.

And start to generate different work approaches that would allow us to move forward,
both on the surface and inside the mine so we could make some contact with the
miners. The first thing I did was ask the people who were there what they were doing,
what they had at their disposal. And then, having an idea of the problem inside the mine,
I could start dividing in groups of work that would allow us to move forward in different
options and organize all the other services that had to do with taking care of the public
opinion and the relatives.
Sougarret selected people for the rescue based not on political clout or seniority, but by
the usefulness of what they had to offer. He created specialized teams and a separate
area for the drillers, engineers, and geologists to work, away from the press and the
miners’ families. Independent teams that had started drilling already, but were not
coordinating their efforts, were asked to leave. It was very clear “who was in” and “who
was out.”

Sougarret quickly determined, for instance, that Rene Aguilar, a psychologist and HR
expert, would be best suited to lead all non-technical aspects of the rescue, such as
managing communications with the press and families, as well as managing the
emotions of the men working under such immense pressures.

Video

ANDRÉ SOUGARRET: Well, René Aguilar, he was Safety and Health Manager at El
Teniente, and therefore we worked together. We knew each other very well, and we both
made a smaller team which allowed me to coordinate with him. He is a psychologist by
profession, and thus this also permitted me to see things that, as an engineer, I was not
able to see. A bit more beyond the technical issues, and see the matter of the
relationships that existed both within the technical group, as well as the relationships
with the different groups: family, government, press, etc.

Q: What would others say are your top three strengths?

VP: Trustworthiness, dedication and respectfulness

Hendrina: - nonjudgmental: make no distinction in any way (work experience,

knowledge, role, et cetera)
- learnable: flexible, willing to learn, open to input of others
- cooperation: eager to tap into and make use of the knowledge and experience of
others
Faith: I think others would say I am a great listener, I am considerate of other people's
opinions and thoughts and I have great follow-through and get things done.
Allen: • Leadership and inspiration
• Technical problem-solving
• Communication

With roles and responsibilities laid out, Sougarret and the rescue team were able to
establish forums to discuss relationships and ensure effective communication and
collaboration. To manage internal communication amongst the search team onsite, the
group developed an operating routine. At 9 a.m. there was a daily coordination meeting
with the heads of all drilling and geology teams. Sougarret and others reviewed
commitments from the day before, discussed next steps, and identified needed
resources. To manage external communication, Golborne and Aguilar set up a twice-
daily meeting to provide the families and the press with updates on progress. Critical, in
their view, was a consistent message to all stakeholders. The atmosphere was one of
openness and transparency.
The difference between these two cases makes clear a number of predictable problems
that can occur in determining roles, responsibilities, and relationships. 

 Don’t cast the wrong roles or assign the wrong parts. Basing decisions on
political expediency or personal interests rather than ability to do the job is a
recipe for resentment and disaster. As Jim Collins notes, “[great companies] start
by getting the right people on the bus, the wrong people off the bus, and the right
people in the right seats.”
 Clearly outline responsibilities, accountabilities, and decision rights. Make
sure these are crystal clear to prevent trouble down the line.
 Don’t mistake responsibility for accountability. Many people may be
responsible for working on a deliverable, but only one person should be
accountable for its delivery. Often, the more people are responsible for
something, the less accountable each may feel and the less chance the project
will succeed. 
 Don’t forget to appoint a single authority in charge. Without someone in
charge and able to hold others accountable, assign decision rights, and make the
tough calls, projects often devolve into turf wars or stasis. 
 Don’t fail to lay out relationships up front. Managers must lay out the rules of
the road early. Specify reporting relationships, dependencies, and communication
channels. Make clear how peoples’ roles and responsibilities affect one another
and what’s expected of people daily. Otherwise, people will be confused about
who’s who, who’s in charge of what, and how they are expected to work together.

3.3.3 Delegate the Work

This stage can be tricky. You may be asking, “Isn’t delegating the same as laying out
roles and responsibilities?” No, not quite. Determining what team members will be
responsible for is not the same as actually assigning deliverables to people
and officially handing off responsibility for those deliverables. In fact, the confusion
between stages two and three is why many implementations stumble.

Note, however, that to “delegate” does not mean to “dump.” It’s a job in itself, a
multidimensional process that involves not only the transfer of responsibility, but also the
transfer of key resources, a team to work with, authority and decision rights, crucial
background knowledge, organizational context, necessary contacts, and the promise of
continued support. A proper “handoff” doesn’t mean keeping your hands off afterward; in
fact, it requires you to stay engaged throughout the execution. (That seems to be a
pattern, doesn’t it?)

Some key questions for managers to consider during the delegation stage:
  Which deliverables should I delegate and to whom?
 How confident am I that this person can deliver?
 What is this person’s current workload and potential bandwidth?
 Have I explained both the details of this deliverable and the organizational
backdrop?
 Have I put the essentials in place (resources, people, decision rights, etc.) for the
person in this role to succeed?
 Have I clearly bounded the task and made clear what’s expected at the end?
 Does the person understand how this deliverable connects to the broader goals?
 Do others know that this person is now responsible for this deliverable?
 Have I publicly communicated the new set of responsibilities?
 Have I established a timeline for check-ins so I can keep up to date on progress
and performance and provide additional information and support?

Healthcare.gov
Having poorly defined deliverables and misjudged roles, it’s no surprise that
Healthcare.gov suffered from bad handoffs at many levels.

White House policymakers failed to pass off responsibility for deliverables in a way that
set up CMS technical staff and contractors to succeed. Right from the start, Obama and
his team provided overly broad guidance for their vision of healthcare, leaving CMS to
figure it out. Even if CMS had been up to the task, that approach would have courted
failure.

During the project, deliverables were constantly evolving. For example, the ACA
required the government to issue regulations on how the federal exchange would work.
CMS would then turn these regulations into “business requirements” to guide developers
in designing a compliant website. Unfortunately, the Obama administration dragged its
feet on finalizing these regulations, handing them off late and without much guidance.
Even worse, White House policymakers kept revising requirements and meddling on the
ground, even after developers had begun coding. 

Q: Looking back at the list of questions presented at the top of this page, which
questions do you feel the Obama administration failed to adequately investigate?

VP: All of them.

Faith: The Obama administration failed to adequately investigate all of these questions.
Having already failed at stage one and stage two of implementation, stage three could
never had succeeded. The work was simply dumped not properly delegated to the
correct people with check-ins set up to troubleshoot through the process. There was far
too much assumption in who would get what done instead of action to assure that the
work would be done.
Allen: Once again, none of the questions seem to have been properly answered during
the Healthcare.gov implementation. And the fact that requirements kept changing
throughout only led to further challenges for developers.

With little guidance or support from the top, key handoffs were missed within CMS too.
The CMS procurement office ultimately selected CGI Federal, a Canadian company with
a large presence in Washington, as the lead contractor for building the federal
exchange. It was not at all clear CGI had the skills for the job, but in any case, CMS did
not make things easier for them. 

CGI reported that the deliverables assigned by CMS were not well specified and, in most
cases, overly broad. “Clarifications” from contract managers were confusing and
sometimes conflicting. Asking two different offices at CMS might result in three different
answers. But with White House and CMS policy staff making key regulatory and policy
decisions during the build, CMS technical staff often did not know exactly what they were
handing off. 

Chilean Mining Rescue

In contrast, the Chilean mining rescue team experienced clean, clear handoffs of
responsibility down the chain of command and laterally between teams and shifts.

President Pinera asked Laurence Golborne, the Minister of Mining, to take time off to
work closely with Sougarret to support the technical work on the ground. Sougarret and
Golborne spoke every day by email and once every three days in person. Sougarret also
had a direct line to Pinera, who wanted regular updates and would mobilize his network
of contacts as needed.

Here’s Sougarret to describe Golborne’s role and the importance of having as many
resources as possible at his fingertips:

ANDRÉ SOUGARRET: Laurence Golborn was the Mining Minister of the government at
the time. He was asked by the President to conduct the efforts to bring the miners back
to the surface. He was not a mining technician. We created a work team in which we
reported the possibilities and alternatives we had to be able to have options for the
rescue.

In addition, he allowed for us to get a spokesperson to deal with public opinion and to
count with other resources that we wouldn't have had in our reach otherwise. Obviously,
he was supportive in providing us with the resources necessary to start the rescue. He
also knew that it was a complex situation. However, he encouraged us, the team and me
in particular, to put in all the effort required to start the rescue. As regards resources, he
always made it clear that everything was available to us.

Members of the rescue team handed off responsibilities well, too. To manage the drilling
efforts, Sougarret appointed Walter Veliz as the overall head of drilling. Sougarret made
clear that Veliz had direct access to him whenever he needed it and full decision-making
authority over drilling decisions. At the next level, Veliz delegated responsibilities for
drilling operations and data analysis to Nicolas Cruz and for ground operations and
logistics to Marcos Bermudez.

This small team developed a protocol for transitioning between day and night drill shifts
and for routine maintenance of machinery. Drills worked around the clock, stopping daily
at 8 a.m. and 8 p.m. for maintenance. Between shifts, workers updated each other on
progress and problems and cooperated to find solutions. 

Note that even though he was near the top, Sougarret spent time in the field, listening to
concerns, asking questions, and answering queries. Without micro-managing, he
provided guidance to his men while generally letting experts make the calls.  

The difference between these two cases demonstrates a number of predictable

problems for managers to avoid when delegating work: 

 Don’t fumble the handoff! Don’t rush to pass off responsibilities prematurely.

Wait until the person is fully ready, knows the job, understands the context, and
seems set up to succeed.  And make sure others know who has the ball. On the
flipside, once you make the handoff, let go. Let people do their jobs, while making
yourself available for questions, clarification, and guidance.
 Don’t neglect to put the essentials in place. Don’t delegate responsibilities that
a person, team, or department couldn’t possibly accomplish because they lack
the necessary resources, people, decision rights, authority, context, training, or
support. That might mean you have to do some work up front.
 Don’t delegate work to someone in whom you lack confidence or trust. Only
bad things will come of this. Second-guessing is inevitable.
 Don’t delegate work to someone who doesn’t have the bandwidth. Overload
seldom leads to success.

Here are some further perspectives and tips on delegation from our faculty and
management experts:

Video

3.3.4 Execute the Plan, Monitor

Progress and Performance, and
Provide Continued Support
For managers, the execution stage is by far the most work. One person who knows why
is former US Secretary of Defense Donald Rumsfeld:

“There are known knowns. These are things we know that we know. There are known
unknowns. That is to say, there are things that we know we don't know. But there are
also unknown unknowns. There are things we don't know we don't know.”

For Healthcare.gov, one known unknown was that Republican governors might refuse to
build their own state-level exchanges, dramatically increasing the demand on the federal
exchange. Acknowledging this known unknown would have fundamentally changed the
scale and scope of the deliverables.

The only way to deal with unknowns—both the known and the unknown varieties—is
to monitor progress and performance. No one can see what’s over the horizon, but
there’s no excuse for failing to see something once it comes into view. Unless you are
managing an extremely routine project, you should expect to find out—through
monitoring—that you need to change course.  All the while, managers should provide
continued support to their teams to keep the plan on track as much as possible. 

In executing the plan, a manager should consider the following questions:

 What is the nature of this work? (Is it relatively routine, somewhat complex, or
completely new?) What, then, should be my management style be?
 How should I treat failure and experimentation? Is it to be discouraged or
encouraged?
 How much information do we have about deliverables or requirements going into
execution? (Are the deliverables clear? Or must we plan to articulate them more
clearly as we go?)
 Do we have a system for prioritizing demands as the project progresses?
 Is essential communication, coordination, and collaboration occurring?
 How can we build in feedback loops so that we are testing our assumptions and
getting helpful information as we go?

In monitoring progress and performance, a manager should consider the following

questions:

 Have I put in place feedback mechanisms (one-on-ones, standups, status

reports, etc.) for team members and me to see if the plan is tracking?
 Is the project proceeding according to the plan we put in place? If not, is there a
legitimate reason?
 Are our due dates and milestones still realistic based on this new information?
 How can I make people feel comfortable speaking up so that problems are
surfaced early?
 How can I seek out information that would be helpful for my team?
  Is this person delivering? If not, why not? What, if anything, can I do to help him
or her see the problem?

In providing continued support, a manager should consider the following questions:

 Am I giving as much to my employees as I’m receiving from them? If not, how

can I be more helpful (even if that means getting out of their way)?
 How can I provide regular emotional or motivational support to this person?
 Am I providing honest performance feedback to this person?
 Am I hoarding information or failing to communicate something that people need
to get their work done?

Healthcare.gov
Having conducted the first three stages of implementation poorly, those on the
Healthcare.gov project unsurprisingly experienced many problems executing the build of
the website. But these problems were aggravated by poor monitoring and lack of high-
level support.

Because the proposed insurance regulations that would affect the exchange kept
changing, the “business requirements” that developers needed weren’t finalized until
February 2013, just eight months before the planned launch. As could be expected,
rushed contractors made many errors and various pieces of the website (built by
separate contractors) failed to integrate. Policy delays were compounded, however, by
CMS’ own delays in communicating requirements to contractors and by contractors’ own
errors and delays in actually coding the exchange. 

As the project continued to slip throughout 2013, CMS had to make a major decision:
speak up and ask for an extension, or cut out crucial time for user testing in order to let
the coders finish the work. Given the complexity of the work and the very public nature of
the product, user testing would be crucial. But CMS went with sacrificing it to meet the
deadline. The government eventually got its feedback—in the form of an historic flop.

Q: In the face of potential catastrophe, many government and business leaders still
move forward, adhering to a deadline despite the dangers. What do you think causes
this pressure? 

VP: Commitments to announced goals, promises and deadlines, fear of public failure
and loss of confidence, hope that it will somehow work out.
Faith: I believe in government and politics there is great pressure created by the need to
succeed, especially being in the public eye. The option of failure does not exist. This
means adhering to a deadline is more important than delivering passed the deadline
even if there is a chance that what is delivered is not perfect. Otherwise it is a broken
promise.
Hendrina: I think the following might cause this pressure:
- budget and deadlines;
- expectations from the stakeholders; 
- expectations from society; 
- sticking to a particular policy and reputation.

Presuming that CMS would get it right the first time, Obama and his team did not closely
monitor its progress—or lack of it. Had they done so, they might have known they were
headed for failure:

 Presentation by McKinsey: In March 2013, the consulting firm McKinsey &

Company delivered a review of the website's progress, noting “evolving
requirements” and “multiple definitions of success,” “significant dependency on
third parties and contractors,” “insufficient time and scope for end-to-end testing,”
and other risky management behaviors. To conclude, McKinsey put forth 13
major potential risks to the project, including “failure to resolve post-launch issues
rapidly” and “long manual processing times.” The biggest risk was “marketplaces
unavailable with system failure” due to lack of testing. They added that CMS
needed “specific support from HHS and the White House” to get the job done and
recommended appointing “a single Chief Operating Officer [to] manage the
process from the top down, and start making decisions to enable developers to
finish their jobs.” They recommended a delay if the White House would consider
one.
 TurningPoint’s technical reports: CMS had also hired TurningPoint Global
Solutions to pressure-test the software. A congressional inquiry found that “over
the course of 12 months—from September 2012 to September 2013—
TurningPoint identified technical and managerial concerns that ultimately were
key factors in the failure of the website.” In fact, the concerns only grew as the
project continued. In August 2013, when the site was still only 62% complete,
TurningPoint reported deficiencies in all aspects of the exchange. Shockingly, the
number of defects actually increased as developers worked more. By early
September, their analysis showed that of the 355,000 lines of code written for the
website, 21,000 of them contained defects, 677 of which TurningPoint said were
“serious.” Yet, CMS had “no strategy or contingency plan for dealing with critical
defects in the system after launch.”
 Concerns from CMS staff: Several CMS staff raised serious concerns. Here’s
what Jeff Grant said in an email to his superiors in July 2013: “[T]he build
appears to be way off track and getting worse. We also finally were told that there
are only 10 developers total working on the build for all functionality. Only one of
these developers is at a high enough skill level to handle complex issue
resolution, which now appears to be required for all aspects of our build…So
while OIS has always said that we had an independent team for development,
they have never revealed the serious substandard level of staffing this team has.
We believe our entire build is in jeopardy.”
 Yet these warnings seem not to have been communicated to those with authority
to make changes. For example, Henry Chao, the head of the CMS technical
 office, testified that neither he nor the lead officials from CGI ever saw the
McKinsey report. Lambrew called the report “typical second guessing” by
consultants. Similarly, neither the TurningPoint reports nor concerns from CMS
staff were ever brought to the attention of key decision-makers.

Q: We’ve talked previously about the reasons managers might ignore warning signs.
What do you think was the cause in this case?

VP: Lack of leadership, inadequate planning, inadequate outsourcing the various

stages of development, lack of clarity of tasks, coordination and control, lack of
competencies, too many unknown about the methodology and ways of organizing
and implementing goals.

Hendrina: Holding on tight to a deadline, not wanting to lose face in that respect. I think
they considered it would be time-consuming to walk the regular paths of decision-
making, asking a certain layer of people for their perspectives. So they just skipped that
layer of key decision-makers.
Faith: I believe at this point they thought they were too far in to do anything. They were
overtaken by confirmation bias and the hope that things would somehow still work out.
There was enormous pressure that this system succeed and the public and Obama was
relying on it. The fact that no proper process had been put in place to assure this could
be achieved further put pressure on those involved.
Allen: It seems likely that with such high stakes for the project's success, no one wanted
to sound the alarm and risk being blamed for delaying the project. In this respect, it's
similar to the Columbia case study. It also appears that there was no clear escalation
path for critical issues. The report from McKinsey was created and identified major
problems, but the findings were not provided to the highest levels of government so they
could take corrective action. This was likely due to the failure of Stage 2: Determine
Roles and Responsibilities. With no one clearly accountable, and multiple people
thinking someone else was in charge, even if there had been a process defined for
escalations, HHS would have struggled with whom should receive the information in the
report.

Video

Chilean Mining Rescue

In contrast, the Chilean mining rescue team effectively executed, monitored progress
and performance, and provided each other with support.

Just listen to how different their model of execution sounds:

Video
ANDRÉ SOUGARRET: Well, what we did there, as it's described, was to form several
groups. One group was in charge of planning and reviewing what was being done with
the drills. And a second group that reviewed the ground, to make sure that we were, or
kind of, that we were actually doing what we were planning. Of course we checked every
day the various drills and the problems they were having. In the afternoon, as the day
passed, we reviewed and measured which drills were advancing faster, and those that
were deviating too much. And that we had to abort.

So as to be able to start another drilling without wasting time. We had to define expert
groups in charge of, on the one hand, the planning, the realization and the evaluation.
And, on that basis, we took measures daily. The important thing was to ask daily what
complications arose, and to share that with the different teams. I think that was a good
way to tackle an problem that was unknown and had too many elements.

But, in the end, we sorted it out together with everyone working in the rescue. As none of
us there had all the answers, something we all did to keep the job on track was to share
the information. Everyone, with their own opinion, with different information, we ended
up working in a way that was joint and collaborative. No one could say what should be
done from beginning to end, but they all contributed a part of the final solution.

Unlike the Healthcare.gov team, the Chilean rescue team were not counting on getting it
right the first time. They knew that wasn’t going to happen and that they would have to
keep trying, failing, and trying something else. The trick was to spend as little time as
possible figuring out what wasn’t working. For example, Sougarret designated a team of
experts to closely monitor the work. If a drill was going off course, the experts would see
that immediately and make adjustments. They didn’t have to drill for hours before
somebody realized they had made an error. Also, the rescue team always had
something new to try when the latest effort didn’t work. They didn’t lose time going back
to the drawing board because—knowing that there would be many iterations of failure—
someone was always back at the drawing board.

Here’s Sougarret again:

Video

ANDRÉ SOUGARRET: As we were uncertain and there were many risks in every one
one of the tasks we were trying to accomplish, we always had sort of an initial plan, but
looked for alternatives, in case it failed. And we didn't wait for it to fail before we acted.
That's why we would go one way or another.

However, we would evaluate alternatives, which came from many ideas present at the
time, but in the end we stuck with two or three, so we could not only make progress in
one but also have the next one as an alternative.

But basically, what we always had in mind was that it could fail. What we were doing
could fail. We needed to have always have an alternative. And that's how we addressed
every activity. We never had one way to do it but two or three alternatives.

Note that team members communicated openly and regularly with each other and with
the public and other stakeholders. There were no siloes. When a problem arose, the
group discussed how to solve it, not who had caused it. The psychological safety of the
group, exemplified by Sougarret’s own conduct, made this possible. 
Finally, note that members of the rescue team provided each other with both work and
emotional support. When people had questions or suggestions, Sougarret and Aguilar
were always willing to listen. This meant that problems weren’t hidden, which would
have been fatal in these circumstances.

The difference between these two cases demonstrates a number of predictable

problems for managers to avoid when executing, monitoring, and providing support:

 Don’t presume that you’ll get it right the first time. Expect that it will take
multiple (ideally, a few, but possibly many) iterations to get it right.
 Don’t put all your eggs in one basket. Options are essential, especially when
you face many uncertainties. If you have only a single course of action and it
fails, then what?
 Don’t turn a blind eye to emerging problems that suggest the project will soon
go off track—or has already. Set up early warning systems such as check-ins,
status reports, standups, and debriefings. 
 Don’t forget to build in feedback loops. And keep the feedback cycles as short
as possible, especially for feedback from your customers.
 Don’t be afraid to give and receive critical feedback or bad news. Fear of
criticism or failure makes failure more likely. Of course, this requires a context of
psychological safety.
 Don’t monitor progress and performance without providing support. Have
an “open-door” policy. Provide at least as much value to your people as they
provide you.

3.3.5 Take Corrective Action (Adjust

or Revise)

This is the stage that typically separates effective from ineffective implementation. No
matter how well you plan, something changes. Can you get the job done anyway?

Good corrective action requires managers to both anticipate problems before they occur

and to respond quickly to problems and rebound after they occur. Not all managers are
intuitive. Even fewer are resilient. That’s not fatal in itself, but it means they need
systems and processes.

In determining whether minor adjustments is necessary, a manager should consider

the following questions:

 What’s the problem? Why isn’t our current approach working?

  Which of the four previous stages needs to be addressed? (Do deliverables need
to be re-defined? Is a new role necessary? Was a handoff missed? Are the
deadlines unrealistic?)
 What are some small changes I can make to keep the plan on track? (Can I
provide more resources? Do roadblocks need to be removed?)
 Will the odds of success increase if I make this adjustment?
 What do other people think needs to be done?

In determining whether a major revision is necessary, a manager should consider

the following questions:

 Is this course of action salvageable at all? Or should we cut our losses and try
another approach?
 Were stages 1-4 done so poorly that it’s easier to just start over?
 What did we learn from this iteration that can be used next time?
 What should we not do next time?

Healthcare.gov
CMS received 18 “documented warnings” between July 2011 and July 2013, yet its
responses were too little and far too late. 

For example, after receiving criticism that the project lacked clear leadership, CMS
assigned its newly appointed COO, Michelle Snyder, to head the Healthcare.gov
program. However, this was not formally announced, her responsibilities were never
made clear, and, as COO, she was still accountable for many other projects. Staff
indicated that they “sort of knew” she was in charge, but weren’t completely sure. As a
result, her impact was minimal.

Despite the delays, CMS leadership failed to increase project staffing until the very end.
As of summer 2013, the project was months behind schedule, yet there were still
relatively few people working on it. One of the reasons was that CMS staff were averse
to bringing up concerns to their leadership, whom they believed would eat up their time
without providing much support. 

In short, officials made small, ineffective, late adjustments to a plan that had already
gone off track.

Chilean Mining Rescue

In contrast, the Chilean mining rescue team made continuous adjustments throughout
the implementation process.
The drill team used two novel technologies—vertical drilling and 3-D mapping—that
allowed them to receive real-time feedback on the direction of their holes. Of course,
workers found it frustrating to be informed over and over that what they were doing
wasn’t working, but this let them keep making adjustments until they located the miners.

The strategy for getting the miners out was also a continual work in progress. Ideas were
evaluated based on availability of materials, cost, service schedule, maintenance,
logistics, and—first and foremost—how long they would take. The team would eventually
settle on three options—Plans A, B, and C—cycling through all three to get the miners
out.

The difference between these two cases demonstrates some predictable problems for
managers to avoid when considering taking corrective action:

 Don’t hide problems and let them compound. Communicate them early so

that they can be solved while they’re still relatively small. Remember: most
problems don’t age well.
 Don’t put off small adjustments. Make them every day to keep the plan on
track. 
 Don’t make minor adjustments when major revisions are needed. If you feel
the plan going extremely off track, strive to cut your losses and start over.
 Don’t point fingers when things go off course. Most problems can be traced
to systemic or process flaws, not individual mistakes or bad apples. Intervene by
focusing on the problem and potential solutions.

3.3.6 Get Closure on the Project and

Agreement on the Output

After many iterations of implementation, the team or organization will have delivered
something. But is it what was agreed upon back in stage one? 

That ought to be obvious, but often it’s not. Even if the original goals and deliverables
were clearly defined, people can have very different perceptions of what finally got done.
Some stakeholders may have not stayed fully engaged during the process and may
have built up their own idiosyncratic (and, at times, self-serving) interpretations of the
outcome.  
Thus, it’s crucial that managers close out the project. This means gaining alignment on
what was actually produced so that there are no unresolved conflicts or dueling
interpretations. If the project was, in fact, a success, the manager should publicly
acknowledge team members, present the output far and wide, and reward employees for
their performance if appropriate. If the project was not a success, the team will have to
take a hard look at whether its goals were unrealistic, its plan was overly optimistic or
poorly managed, or its people were underperforming.

In getting closure and agreement, a manager should consider the following questions:

 What does “done” look like? Are we there yet? What, if anything, remains to be
accomplished?
 Are people on the same page as to whether the project was a success?
 Have I kept key stakeholders in the loop so that they are not surprised by the
output?
 How should I recognize my people for their contributions?
 How should we publicize this project? 

Healthcare.gov
Leading up to the launch of Healthcare.gov in October 2013, the CMS team grew
increasingly anxious about what would happen. No one knew how many errors the site
contained or whether it would work as planned. Henry Chao emailed his colleagues that
he was concerned that CGI, the lead contractor, would “crash the plane at takeoff.” 

Yet right up to the day before the launch, the White House was continuing to receive
positive news. Dennis McDonough, Obama’s Chief of Staff, was quoted as saying:
“Based on the reports I’m getting, I think we’re going to knock your socks off tomorrow.” 

Here’s Schlesinger to describe the lack of alignment the day before the launch and what
happened on October 1st:

Video

CMS blamed CGI for its shoddy work, while CGI blamed CMS for lack of support and
unrealistic requests. Meanwhile, Obama gathered his senior health care team in the
Oval Office to find out what had gone wrong. But neither Lambrew, Tavenner, nor
Sebelius really knew what had gone wrong, whether the situation was improving, or how
it could be fixed. They had never been close enough to the implementation to know such
things.

Q: How important do you think it was to determine what went wrong?

VP: Project managers do not know what they have done to be able to determine what
went wrong, so in order not to repeat such a catastrophe, it is very important to analyze
the deficits that led to the extreme failure of the project.

Hendrina: Because then it would be clear what could be the points of improvement. If
you have become wise through trial and error, you'd better learn from it. Also: it's a good
lesson in responsibility and accontability.
Faith: I think its incredibly important to determine what went wrong. Clearly, there was
failure from the beginning in establishing a process of implementation of this program
and this needed to be identified now that everyone had seen the failure. There was no
ability to ignore it at this point as it was publicly shown as a failure. To prevent this from
happening again and be able to potentially fix the problem they needed to identify ALL
the things that went wrong. They could not let this happen again and repeat these errors!

Chilean Mining Rescue

As the last Chilean miner was lifted out of the mine, there wasn’t any question that the
goal had been met and the deliverable delivered. 

This beautiful moment was the result of thoughtful, disciplined management. Everyone
was aligned because the team had invested enormous time in communicating, in
coordinating, and in updating stakeholders daily throughout the project.

Here’s Sougarret on the importance of regular, transparent communication with

stakeholders:

Video

ANDRÉ SOUGARRET: What we did was to always be consistent with the strategy we
had. The idea was to be able to inform the different entities or those interested in this
rescue, tell this story over and over again in different languages. In such a way as to
eliminate anxiety, on one side, in the first stage, to know what was going on with the
miners, and in the second, when we were going to rescue them. I believe it was
important, first, to generate credibility in the work we were doing.

For this, what we did first was inform the relatives on the progress of the various jobs
and, after that, we went to the press. Especially because there was a strong media
pressure to know what was going on, how we were going to solve the problem. What we
always did was to tell exactly the same story to the relatives and to the press, baring in
mind that we needed to use a different language.

We explained the relatives, using a simple language, what we were doing. And as for the
press, we gave them facts about what was happening day by day. Basically, what we did
was try to understand what expectations they had, what it was they needed in terms of
communication and deliver that information in a timely manner and using a language
that made sense to them.

On the flipside, key stakeholders—the President, the government, companies, families,

the press, and others—made it a point to stay involved throughout the process and offer
whatever support they could. After the rescue, the President acknowledged the bravery
of the miners and immediately provided them with health and mental services. He also
commended the rescue team for its ingenuity and persistence, gave individual rewards,
and thanked them for their tremendous work. 

Q: As a project comes to an end, what steps do you take to make sure everyone is in
alignment and there are no surprises? This is a shared reflection, so you may wish to be
vague on the details.

VP: We discuss and report the result, compare the extent to which it has deviated from
the planned, discuss whether the result corresponds to the goal, form good practices
based on experience.

Hendrina: To get updates and feedback from each and every team involved; to double-
check whether goals, budgets, and deadlines are being met. Doing this in both an
understandable and timely manner, so there wouldn't be any misinterpretation.
Faith: At the end of project to make sure everyone is on the same page we have
debriefs on the project. We communicate openly how we felt the project went and what
were the successes and failures. By having psychological safety we are able to learn
things from each other that we may have not even considered on our own. This ensures
there is a common sentiment to how the project did.
Allen: Thankfully, our company is very metrics-driven and we have several reporting
mechanisms and dashboards to provide ongoing status updates for projects and other
activities. Almost anyone at the company can check on the status of a project and can
see if something is on track or going poorly.  

When projects are successful, we recognize team members through various ways:
• #kudos Slack channel
• At Our Best awards
• Emails or Slack messages to the individual and their manager
• Special Awards at quarterly all-hands meetings

When projects are unsuccessful, we draft and review post-mortems and aggregate the
lessons learned to identify common themes or trends in those failures.

On my team in particular, I encourage team members to share a "Win of the Week" and
a "Flawless Fail". This allows team members to celebrate each others successes, but
also to learn from mistakes. We applaud the failures as much as the successes to
continually build psychological safety and camaraderie amongst the team.

The difference between these two cases demonstrates some predictable problems for
managers to avoid when closing out:

 Don’t forget to wrap things up. Make sure everyone agrees on what was
promised, produced, and delivered. Test people’s perceptions of whether they
believe implementation was effective and make sure everyone is aligned on what
was actually accomplished and whether or to what extent it fulfilled the original
goal.
 Don’t keep stakeholders in the dark until the end. Provide regular updates
throughout the project. That way, even poor results will only be disappointing
rather than feeling like a betrayal.
 Don’t forget to acknowledge everyone who chipped in and contributed,
however small the role. Implementation is a team effort.

3.3.7 Conduct a Retrospective

Review

Unlike a baseball game, it ain’t over even when it’s over. A thorough review is always
necessary, especially if the group—or some other group—will be doing something
similar in the future. Yet this is the stage likeliest to be skipped. 

One reason is that most managers are rewarded for action and efficiency, not
thoughtfulness and consideration. 

Second, if things went wrong, hashing them out can be painful and contentious. And if
the project succeeded, people may not see much point in a thorough review.

Yet in the long run, reflection is not a waste of time. It saves time if it prevents teams
from repeating mistakes. It can also save time by promoting small improvements and
occasionally a breakthrough innovation. Failures can be very productive if they teach the
team a valuable lesson. As the philosopher Kierkegaard put it, “Life is lived forward, but
understood backward.”

In short, the value of successes and failures depends on how much a team learns from
them. But that takes discipline and that’s why it’s part of the implementation process. 
In conducting a review, a manager should consider the following questions:

 What did we set out to do? What were our stated goals and targets?
 What happened? Why?
 What should we keep doing/sustain?
 What should we quit doing, improve upon, or change?
 What is our action plan to actually make improvements?
 How should we document these lessons so that we and others in the
organization can best access them in the future?
 How else should we share our lessons with the organization?
 How well did I manage the implementation process?

Healthcare.gov
In the aftermath of Healthcare.gov’s crash, no formal review was conducted by the
project managers. Instead, President Obama decided that the project needed new
leadership. 

He asked US CTO Todd Park to devote 100% of his time to fixing the website and
paired him with Jeff Zients, the incoming head of the National Economic Council, who
had a private sector background and experience fixing government programs. The duo
was given unlimited resources for the fix. 

Immediately, Park and Zients began assembling a team (which would become known as
the “tech surge”) to help diagnose the problems and review the work that had been
done. Park called Mikey Dickerson, a Google executive with expertise in site reliability,
to assist and eventually lead the fix of the site on the ground. The group visited
contractors’ offices to determine whether the site was salvageable or whether they would
have to start over.

Q: What changes to implementation do you think this group might make for “the Fix”?

VP: Assembling competent team, analyse and diagnose the "product", set a plan,
consider the options, choose the way of acting, execute their plan.

Hendrina: For starters: set a SMART goal; be sure to have neatly outlined roles and
responsibilities. From that point, it is easier to delegate the work. It'd be good to keep
monitoring progress and supplying guidance along the way while executing the plan. If
necessary: adjusting and revising. So altogether, I think the whole implementation
process should be turned upside down.
Faith: First and most importantly, this group must establish a clearly defined SMART
goal. If they cannot do this then they will truly fail again. The failed implementation to
create this website the first time did not have this, instead it was just assumed things
would happen. Without a clear goal and set deliverables the process of implementation
cannot succeed. From there they can identify or find the correct people and establish
their roles, responsibilities and relationships and have work properly delegated to them.
Execution, corrective action and closure must also be put in place. And finally there must
be another review at the end when they believe they completed implementation.
Essentially for "the Fix" the process of implementation must actually take place.
Allen: Immediately establish clear answers to some of the stages discussed in this
module:
• Who is in charge and accountable for "the Fix"?
• What does success look like?
• What specific milestones or deliverables are needed on the path to success?
• What does a successful Healthcare.gov website actually entail?

I would expect that clear channels of communication were created by Park and Zients,
including ongoing monitoring of progress of all work delegated to teams working on "the
Fix".

Chilean Mining Rescue

After accomplishing the seemingly impossible, the Chilean mining rescue team took a
much needed break. They did not conduct a formal retrospective. But business leaders
and academics flocked to Chile to try to understand how the group had pulled it off and
to uncover some of the critical factors behind their success.

Here’s Prof. Edmondson with some final takeaways:

VIDEO

The US Army’s After-action Reviews (AARs)

To help you in conducting your own reviews, we spoke with one of the few organizations
that has institutionalized them in its implementation process: the US Army. 

Since the 1970s, after-action reviews (AARs) have been standard army procedure.
Immediately after a mission, event, or activity, all participants meet to review their
assignments, identify successes and failures, and look for ways to improve. The process
may be formal or informal, involve small or large groups, and last for a few minutes or a
few days. But discussion always revolves around the same four questions:

1. What did we set out to do?

2. What actually happened?
 3. Why did it happen?
4. What are we going to do next time? In particular:
1. What will we sustain/continue to do?
2. What will we improve/change?

Here’s Colonel Paul Reese, former Director of the Center for Army Lessons Learned
(CALL), on the AARs:

Video

It’s time to conduct your own AAR. Consider your description of a failed implementation
project from earlier in this module.

VP: As a project selection consultant for financing, I had a case in which in the almost final phase of obtaining approval for
an interim loan, the client did not present a breakdown of its debt at book value due to inefficiency and most likely
reluctance on the part of management.
SUBMITTED ON APRIL 17, 2022 04:59 AM

1. What did you set out to do?

2. What actually happened?
3. Why did it happen?
4. What are you going to do next time? In particular:
1. What will you sustain/continue to do?
2. What will you improve/change?

VP: What did we do? - Study for financing a significant project

What actually happened? - In the final phase we did not receive an important part of the
documents

Why did it happen? - Lack of documents or unwillingness to provide them

What will we do next?

In particular:

What will we maintain / continue to do? - Fixing more specific mandatory conditions for
conducting an evaluation process

What will we improve / change? - The contractual framework

Notice how the Army conducts reviews for all events and activities—not only failures, but
also successes. Although the four questions seem simple enough, conducting such a
review is a skill. Some may have it naturally, but generally—like all our other
management skills—it needs to be learned and practiced. 
Here’s Col. Reese on some keys to running effective AARs so that the organization can
improve implementation:

Video

You’ll hear more from Col. Reese in the next lesson, as we look at how to build AARs
and other learning processes into the fabric of an organization.

For now, the two cases and Col. Reese’s commentary demonstrate some predictable
problems for managers to avoid when conducting a review of the implementation
process:

 Don’t forget to conduct a review of the implementation process. For

organizations that expect to learn and improve, reviewing past projects is crucial.
So this needs to be formalized into your implementation process. Remember: if
you haven’t conducted a review, the project’s not through.
 Don’t neglect to clarify what you set out to do and what actually
happened. That is, don’t just jump into a discussion of what can be improved.
That will only create confusion if you haven’t agreed on how well you fulfilled the
original goals. In fact, teams can learn a lot when there are initial disagreements
on what you set out to do and what actually happened.
 Don’t neglect to discuss what went right and what should be
sustained. AARs should not only focus on what the team did wrong, but also on
what people did right, so that those things can be continued. It’s not always
obvious what went right; you may know that you achieved your goal, but
misunderstand how it happened.
 Don’t forget to document lessons learned for you and others to reference.
Make it clear where lessons will be documented and how they can be accessed.
 Don’t forget to discuss how you might incorporate these lessons into future
projects.That is, don’t just talk about the things you’ll do differently next time.
Create an action plan so that you’ll actually do them.
 Don’t neglect to provide clear and direct feedback on what people did well
and what they could improve upon. Then coach them on how to make those
particular improvements. Often, this is best done privately.
 Don’t waste the opportunity to get feedback on your own
performance. Conduct your own personal review of your performance so that
you can become a better manager.

3.3.8 Learning from the Crash

One of the consequences of the failed Healthcare.gov implementation (and its

subsequent fix) was the establishment of the US Digital Service. 
Among other things, the USDS has worked to create standards for how technology
projects should be managed across government. Although its “Digital Services
Playbook” is intended for software and technology projects, a number of the “plays” are
simply the elements of effective management. Here is the full list:

DIGITAL SERVICE PLAYS

1. Understand what people need
2. Address the whole experience,
from start to finish
3. Make it simple and intuitive
4. Build the service using agile and
iterative practices
5. Structure budgets and contracts to
support delivery
6. Assign one leader and hold that
person accountable
7. Bring in experienced teams
8. Choose a modern technology stack
9. Deploy in a flexible hosting
environment
10. Automate testing and deployments
 11. Manage security and privacy
through reusable processes
12. Use data to drive decisions
13. Default to open
Q: Based on this list, what might be some other management “plays” that you could
run? 

VP: Monitoring, evaluation, and feedback.

https://safety.fhwa.dot.gov/shsp/fhwasa10024/chapter1.cfm

Hendrina: - making good use of the DA/DI method when it's suitable, to be sure nothing
is being overseen;
- creating transparency around the different roles, responsibilities, and accountabilities:
people should always know where to go in case of questions, input, and concerns.
Faith: - Ensure psychological safety is experienced throughout the entire organization
- Understand the role of leader and manager 
- Reflect on completed projects in order to learn for the future 
- Always check-in 
- Ask questions versus assuming
Allen: • Require open and honest feedback
• Define clear escalation channels and forms of communication
• Acknowledge and encourage when team members raise concerns, even if they turn
out to not be critical issues
• Ensure everyone who needs to be involved on a project is included in communications
and has visibility to status updates

Mina Hsiang works for the US Digital Service. Here she discusses the development of
the Playbook and its usefulness:

Video

Note the usefulness of having a playbook or simple checklist to which managers can
refer back when monitoring. If a project is going off track, managers can ask whether or
not their team has “run the play” correctly.

Video

So the US government, just like most companies, is still working on how best to improve
its own implementation processes. 

Next week, we’ll discuss how organizations to learn from their implementation failures
(and successes) and improve their processes and operations. We’ll hear more from
Colonel Reese and the Army Lessons Learned department, and we’ll study the methods
through which you can solicit and give constructive feedback.