USING FRAMEWORKS TO FRAME OUR THOUGHTS - AGILE METRICS (Source: XBOSoft, Inc.

) ONE-PAGE MAGIC: Created by Mike Abby &Ken Martin

By using metrics, we can know what we are going to get!

MINIMISE REWORK RESOURCE OPTIMISATION KNOW THE FUTURE
WHY
Want to minimize mistakes and doing things over (rework) Want to utilize resources effectively Want to predict when things could go wrong
METRICS?
WANT TIMELINES FIX DEFECTS EARLY NO PRODUCTION DEFECTS
Want to do what we say we will do, when we say - timelines Fix defects early – later is more expensive Production defects are bad news

FROM Metrics should be easy to capture have a valuable purpose toward improvement…
METRICS? EVALUATE GUIDE INSIGHTFUL ACTIONABLE
Some say no metrics needed…
HOW TO
IMPROVE VELOCITY WHERE TO IMPROVE? RESOURCE UTILISATION? WORKING ON THE RIGHT THINGS?
AGILE Is velocity all you track? What is sacrificed for How can we know what and how we are doing or How do we know if we’re using resources How do we know we are spending our
velocity? where to improve? effectively effort/time in the right place?
Using frameworks to organizing our thoughts…
FRAMEWORK
USE BENEFITS ORGANISE AND CATEGORISE ALL FUNCTIONS & PROCESSES COVERED DEVELOPS RELATIONSHIPS
Putting metrics into a framework helps to categorize and organize It makes sure wall functions in our organization and process covered It develop relationships between elements in our process
Many software metrics programs have failed because they had poorly defined, or even non-existent objectives. GOAL –> QUESTION -> METRIC. Answering questions that lead to decisions -> VALUE
EXAMPLE GOAL: Reduce total cost of development EXAMPLE GOAL: Reduce total cost of testing-effort EXAMPLE GOAL: Reduce the number of new feature bugs
BASILI’S GQM
QUESTION: Which areas have the QUESTION: Which functional areas have QUESTION: How long does it take to QUESTION: How complete are my QUESTION: How much time do I
highest re-work rates? the most defects? repair defects? requirements? spend testing?

GQM METRIC CATEGORY QUESTION METRIC TOOL USED NEEDED

W/S EXAMPLE Customer Satisfaction Are we meeting schedule? Schedule Variance Project Management Tool None
Agile Goals are Speed, Quality and Customer Satisfaction. What Questions Tell Me If I’m Reaching Those Objectives?
AGILE
SPEED: Technical Debt, Schedule, Planned versus Unplanned QUALITY: Working Product, Requirements, Owner Participation
REQUIREMENTS SCHEDULED UNPLANNED VS. PLANNED

SPEED • Did I get them in time? • Am I on schedule? • Did I estimate well?

• Do I understand them? • Can I be more productive? • Did I finish what I said I would?
Did we deliver working product? • How much wasted time? Rework? • Do I make mistakes that cause more (unplanned) work?
PRODUCT OWNER REQUIREMENTS TECHNICAL DEBT

QUALITY • Did my end users find defects? • Are my requirements delivered when needed? • Is my software performance getting worse and worse?
• Did product owner give feedback regularly? • Did the delivered product match the requirement? Or wrong? • Am I prepared if my lead engineer leaves?
• Are there less defects than before? • Do engineers have trouble understanding the requirements? • Do I mess things up when fixing things?
TEN AGILE METRICS ONE-PAGE MAGIC: Created by Mike Abby &Ken Martin
BE – PEOPLE DO – PROCESS HAVE –RESULTS
• Communicative • Iterative (sprints) • Speed
AGILE • Collaborative/Cooperative • Daily stand-ups /Face to face communication • Quality
Agile Is a Way of Thinking • Flexible and willing • Post mortem – end of sprint
People “Be-Do-Have” • Knowledgeable-multi • Delivery meeting – end of sprint
• Initiative/responsible Process • Planning meeting – before sprint
• Responsive/adaptive • Self organizing
METRICS PROCESS UNDERSTAND: 1) What and why 2) Goals and objectives EVALUATE: Ask yourself questions IMPROVE: Measure to answer those questions and take action

1) Am I on schedule? 2) Did I estimate well? 1) Is my technical debt accumulating? 2) Am I wasting time

5. DEFECT
• Tracking our velocity over time tells us our progress and capacity and helps us forecast based on our REWORK RATE • We can’t get faster if we are always fixing things
10. VELOCITY capabilities (now and in the future) • Depends on valid data whereby people are logging real hours to tasks they take on.
• Measurement : Sum of all the approved estimates of the stories • Metric: Fix Effort Ratio : Time to fix defects/Total effort expended (hours)
• One point in time measurement has little meaning, so need to track over time / another metric
• Metric: Velocity(T)-(T-1)/Velocity(T) 1) What is throwing me off schedule? 2) Am I biting off too much?
4. TOTAL • Measurement: Sum of work done above the original estimates for those stories in the plan.
1) Did my end users find defects? 2) Are there less defects than before?
UNEXPECTED • Are we underestimating difficulty?
9. DEFECT • We don’t want the end user to find defects DRE GOAL WORK • Many reasons: a) Incomplete user stories b) Misunderstood user stories c) Bigger and
REMOVAL • Any defect found by the user influences quality Estimated (by Capers Jones) that defect more complex than I thought? d) Not enough time for research e) Too hard...
EFFECTIVENESS • Metric: Defects in found in production during (90 removal effectiveness differs by different • Metric: (Total work done – Planned work)/Planned work
day) period post release (DFP)/DFP + Defects found levels of process capability maturity levels:
Pre-release 1) Level 1: 85% 2) Level 2: 89% 3) Level 3: 1) Are my customers/end users happy? 2) Is the architecture prohibiting me?
91% 4) Level 4: 93% 5) Level 5: 95%
3. DEFECTS IN • If defects make it to production, that’s bad!
1) Am I on schedule? 2) Am I overcommitting? 3) Is my rate of work sustainable? PRODUCTION • But what is worse if an important defect stays unfixed
AVG. FIX TIME • Defects that take a long time to fix represent either technical debt or poor process
8. ADDED WORK • Are we making promises that maybe we should not – overcommitting and unsustainable
• Metric: Time to fix Defects (released to production- usually P1 defects found) / Total P1
• Measurement: Sum of work (new stories added) over and above the original plan
Defects (post production)
• Metric: New stories added/Original Stories Planned
1) Why are things breaking? 2) Can I get faster?
1) Am I allocating resources optimally? 2) Do I have trouble understanding requirements?
• Reduce rework = Increase speed REWORK REQUIREMENT ERRORS
7. WORK • Let’s us se we are spending our time and adjust our priorities 2. DEFECT • Discover when defects are injected • Change in requirements
CATEGORY • Measurement: Sum of work (for each category) CATEGORIZATION • Reduce and prevent defects from happening in • Don’t understand requirements
ALLOCATION • Metric: Divide by total work to get % allocation ALLOCATION %
Categories could include: the first place. • Incomplete requirement
a) Technical debt – refactor - redo b) Regression testing c) New feature development d) Research e) • Drive down the Fix Effort Ratio leading to better • Requirement does not exist
quality and resulting in higher velocity • ‘oh, I meant this...’
Requirements review f) Defect fixing g) Other
• Examine defect rates per category
1) Have the right people on my team? 2) Did I estimate work well? 3) Can I attain quality sustainably?
1) Delivering what agreed? 2) Feedback from PO? 3) Will meet the release date?
6. WORK • Sustainable” - We can’t sustain our effort if we are continuously putting in overtime 10) WORKING
SUSTAINABILITY • Overtime is necessary, but needs to be managed and causes mistakes SOFTWARE WORKING SOFTWARE METRIC
• Mistakes manifest themselves in many ways, such as technical debt or defects DELIVERY RATE • Does the software work or not? # story points delivered that ‘work’ / #
• Overtime often results from poor work estimates and directly proportional to technical debt • Working does not mean perfect story points that were to be shown to
• Metric: Unplanned Overtime Work/Total Work • Working means can show in to get feedback the PO but not be shown to work
Source: https://pmaspire.com/ THE USE OF AI IN A PMO DASHBOARD Created by Mike Abby & Ken Martin
Using AI in a PMO Dashboard means using smart algorithm to give real time project status of the project which helps stakeholders in effective decision making

AI CAN HELP ELIMINATE NEED FOR HUMAN INPUT AI MAKES SURE NOTHING IS OVERLOOKED INTEGRATION WITH COMMS TOOLS AI REDUCES ADMIN TIME FOR PROJECT MANAGERS
AI PM systems are able to effectively handle AI helps PM’s save time on their various Some AI PM tools integrate with AI can monitor pattern as an assistant to PM’s. Studies have
USE OF AI IN scheduling, reminders, and follow-ups to eliminate tasks by helping to make sure that nothing is comms tools like Slack & PM tools like shown that PMs spend more than 50% of their time on
PM the need for human input overlooked JIRA to solve issues administrative tasks
SOFTWARE AI CAN HELP WITH PREDICTIVE ANALYTICS FOR PROJECTS AI HAS ABILITY TO MONITOR PROJECT TEAMS’ PRODUCTIVITY
AI can use its ability to manage complex analytics to observe how a project is moving & make educated AI has ability to monitor project teams & make predictions based off of the patterns of habits and
predictions about future behaviours that it sees
CONTINUOUS IMPROVEMENT TO ADD VALUE A SURVIVING PMO IS AN ADAPTABLE ONE USE OF PMO TOOLS TO SHOW VISIBLE VALUE
A value add PMO looks for opportunities to add value to Projects are becoming more complex & using new ways to deliver projects such as A big challenge for a PMO to survive is to understand what is
portfolio management, PM processes, PM tools and PM SCRUM, Lean and Six Sigma., For these new ways, a PMO needs adaptable value to senior management & provide this value in terms of
PMO knowledge processes, governance, reporting & metrics metrics via automated reports
CHALLENGES PROJECTS NOT ALIGNED TO BUSINESS STRATEGY GETTING THE PROJECT BASICS DONE RIGHT FINDING THE RIGHT PMO TOOL A KEY TO PMO SUCCESS
A PMO need to understand business strategies and how With all the challenges that a PMO faces, the one challenge that should not exist This is not a simple task & the success of a PMO can rest on the
the current projects are aligned with the business but often does, is getting projects basics done right – business case supported success of the tool it chooses & if tool is too complex or does
strategy & what value delivered projects, status reporting, resource mgt. show value reports, it will fail
PROJECTS HAVE NO BASELINE (Source: Wellingtone) MILLIONS WASTED ON PROJECTS (Source: PMI.org) A LARGE NUMBER OF PROJECTS FAIL (Source: Geneca)
INDUSTRY Over 1 in 3 projects have no baseline Every $1 Billion invested in USA, $122 M wasted due to poor project performance 75% of business executives anticipate software projects will fail
GAPS HALF OF PROJECT MANAGEMENT OFFICES WILL CLOSE IN 3 YEARS (Source: KeyedIN) MOST PM EXECUTIVES DON’T KNOW IF THEIR PROJECTS ALIGN WITH STRATEGY (Source: Changepoint)
50% of all Project Management Offices (PMOs) close within just 3 years 80% of PM executives don’t know how their projects align with business strategy

KEY NO AI BASED DASHBOARD FOR PROJECT SPONSORS / PMs NO REAL TIME CHANGE MANAGEMENT LACK OF RISK SCORING
CHALLENGES AI based Dashboard is Missing for Project Sponsors / PMs Change Management is not real time/integrated Risk Scoring is missing & unavailability of automated risk notification
WITH PM LACK OF INTEGRATED ISSUE MANAGEMENT & ESCALATION NO PROJECT BASELINE, VERSIONING & FORECASTING LACK OF REAL TIME PROJECT STATUS REPORTING
SOFTWARE Integrated issue management and escalation is missing Project Baseline , Versioning and Forecasting is missing Some PM software tools lack the capability of real time project status

PMO PROJECT BUILDER SCHEDULE MANAGEMENT PERSONAL KANBAN BOARD PROJECT BUDGETING INTEGRATED ISSUE MANAGEMENT
DASHBOARD
PROJECT BASELINING DEPENDENCY MANAGEMENT RESOURCE MANAGEMENT RESOURCE RATE CARD INTEGRATED CHANGE MANAGEMENT
SOFTWARE
FEATURES TASK MANAGEMENT MILESTONE MANAGEMENT RESOURCE CALENDAR INTEGRATED RISK MANAGEMENT SYSTEM GENERATED STATUS REPORTS
USER ROLE PM ROLE PROJECT SPONSOR ROLE ADMIN / PMO HEAD ROLE
EXAMPLE
- User can view risks, issues & change requests - PM overall manages the project. - Create projects & create users - The master role & admin of PMO software
PMO
- User can view tasks, update & time sheet entry - View summary reports of their projects - Assign roles to users related to their portfolio - Create projects, create users, assign roles
DASHBOARD
- User can raise risk, issues, change requests - Create milestone & tasks & allocate to the users - Will have a comprehensive summary Dashboard - Setup the parameters of risk and issues scores
ROLES
- View project & task summary in dashboard - Raise & manages risk, issues, change requests - Will have generated reports of all projects - Complete visibility of all organisational projects
REPORT PROJECT ISSUE PROJECT FORECASTING PROJECT WISE MILESTONE TASK ALLOCATION PROJECT STATUS PROJECT RISK RESOURCE PROJECT
EXAMPLES REPORT (EAC) REPORT PRODUCTIVITY REPORT REPORT REPORT REPORT REPORT RATE REPORT BURNDOWN CHART