The Measure Phase, deep-dive course
Turning a validated problem into trustworthy numbers — data collection, measurement system analysis, and process variation, worked through a mid-sized IT project.
What is the Measure phase?
Define set the boundaries of the problem. Measure turns that problem into numbers everyone can trust. The team builds a data collection plan, checks that its measurement system isn't lying to it, and establishes a validated baseline that every later DMAIC phase will be judged against.
Think of Measure as the "instrumentation" phase — before you can improve a process, you need gauges you can trust, pointed at the things that actually matter to the customer and the business.
Goals of the Measure phase
Baseline
Establish a reliable, data-based picture of current performance — not opinion, not anecdote.
Trustworthy data
Confirm the measurement system itself isn't the biggest source of variation, using MSA.
Focus
Narrow from every possible metric down to the vital few that reflect customer and business impact.
Launch pad
Hand the Analyze phase a clean, validated dataset ready for root-cause investigation.
Key objectives
| Objective | Question it answers |
|---|---|
| Data Collection Plan | What will we measure, how, how often, and by whom? |
| Operational definitions | Does everyone measure a metric like "resolved" exactly the same way? |
| Performance measures | Which few metrics actually reflect what the customer and business care about? |
| Measurement System Analysis | Can we trust the measurement system itself? |
| Baseline performance | What is the current, validated level of performance — mean, variation, defect rate? |
| Process behavior | Is the process stable, or are shifts, trends, or special causes already visible? |
Importance of the Measure phase
- Prevents solving the wrong problem. Analyze and Improve built on unreliable data waste the entire project.
- Creates a fact-based baseline that proves the improvement later — this is how the financial benefit claim from Define gets validated.
- Surfaces early clues. Patterns in run charts often point Analyze toward the right root causes before a single fishbone session.
- Protects credibility. A project built on measurements no one trusts loses sponsor support fast, no matter how good the eventual fix is.
Tools used in the Measure phase
| Tool | Purpose |
|---|---|
| Data Collection Plan | Defines what to measure, how, how often, by whom, and where the data lives |
| Operational Definitions | Ensures every person and system records a metric exactly the same way |
| Check sheets / data collection forms | Simple, structured capture of manual data at the source |
| Sampling plan | Determines how much data — and how it's selected — is enough to be representative |
| Measurement System Analysis (Gage R&R / DDA) | Confirms the measurement system itself is accurate and consistent |
| Pareto chart | Prioritizes categories of defects or delays by frequency or impact |
| Cause-and-effect (fishbone) diagram | Organizes potential causes of variation for later root-cause testing |
| Run chart | Plots data over time to reveal trends, shifts, and other non-random patterns |
| Control chart | Distinguishes common-cause from special-cause variation |
| Histogram / normal probability plot | Shows the shape of the data's distribution |
| Process capability analysis (Cp/Cpk) | Compares process performance against customer specification limits |
Continuing the mid-sized IT project
This course follows the same "Service Desk Project" from Define into Measure.
Where we left off
- Validated problem: average IT incident resolution time is 3.6 business days against a 1-day SLA.
- Goal: reduce average resolution to 1 business day within 6 months, with password/access tickets under 4 hours.
- Scope: ticket intake through resolution and closure, for a 2,200-employee company running ~850 tickets/month.
Now Measure has to prove that 3.6-day number is real, break it down in a way Analyze can use, and confirm the ticketing system's timestamps can actually be trusted before anyone starts hunting for root causes.
Importance of Data Collection in the Measure Phase
Every tool later in Measure — MSA, run charts, capability analysis — is only as good as the data feeding it. Bad collection habits don't just add noise; they can manufacture patterns that don't exist, or hide ones that do.
Consistency
The same event is recorded the same way, every time, by every person or system.
Completeness
No systematic gaps — missing weekends, missing a category, missing a shift.
Timeliness
Data is captured close to when the event happens, not reconstructed from memory later.
Relevance
The data actually connects to the CTQs identified back in Define.
Data Collection Plan
A data collection plan is a one-page reference the whole team follows so nobody improvises a different method halfway through.
| Element | IT example — ticket resolution time |
|---|---|
| Metric | Ticket resolution time |
| Operational definition | Elapsed business hours from ticket creation timestamp to status change to "Resolved," excluding time in "Pending Customer" |
| Data type | Continuous (hours) |
| Source system | ServiceNow |
| Collector | Automated nightly export (no manual entry) |
| Frequency | Continuous capture; reviewed weekly |
| Sample / population | 100% of tickets (full population, since it's automated) |
Operational Definition Information
An operational definition is a precise, unambiguous description of how a metric is measured — specific enough that two different people, on two different days, get the same result from the same event.
| Vague (avoid) | Operational (use) |
|---|---|
| "Ticket is resolved quickly" | "Elapsed business hours from ticket creation to status = Resolved in ServiceNow, excluding Pending Customer time" |
| "Customer is satisfied" | "Post-closure survey score of 4 or 5 on a 5-point scale, captured within 48 hours of closure" |
| "High-priority ticket" | "Ticket tagged Priority 1 or Priority 2 per the ITSM severity matrix at time of creation" |
Why this matters for the IT example
Without a tight operational definition, one analyst might stop the clock when a ticket is "reassigned" while another stops it only at "closed." Both would produce different resolution-time numbers for the exact same ticket — and the team would spend Analyze arguing about whose data is right instead of fixing the process.
Performance Measures and Their Selection
Performance measures translate the customer and business needs identified in Define (VOC, CTQs) into indicators you can actually track over time.
| Category | Example IT metric | Why it matters |
|---|---|---|
| Outcome (Y) | Average ticket resolution time | Directly tied to the SLA and the project goal |
| Process (X) | Time in queue before first response | An upstream driver of the outcome metric |
| Quality | SLA breach rate (% of tickets over 1 day) | Captures defects, not just averages |
| Customer experience | Post-closure CSAT score | Confirms speed isn't being gained by cutting corners |
| Cost | Labor hours per ticket | Ties the metric back to the financial benefit case |
Customer Satisfaction Scores
Speed metrics alone can be misleading — a technician can close a ticket fast and still leave the employee's problem unsolved. Customer Satisfaction Scores (CSAT), and sometimes Net Promoter Score (NPS), are collected alongside operational metrics to catch that gap.
IT example — CSAT
- A one-question survey ("How satisfied were you with this resolution? 1–5") is sent automatically when a ticket closes.
- Baseline CSAT is 3.1 out of 5, down from 4.2 two quarters ago — tracked in parallel with resolution time so the team can confirm the fix improves speed and satisfaction, not one at the expense of the other.
Key Factors When Selecting Measurement Methods
| Factor | Question to ask |
|---|---|
| Relevance | Does this metric connect back to a CTQ from the project charter? |
| Data availability & cost | Can it be captured automatically, or does it require manual effort that adds cost and error? |
| Data type | Is it continuous (time, dollars) or discrete (pass/fail, category) — this drives which analysis tools apply later |
| Precision vs. effort | What's the minimum precision needed to make a good decision, without over-engineering collection? |
| Measurement system capability | Can the measurement method itself be validated with an MSA? |
| Timeliness | Can data be captured close to the event, at a frequency that supports timely decisions? |
| Definitional clarity | Can it be written as a tight operational definition everyone will apply the same way? |
Sampling Plan
When it isn't practical or necessary to capture every single data point, a sampling plan defines how much data — and how it's chosen — will still represent the real process.
| Element | Consideration |
|---|---|
| Population | The full set of items the data should represent (e.g., all incident tickets in a quarter) |
| Sampling frame | The actual list you can draw from (e.g., the ServiceNow export, which should match the population) |
| Sample size | Large enough to detect a meaningful difference, balanced against cost and effort to collect |
| Sampling method | Random, systematic, or stratified — stratified is common when subgroups (categories, shifts) matter |
| Frequency | One-time baseline vs. ongoing sampling throughout the project |
| Bias avoidance | Avoid convenience sampling (e.g., only reviewing tickets that are easy to find) |
IT example — sampling plan
- Resolution time itself is captured automatically for 100% of tickets — no sampling needed there.
- But manual ticket-categorization accuracy is audited using a stratified random sample of 100 tickets per week, split evenly across each of the five ticket categories, so no category is over- or under-represented in the audit.
Ensuring Measurement System Accuracy: MSA Fundamentals
Before trusting any number the team collects, Measurement System Analysis (MSA) checks whether the measurement system itself is adding meaningful error — because you can't improve a process you can't measure reliably.
DMAIC Measurement Systems: MSA for Data Types
| Data type | MSA method | What it checks |
|---|---|---|
| Continuous (e.g., resolution time in hours) | Gage Repeatability & Reproducibility (Gage R&R) | How much of total variation comes from the measurement system itself — repeatability (same person/tool, twice) and reproducibility (different people/tools) |
| Discrete / attribute (e.g., SLA breach yes/no, ticket category) | Discrete Data Analysis (DDA) / Attribute Agreement Analysis | How often different people classify the same item the same way, and how that compares to a known correct answer |
Try it: Gage R&R explorer
Pick a scenario to see how total variation splits between real part-to-part differences and measurement system error.
Acceptable Level of Measurement System Variation (Rules of Thumb)
| %GRR (Gage R&R) | Verdict | Typical action |
|---|---|---|
| Under 10% | Acceptable | Measurement system is fine — proceed with confidence |
| 10% – 30% | Marginal | May be acceptable depending on cost and criticality — consider improving it |
| Over 30% | Unacceptable | Fix the measurement system before trusting any data it produces |
For discrete/attribute systems, a common rule of thumb is requiring at least 90% agreement — both between appraisers and against a known standard — before trusting the classification.
Cause-and-Effect (Fishbone) Diagrams
A cause-and-effect (Ishikawa) diagram organizes every plausible cause of a problem into categories, so the team can test them systematically in Analyze rather than jumping straight to a favorite theory. Click a category below.
People
Process Variation
Every process varies. Measure's job is to tell the difference between variation that's just the process breathing normally, and variation that's a signal something specific changed.
| Common cause | Special cause | |
|---|---|---|
| What it is | Natural, expected variation built into the process | An unusual, identifiable event or factor |
| Pattern | Random scatter around a stable average | Trends, shifts, oscillation, or outliers |
| IT example | Resolution time naturally varies ± a few hours ticket to ticket | A VPN outage suddenly doubles resolution time for two days |
| Right response | Change the process itself (that's Improve's job) | Investigate and address the specific event |
Why this distinction matters
Reacting to every random blip as if it were special-cause "fire-fighting" wastes effort and can make a stable process worse. Ignoring a real special cause because "the average still looks okay" hides a fixable problem. Run charts and control charts, next up, are how Measure tells the two apart.
Run Charts: Reading the Six Signal Patterns
A run chart plots a metric over time against its median. Six recurring patterns signal that something non-random is happening. Explore each one below using the IT case study's daily ticket resolution time.
Normal Distribution — and What to Do When Data Isn't
Many Six Sigma tools (control limits, capability indices, some hypothesis tests) assume the data follows a normal, bell-shaped distribution. Real process data doesn't always cooperate — explore the shapes below.
What do I do when data is non-normal?
Find the cause first
Mixed populations, outliers, or a natural boundary (time can't go below zero) often explain the shape before you reach for a statistical fix.
Consider a transformation
A log or Box-Cox transformation can make skewed continuous data behave normally enough for standard tools.
Use non-parametric tools
Methods that don't assume normality (e.g., median-based tests) work directly on the data as collected.
Stratify mixed data
If two populations are blended (e.g., two ticket categories with very different times), split and analyze them separately.
Actions and Results Verification in the Measure Phase
Before handing off to Analyze, the team verifies that Measure was actually executed the way it was planned — not just that a baseline number exists.
- Data was collected per the approved Data Collection Plan, not an improvised shortcut.
- Operational definitions were applied consistently by every collector and system involved.
- MSA (Gage R&R or DDA) was completed and met the rules-of-thumb thresholds — or a documented plan exists to fix it.
- Baseline performance is calculated and documented: mean, variation, defect rate, and sigma level or DPMO.
- Run charts / control charts have been reviewed for obvious special causes before the baseline is finalized.
- Any measurement system fixes made mid-Measure were re-verified afterward.
- The sponsor has been briefed on the validated baseline and agrees it reflects reality.
IT example — verification outcome
The team's Gage R&R on the automated timestamp system came back at 4% — well within the acceptable range — so the 3.6-day baseline is confirmed as real, not a measurement artifact. A quick attribute check on ticket categorization found 88% agreement, just under the 90% rule of thumb, so a short retraining on category definitions is logged as a follow-up before Analyze begins.
Measure Phase Outputs and Deliverables
Checks are saved in your browser as you go.
Frequently Asked Questions for the Measure Phase
How do I gather data for defect rates, step by step?
- Write the operational definition of a "defect" first — e.g., any ticket that breaches the 1-day SLA.
- Identify the system of record (ServiceNow) so everyone pulls from the same source.
- Determine total opportunities (all tickets in the period) and defectives (tickets breaching SLA).
- Pull data over a representative window — long enough to cover normal operating conditions (e.g., 90 days, including a month-end peak).
- Calculate the defect rate: defectives ÷ opportunities (or convert to DPMO for a more rigorous sigma-level comparison).
- Stratify by category, shift, and agent to see where defects concentrate.
- Run a quick MSA / audit sample before trusting the number — a wrong "defect" definition invalidates everything downstream.
What is an operational definition of an indicator?
A precise, unambiguous description of how a metric is measured — specific enough that two different collectors get the same answer for the same event. See the Data Collection section above for worked examples from the IT case study.
What is a sampling plan, and when do I need one?
A sampling plan defines how much data, and how it's selected, will represent the full population when capturing everything isn't practical. You need one whenever data collection is manual, expensive, or slow — automated system data (like timestamps) can usually just be captured in full instead.
Should I collect continuous or discrete data?
Continuous data (time, dollars, counts on a scale) carries more information per data point and generally needs a smaller sample size to draw conclusions. Discrete data (pass/fail, category, yes/no) is often easier and cheaper to collect but needs larger samples to detect the same size of effect. Where possible, favor continuous measures for your primary metric.
How much data is enough?
Enough to cover the normal range of process conditions (shifts, weeks, categories) and detect a difference that actually matters to the business — not an arbitrary round number. When in doubt, involve someone comfortable with sample-size calculations rather than guessing.
Measure Phase Survival Challenge
You've got three hearts. Answer wrong three times and it's game over — survive all the questions to win the belt.
Final score:
Five-Question Review
A short, focused check on the core Measure-phase concepts.
Quiz complete
Score: / 5