MICHAEL LAW: Now that I've briefly introduced you

to some of the other study designs that are out there, let's move on
and talk about interrupted time series.
So we'll start by talking about a single group interrupted time series.
This is a time series that follows one group at multiple time periods,
before and after intervention.
So you can see what I've outlined below.
This is a hypothetical time series, that has
eight measurement points before, an intervention, and then 16 points
in total.
So there's eight observation periods before the intervention, in this case,
and eight after.
There's no need for it to be balanced.
You could have eight and 18, or you could have 18 and 18.
So in an interrupted time series, the design
is to compare longitudinal trends before and after the policy change.
So in a sense it's like a pre-post, except instead of one observation
before and one observation after, you're adding
on a whole series of observations so that you can establish the trend.
Now think back to some of those biases that we talked about.
Things like maturation, that trend is going
to help you address that and get a sense for what
was happening over a longer period of time,
before the change that you're interested in.
In a single interrupted time series, the major assumption
is that the existing level and trend in the outcome among those
exposed to the intervention would remain the same absent the intervention.
So let's throw a class size example.
So here you'll see that I've taken the 78 and the 83 from the example
that we've been talking about with edX.
What I've then done is added on a number of hypothetical points before.
Now you'll see this line is tending upward
in terms of trends-- that's to denote the little bit of grade inflation
that we've talked about a few times.
And so in an interrupted time series, essentially you're
going to take this linear trend beforehand,
and you're going to assume-- so your counterfactual assumption-- is
going to be that that trend would have continued in to the future.
So you see right here, this is the counterfactual.
You're then going to use the observed data
points that you have to establish an observed line, or a level and trend.
And then you're going to use that to figure out whether or not
there was change in the level and/or a change in the trend of the outcome
after the intervention-- so in this case, the smaller class sizes.
To formalize this a little bit, you can see here the outcome of interest
here is along the side.
This is the observed line that we were talking about.
The level change denotes the immediate change in the outcome.
So the expected value for the outcome at this first point post intervention
would have been here.
In actual fact, we observed it here.
So this difference between the two of them
is the level change-- that's the first outcome.
And then you can also see that this line was sloped upward
before the intervention, and so the counterfactual thus
carries that slope with it.
The observed line here is flattened out--
there's been a downward slope change.
And so that's the second outcome of interest.
It's important to recognize that either, both, or neither of these things
might happen.
For example, say we had the same observed and counterfactual line,
as in the last slide-- the different outcomes one might see.
For example, if there's no change at the level or the trend in the outcome,
you would expect the observed line in the post period
to follow that counterfactual exactly, something like that.
You might also observe just a change in the level.
In which case, you would expect the observed line in the post period
to follow the counterfactual in terms of slope, but at a different level.
So there is a constant change over the entire post period.
This would just be a change in level, but no change in trend.

You might observe just a change in the trend,

which might look something like this.
Where there's no immediate change, the expectation just after the intervention
is more or less the same.
But afterward, you see this downward change in the trend.
You could also see an upward change in the trend, like this.

And finally, you might get both a change in level and trend.
So something like this, where there's an immediate change in the level--
but then if you look at just the level change, there's also a trend change
involved as well.
So this would be the level, and this would be the trend.
You can see how this gives you an enormous amount of flexibility.
It allows you to fit a whole number of shapes,
so long as the data conforms reasonably well through a linear trend over time.
The threats to internal validity that you're
going to have to look out for, for the biases,
start to get pretty specific when you get into an interrupted time series.
So first off, there's the potential for a history threat--
if something aside from the policy affected the outcome.
But the important part here is that it was implemented
near the same time as the policy.
So if something happened three years before the policy,
it's not going to be an issue.
What you would worry about is something that's contemporaneous,
that happened at the same time.
In that case, it's going to be difficult to tease those two things apart--
unless you potentially had a control group, which
we'll talk about in a minute.
The other threat to validity is instrumentation.
And this is, again, a changing measurement.
But the same caveat that applies to history
applies here, that it occurred near the same time as the policy.
So what you need to do and be careful of, is that neither of these changes
happened close to the intervention that you're interested in.