Understanding Doctor'S Rights: Effective Use of E-Discovery: Edicare Verpayment Udits

M EDICARE O VERPAYMENT A UDITS
UNDERSTANDING DOCTOR’S RIGHTS:

Effective Use of e-Discovery
Barraclough Ltd.∗
1
Abstract
When health care providers are faced with a Medicare Audit, they have
a right to receive an understanding of the way in which an overpayment de-
mand is derived. This includes not only the analysis of various claims in the
sample, but also the statistical method used to calculate the numbers. Our re-
search indicates that it is common practice for auditors to provide incomplete
information regarding their statistical work. The administrative law rules for
discovery of information by the health care provider are more restrictive than
for the Federal Rules of Civil Procedure (FRCP). This paper discusses this
problem, and argues that basic rules of eDiscovery are relevant to a health
care provider in protecting due process and their rights during an audit.
Keywords: RAC; Recovery Audit Contractor; CMS; Medicare Fraud; Au-

dit; Overpayment; Overpayment Demand; Statistics; Forensic Statistics; De-
fense Strategy; Extrapolation; Discovery; e-Discovery; Federal Rules of Civil
Procedure; Medicare Program Integrity Manual
∗ BarracloughLtd. is a New York State corporation that specializes in providing Research

ArchitectureTM and expert advisory services, including forensic statistics for Medicare appeals.
Address: 135 East 54th Street 4-B, New York, N.Y. 10022-4509 Tel: (646) 416-6592
eDiscovery & Doctor’s Rights http://www.medicare-audit-defense.com
Abbreviations & Sources

ALJ
Administrative Law Judge.
CD
Compact Disk, a storage medium for data.
CFR
Code of Federal Regulations. Most of the relevant law comes from 42 CFR 405 titled
Federal Health Insurance for the Aged and Disabled.
eDiscovery
A term that refers to discovery of evidence that is stored in electronic form. Some-
times abbreviated with a hyphen, e.g., eDiscovery.
eDocument
A term sometimes used to refer to any electronic data. The term “document” is par-
tially mis-leading since instead of being a “document” it could refer to a database.
Under the revised FRCP, electronic information is considered to be document in na-
ture.
Error Validation Review
Sometimes referred to as a “Probe Review” (see below). A process in which the
contractor verifies the cause of a potential error in claims made by the health care
provider. Details are found in PIM 3.2(A). NB: This review “does not allow projec-
tion of overpayments”.
FRCP
Federal Rules of Civil Procedure. Note: These are generally not applicable to Medi-
care appeals, since these appeals are managed by administrative law.
MAC
Medicare Appeals Council.
PIM
Medicare Program Integrity Manual.
Probe Review
Often confused with a probe sample. The Probe Review is a non-statistical review of
a selection of cases from a health care provider solely to determine if further more
detailed investigation is warranted.

c 2010. All Rights Reserved 3 Barraclough Health Analytics
Probe Sample
A required first stage of a statistical procedure intending to take a statistically valid
random sample. The probe sample allows the statistician to determine how large the
sample size should be.
QIC
Qualified Independent Contractors.
Spreadsheet
An electronic compilation of the statistical data using a calculation program such as
Microsoft Excel or OpenOffice. Sometimes called a “worksheet”.
SVRS
Statistically Valid Random Sample.
Terra Incognita
Refers to unknown “territory”; here it refers to potential legal procedures that have
not yet been recorded.
UR
Uniform Rules Relating to Discovery of Electronically Stored Information, (Draft for
Approval), National Conference of Commissioners on Uniform State Laws, Pasadena,
California, July 27 – August 3, 2007.
Worksheet
See Spreadsheet.

1 The Specter of a Medicare Audit

The rising number of Medicare Audits may be one of the reasons why more physi-
cians are “dropping Medicare”. In describing the effect of one audit, C. L. Gray
wrote:
“In independent reviewer (who was paid a percentage of the audit)
reviewed 86 patient records and “found” the physicians had “fraud-
ulently billed” Medicare for $351,820. After spending a year fight-
ing the allegations, eventually, eventually all charges were dropped.
The physician group was vindicated but only after spending almost
$100,000 defending themselves. The independent reviewers were clearly
after money, not justice.”1
American Medical News describes these audits also in a negative light:
“Aggressive tactics, vague charges, interrupted patient care, drained “Medicare
staff resources and administrative headaches. These are some of the Bounty
problems doctors say they encountered during their experiences with Hunters”
Medicare’s newest breed of auditor.”2
Physicians and all health care providers should worry about the Medicare Audit.
An audit drains away valuable resources at the office. It can lead to financial
collapse. It now is commonplace for any report of an audit to be accompanied Bankruptcy
by a form to be submitted to CMS in case the health care provider wishes to file for
bankruptcy!
How can an audit lead to bankruptcy? It is simple — the amount de-
manded for repayment to the Medicare Trust Fund is so great that it exceeds
whatever cash the health care provider has on hand, and also exceeds their credit
line.
These enormous demands for repayment do not come about as a result
of a comprehsnsive review of claims. It is an educated guess. The result of an
extrapolation from a sample of claims examined in the audit. This extrapolation
1 C.
L. Gray, Why Doctors Are Abandoning Medicare. FoxNews.com January 14, 2010 at
http://www.foxnews.com/opinion/2010/01/14/c-l-gray-medicare-doctors-deserting-obama/
2 David Glendinning, “Medicare audit overreach? Doctors think so, but audits aren’t going
away; Physicians decry the heavy-handed methods of Medicare’s “bounty hunters,” but Congress
and the White House are expanding the program nationwide. American Medical News July 7, 2008
available at http://www.ama-assn.org/amednews/2008/07/07/gvsa0707.htm

has two aspects: First, a review of cases to determine if all billed procedures were
medically necessary3 ; and second, a statistical analysis that first takes a sample of
the health care provider’s cases and then uses it to make an extrapolation of the
overall repayment demand.
Here, in this paper, we are going to ignore the medical necessity part of
the argument and focus on the statistical side. This is the question:4
How does the auditor arrive at these giant numbers for the repayment demand,
and if the statistical procedure is flawed what recourse has the health care
provider ?
2 Types of Statistical Analysis Leading to the Overpay-

ment Demand
One of the foundations to an audit is the statistical analysis used for various cal-
culations. Generally, there are three types. First, there is analysis that identifies in
the first place which health care providers shall be “targeted” for an audit. Sec-
ond, there is the analysis that leads to the selection of a sample that later will be
analyzed. Third, there is the calculation made that applies the results from that
sample to the entire “universe” of claims, and thus magnifies by an order of mag-
nitude or more the amount it is claimed must be returned to Medicare.
The statistical sampling and extrapolation approach is used as it is rec-
ognized that it is infeasible to go over every single claim.
2.1 Initial Targeting of Health Care Provider

The Program Integrity Manual (PIM) provides a general look into the process and
associated criteria leading to the initial targeting of a health care provider. Essen-
3 There are other possible reasons that may be cited, but “medical necessity” is the most impor-
tant.
4Note that generally appeals are based on a hearty discussion of “medical necessity”. The
hard-fought hearings revolving around that issue are characterized by piles of paper containing
the relevant patient records. These lengthy discussions may take hours with the ALJ, but they
are beyond the scope of this paper. Here we are concerned strictly with the statistical side of the
house. In particular, identification of the different arguments that can be made to illustrate an
invalid statistical analysis was done, and what information it takes to develop those arguments.

tially, the work involves looking at “patterns of claims [that may] indicate poten-
tial problems.” Any analysis “should include simple identification of aberrancies
in billing patterns within a homogeneous group, or much more sophisticated detec-
tion of patterns within claims or groups of claims that might suggest improper
billing or payment.” (emphasis added) (PIM 2.1(C) para. 1)
What this amounts to is that contractors use their giant mainframe com-
puters to look for patterns by sifting through hundreds of millions of claims. At
the most elementary level, the contractors identify and examine “homogeneous
groups” of health care providers, and compare their billing practices. Any “out-
liers” automatically become more likely to receive an audit.
“Using claims data, contractors shall determine the degree to which
a potential error is widespread and decide if the potential error meets
the deviation indicators established.”(PIM 3.2 para. 1)
The definition of “homogeneous group” is not provided, and may not be
as easy to define as suggested in the PIM. We do not know of any appeal based
on an argument that the group of peers chosen by the contractor fails to be repre-
sentative of the health care provider under investigation, although it appears that
under some conditions this might be a valid argument.5
The PIM (3.10.1.4) gives the contractor great leeway in determining whether
a sustained payment error exists including but not limited to error rate determina-
tions, conducting probe samples, doing any type of general data analysis, exam-
ining the health care provider’s history, taking into account information from law
enforcement investigations, acting on allegations of wrongdoing by current or for-
mer employees or other audits or evaluations. This is one of the most comprehen-
sive authorizations for a general “fishing expedition” as one could imagine. “Fishing
There also are other ways to identify targets for auditing, but these are Expedi-
not defined very clearly. In fact, no defined criteria exists except perhaps that there tion”
must be a reasonable belief that the health care provider somehow is abusing the
system.
2.1.1 The Error Validation Review
If a specific health care provider has been surfaced for further investigation, then
the auditor may conduct an “error validation review”. This involves taking a
5In addition, it is not known how far if at all the discovery process is allowed, should the
appellant wish to pursue this argument.

more detailed look at a small number of claims they have submitted.

Sometimes this process is called a “probe review”, but this is an unfor-
tunate turn of phrase, since it easily can be confused with the “probe sample”
that is part of any valid statistical analysis. This confusion regularly shows up in
case-related correspondence from the contractor.
In the error validation review, the PIM specifically points out that this
review “does not allow projection of overpayments to the universe of claims re-
viewed.” (PIM 3.2 (A)) What this means is that in order to generate a valid extrap-
olation, a completely separate and distinct analysis must be undertaken.
As we shall see later, it is not uncommon for contractors to over-ride
this restriction and attempt to use the cases they have analyzed during an error
validation review so as to project the overpayments “to the universe of claims”.
When this happens, it is an invalid analysis on the part of the contractor, and is a
direct violation of the procedures specified by the PIM.
2.2 Sample Analysis

The taking and analysis of a subsequent and statistically valid random sample
(SVRS) of the health care provider’s claims is “used to calculate and project (i.e., ex-
trapolate) the amount of overpayment(s) made on claims.” (PIM 3.10.1.2) This
next step in the audit process is not mandatory but rather is used only when the
auditor has by other means identified a “sustained or high level of payment er-
ror”.
How the contractor goes about taking their sample is critical to the fair-
ness and accuracy of the process. It is in this part of the audit where we have
found the most appeal-worthy points. The PIM (3.10.1.3) defines six steps needed
for a valid statistical sampling. These are:
1. Selecting the provider or supplier;
2. Selecting the period to be reviewed;
3. Defining the universe, the sampling unit, and the sampling frame;
4. Designing the sampling plan and selecting the sample;
5. Reviewing each of the sampling units and determining if there was an over-
payment or an underpayment; and, as applicable;

6. Estimating the overpayment.
The contractor is given a most liberal flexibility in choosing which sta-

tistical approach to use, provided that whatever it uses is properly employed.
The PIM allows different sample designs (3.10.4.1) and provides guidance on sim-
ple random sampling (3.10.4.1.1), systematic sampling (3.10.4.1.2), stratified sam-
pling (3.10.4.1.3), cluster sampling (3.10.4.1.4) and different design combinations
(3.10.4.1.5). As to the technical differences between these approaches, we will not
go into detail here, but will emphasize only that the one element all of these sta-
tistical methods have in common is that they must be valid and represent fair and
accepted statistical practice.
A statistician or “a person with equivalent expertise in probability sam-
pling and estimation methods” must be used. (PIM 3.10.1.5) It has been our expe-
rience that each contractor has a boiler-plate way of doing their work, and regard-
less of the particular statistician used, the actual work product tends to be more
or less identical from case to case.
As regards to Step 5, above, in the audit, the PIM requires that a valid
probability sample be taken and defines what this is. A frequently quoted passage
from the PIM is:
“If a particular probability sample design is properly executed, i.e., defin-

ing the universe, the frame, the sampling units, using proper random-
ization, accurately measuring the variables of interest, and using the
correct formulas for estimation, then assertions that the sample and its re-
sulting estimates are ‘not statistically valid’ cannot legitimately be made. In
other words, a probability sample and its results are always ‘valid.’ ” (em-
phasis added) (PIM 3.10.2)
It is common for the contractor to quote the italicized part of this passage
in an attempt to summarily reject any claims that the statistical work is invalid.
This passage appears in a surprisingly large number of overpayment demands or
subsequent calculations, but its use compels further analysis.
First of all, it is clear that this statement is valid only if the statistical
method being used is valid and meets all of the PIM criteria. As a consequence,
it should not as often as it is be used as a general argument that all of the work
automatically may not be challenged based on it being an “invalid sample”.
What this means in effect is that if it can be shown that a valid statistical analysis
was not performed, then the entire overpayment demand is invalid except for specific

claims that have been identified and studied in detail.
2.3 The Extrapolation

The extrapolation is simply the application of the results from the (valid) sample
to the entire “universe” of claims being reviewed. It is about the same as political
polling. A polling sample of Democrats, Republicans or “Tea Baggers” is taken in
a state during an election, then their opinions are “extrapolated” to the state as a
whole.
Here, the level of error in the payments made to the health care provider
is extrapolated across all of the claims submitted so as to come up with an estima-
tion of the total amount of the overpayment.
There is no need to go into the details of the mathematics for an extrap-
olation. It only need be remembered that any extrapolation must be based on a
valid sample in the first place. If it is not, then the extrapolation may not possibly
be valid.
2.4 The Revision Problem

One problem that frequently occurs involves the change in sample size resulting
from that part of the appeal involving medical necessity. The problem is this:
When the contractor has based their overpayment extrapolation upon a sample
of a certain number of cases, what is the result when a number of those cases are
ruled in favor of the health care provider, thus substantially reducing the size of
the sample? For example, suppose the contractor sampled thirty cases, but twenty
were ruled in favor of the health care provider and “thrown out”, leaving only ten
cases indicating a problem? Then can we say that only those ten cases remain
valid as an appropriate sample size to apply to the entire universe of claims and
thereby derive a valid overpayment extrapolation? Is it fair to the health care
provider to have the overpayment extrapolation based on a sample that is merely
1
3 -rd in size of the originating sample?
Many would argue that it can not be fair, particularly if it can be shown
that a sample size of ten is grossly inadequate. The problem is made even worse
if, as is frequently the case, the original sample taken itself was far too small.

3 The Structure of the Appeals Process

Medicare is complex. There are four types of Medicare coverage: A, B, C & D.
For all types of Medicare, the mechanisms for appeal are roughly consistent. See
Figure 1 on page 33. We are concerned here with plans A & B.
Medicare as a government agency does not directly handle the work of
processing forms and making decisions about coverage. This is handled by con- 120 Days
tractors Medicare hires to process the billions of claims it receives. If they find a
health care provider has been overpaid, notification of this is called a “determina-
tion”. The determination notice is the first point when the health care provider is
made aware of a potentially devastating demand to repay monies to the Medicare
Trust Fund. Since auditors may take a commission on this repayment, critics call
them by the pejorative term “Medicare Bounty Hunters”.
At the initiation of the process for an initial determination, a list of the
information that must be spelled out for the health care provider is provided in
42 CFR 405.921(b)(2)(i-vi)6 . It is quite general and includes “[t]he basis for any
full or partial denial determination of services or items on the claim”. In practice,
as regards regarding any statistical process that has been used, there is very little
useful information provided at this stage.
3.1 The Redetermination by the Medicare Contractor

The first level of appeal goes straight back to the same contractor making the
initial decision regarding coverage. This first appeal is called a redetermination,
and it is almost pro forma. The health care provider merely writes back to the
contractor and requests they revisit the decision. According to Medicare “The
appeal is a new and independent review of your claim.” This means that the
person inside the contractor who conducts the redetermination may not be the
same individual who made the initial determination (even though they work in
the same company).
In the cases we have examined, the redetermination almost always sim-
ply reaffirms the original determination. When parts are reversed, it most often
is on narrowly technical grounds, such as clerical errors, but rarely is it based on
substantive grounds.
6This is the complete reference. Hereinafter we will delete the preamble “42 CFR” and replace
it by the section symbol §.

The responses provided by the contractor vary according to the stage

of the appeals process, but how they are made and the informatioin they must
contain is spelled out in that administrative law found in the Medicare Program
Integrity Manual (PIM) and the Code of Federal Regulations (CFR).
When the health care provider makes the decision to appeal, § 405.946(a)
specifies the type of information that must be submitted in the request for a re-
determination. “[A] party must explain why it disagrees with the contractor’s
determination and should include any evidence that the party believes should be
considered”. For the most part the request for a redetermination is more or less
routine and generally focuses on issues regarding medical necessity rather than
any issues relating to statistics. It is at this stage that the health care provider may
through their attorney “reserve” any issues regarding statistics until a later stage
in the process. This has the effect of inserting the issue of possibly improper statis-
tics into the agenda, but avoids having to go into detail, which in any case usually
is impossible at this time due to lack of information, as we shall see.
At this stage in the process, the health care provider is unlikely to have
even the barest description of any statistical process that has been used against
them.
When the contractor reports back the redetermination, § 405.956(b)(1–10)
specifies the information to be provided in explaining the results. It is more de-
tailed in nature and includes “[a]n explanation of how pertinent laws, regulations,
coverage rules, and CMS policies apply”. Again, this part of the appeals process
usually ignores statistical issues and focuses almost exclusively on problems of
medical necessity.
3.2 The Reconsideration by a Third Party

If the health care provider is unsatisfied with the redetermination, then the process
moves to the next level.
Litigation costs continue to rise. The amount the health care provider
has expended by this point is relatively large in comparison with any additional
costs required to continue the appeal another notch. Taking the case to the next
level almost is automatic.
This next level of the appeal is called the reconsideration. In contrast to the 180 Days
first level, here an independent entity is consulted. These entities are called Quali-
fied Independent Contractors (QIC). These organizations “have their own physicians

and other health professionals to independently review and assess the medical ne-
cessity of the items and services” at issue in the appeal.
The reconsideration tends also to be focused almost exclusively on the
issue of “medical necessity”. The health care provider is given yet another chance
to back up their pleading by submitting additional information to the QIC.
What may be submitted is documented in § 405.966(a–b). Of particu-
lar note is the “use it or lose it” clause found in § 405.966(a)(2) which warns that
evidence not submitted at this time may be precluded from “subsequent consid-
eration of that evidence”.
The reconsideration is performed by a QIC acting as a neutral third party.
After it makes a decision, § 405.976(b)(1–11) specifies the information that must be
reported back to the health care provider. It is similar to what is needed for a re-
determination except there is additional detail needed including “an explanation
of the medical and scientific rationale for the decision”. (§ 405.976(b)(4))
It is of note that at this stage in the appeals process, the CFR still does not
refer to statistical methodology. As a result, the substance of the process still tends
to remain focused on medical necessity and related issues. Although it is possible
at this stage to challenge the statistical side of the issue, in practice this is not well
advised, since QICs in general are not designed to be as proficient on the statistics
side.
Nevertheless, a prudent strategy for the attorney here is to request more
information on the statistical side. The type of data requested would include items
such as:
• More details regarding the sampling process;
• Electronic spreadsheets showing the calculations;
• A fuller explanation of the statistical method employed; (some experts refer

to this as the “experimental design”.)
• Details regarding any probe sample that was taken during the analysis.
It has been our experience that these requests for more detailed informa-
tion on the statistical side frequently will result in an incomplete response consist-
ing only of selected data. See page 24. However, the providing by the contractor of
incomplete data puts the health care provider in a stronger position to file the next level in
the appeal.

3.2.1 Timing
The QIC generally will provide a decision within 60 days. The timing is important.
Note that the deadline for the redetermination by the QIC is 120 days, and for the 1-Year
reconsideration 180 days. As a consequence, when the 60 days to get a response Mark
is added on top, then an entire year already has passed from when the health care
provider received for the first time notice of the overpayment demand.
We are not sure what percentage of appeals reach this stage, but it is
reasonable to expect that as calendar stretches out, the number of health care
providers having the determination and ready funds to continue may tend to thin
out.
3.2.2 QIC Results
In the cases we have examined, it is not uncommon for the QIC to “give a little”
and overrule the contractor on a few of the claims. When this is done, the result
is “partially favorable, partially unfavorable”. We have never encountered a re-
consideration that completely reversed the previous rulings, and quite a number
are “unfavorable”, meaning that nothing has changed, and the full overpayment
demand stands.
There is a disturbing tendency for some attorneys at this point to advise
the health care provider to compromise and “go with what we’ve got from the
appeal”. If as a consequence of the appeal thus far the amount of reduction in the
overpayment demand is great enough in comparison to the legal fees billed, this
to many may appear to be a good deal. “Get out while you are ahead.”
Nevertheless, in every case we have examined, the statistical methods
used are so faulty that it would seem well worth it for the health care provider to
continue the fight.7
7 Up to this stage, the bulk of the information at issue generally still concerns the medical
necessity issue and not statistics. One hypothesis to entertain is that since the process is dominated
by medical necessity issues, it is much less common to involve statistics in the litigation. As a
consequence, the CFR and implementing regulations from the PIM were not designed to cope
with arguments regarding statistical methodology. There may be an indication of this lacunae in
the discovery process, which is more restricted than the general Federal Rules of Civil Procedure
(FRCP). See page 21.

3.3 Appeal to Administrative Law Judge (ALJ)

After reconsideration, the health care provider has 60 days to request a hearing
before the Administrative Law Judge (ALJ). 60-Days
The Medicare appeals process differs from a standard Federal Court case
in important ways. It is based on administrative law. First, any hearing before the
ALJ will not have an adversary; Second, discovery of information is not automatic
— the ALJ must rule on what additional information may be submitted into the
record.8 If the ALJ makes a decision to accept the new information you wish
to submit, then there is a 10 day time frame to do this. The days are measured
beginning on the day the health care provider receives a notification that a hearing
will be granted. The ALJ is required to give at least a 20 day window between the
notification of the hearing and when it will take place.
The request for an appeal is not exactly a legal brief, but the health care
provider is given an opportunity to submit a detailed set of reasons why it still
disagrees with the reconsideration or redetermination. In addition, there is an
opening to provide additional evidence.
The ALJ hearing is not an adversarial process. Most often, the ALJ simply
listens to the information being presented, and asks a few questions of the parties
involved.
If the health care provider has retained an expert in statistics to help re-
fute the contractor’s analysis, then this expert should be brought into the hearing,
qualified, and then given the opportunity to submit an explanation of the prob-
lems they have discovered.
The submission of an expert report that refutes the statistics generated 42CFR
by the contractor is a useful tool that facilitates heightened awareness of this issue 405.1036(c)
for the ALJ. Generally, a challenge to the statistics will focus on several key points:
1. Targeting. The health care provider is being compared to a group of other
health care providers that are not its peer, and thus its billing is not out of
line as suggested, and it should not have been targeted in the first place.9
2. Sample. The sample that forms the basis of the analysis is not valid. For
example, it is too small, or there are some other problems with it, such as a
faulty “experimental deign”.
8 There is an amount in controversy floor of $120 dollars.
9 We know of no cases where this argument has prevailed, but we also know of no case where
it has been vigorously pursued.

3. Extrapolation. The extrapolation is not valid because the sample was not
valid in the first place, or because the contractor somehow violated the Medi-
care Program Integrity Manual (PIM).
4. Documentation. The documentation of the statistical method provided to the

health care provider is incomplete, and as a result it is impossible to verify
any calculations that have been made,10 and such poor documentation is a
violation of the PIM.
3.3.1 Additional Demands
The health care provider at this point must make a detailed demand for discovery
of additional information from the Medicare contractor.
In a few cases, the ALJ has examined the expert report regarding the
statistics and then as a consequence issued an order for the Medicare contractor
to provide more detailed information regarding the methodology they used. This
may force the contractor to expose more details about how they conducted their
work. On the other hand, the timing involved is quite strict and this occasionally
may cause untoward problems. In several cases, for example, the response of
additional data to be supplied by the contractor has come back at the last minute,
making it more difficult for the health care provider in a timely manner to analyze
the new information and re-issue an updated expert report.
The most important information that can be provided by the contractor
is the actual spreadsheets used for conducting the sample and performing the
extrapolation, but these rarely are produced. See page 24.
3.3.2 The ALJ’s Options
The ALJ can issue a favorable or unfavorable opinion. They also have the option
of remanding the case back to the QIC with instructions to take into consideration
the new information regarding the statistics (or anything else).
10 If this argument prevails, the ALJ usually will not rule automatically in favor of the health
care provider, but instead may demand the documentation from the contractor, or in some cases
remand the case.

3.4 Further Appeals

Beyond the ALJ, there are two additional levels of appeal that the health care
provider may pursue. These are:
1. The Medicare Appeals Council (MAC). The health care provider can appeal the
ALJ’s decision to the Medicare Appeals Council (MAC).11
2. Federal District Court. If the health care provider disagrees with the MAC’s
decision, it may proceed to Federal District court.
Although these may be critical parts of the overall appeals process, they are be-
yond the scope of this paper. In Figure 1 on page 33, this part of the process is
marked as terra incognita. For the time being, we are assuming that all issues of
eDiscovery and statistics will be resolved as the case reaches the ALJ level. In ad-
dition, the number of cases that go up to these levels is very small in comparison
to those resolved at the preceding levels.
fff
4 Health Care Provider Rights in Cases of Statistical Malfea-

sance
The health care provider wishes to base at least part of their appeal on the lack of
a valid statistical analysis. In order to compose this argument, an expert working
on their behalf, must be able to examine in detail all of the statistical work that was
the basis of the initial sampling and downstream extrapolated refund demand.
The PIM provides guidelines on the amount and types of information
that the auditor must provide in its explanation to the health care provider. The
specifications for this are in § 3.10.4.4.1 “Documentation of Universe and Frame”.
Of particular concern is the statement:
“Sufficient documentation shall be kept so that the sampling frame

can be re-created, should the methodology be challenged.” (3.10.4.4.1)
11
The MAC will not review either an ALJ’s remand of a case to the QIC, or an instance where
the ALJ has affirmed an earlier QIC’s dismissal of a request for reconsideration.

There is a similar stipulation regarding worksheets, which in all cases refers to

electronic spreadsheets.
“The . . . contractor . . . shall maintain documentation of the review
and sampling process. All worksheets . . . shall contain sufficient infor-
mation that allows for identification of the claim or item reviewed.”
(emphasis added) (PIM 3.10.4.4.3)
Finally, there is a requirement that “each step in the overpayment calculation
[must be] clearly shown”. (3.10.4.4.4)
The PIM demands that each item in the review is documented in the
worksheet, and as well that how any overpayment calculation was made also
should be visible. Since the overpayment calculation rests upon a valid sample,
this means all information regarding the sampling process also must be shown.
In sum, the health care provider has a right to have all of the information “Show
necessary to validate whether or not a statistical method validly was applied. The Your
Work”
contractor is required to “show their work”.
A corollary of this is that should the contractor fail to provide complete
information regarding all of the statistical work, then the health care provider is
not being given fair due process, and the PIM is not being followed.
4.0.1 Due Process and Targeting
The PIM 3.2(A) specifies that “Contractors must document their reasons for select-
ing the provider for the error validation (probe) review.” The PIM also suggests
several reasons why a health care provider may be targeted including “high vol-
ume; high cost; dramatic change; adverse impact on beneficiaries; and/or prob-
lems which, if not addressed, may escalate”. (PIM 3.2(B) para. 2)
It might be possible for contractors to develop quantitative information
that identifies a certain type of health care provider that is most likely to cave
in and pay back the overpayment demand without contesting it, or to find those
health care providers that are probably unable to finance an appeal. In addition, it
is possible by review of historical records for the contractor to find various “sweet
spots” in the auditing “market”, and thus maximize the amount of revenue they
recover as a percentage of the amount of resources they exhaust carrying out audit
activities.
Although the data analysis used to target the health care provider in the
first place is a critical part of the process, there does not appear to be any appeal

right. This means that the health care provider could possibly be targeted as a
result of some type of biased profiling, but if this were the case, it could not be the
basis of an appeal.
5 eDiscovery
The first chance for substantial discovery of information that may tend to show
the mis-use of statistics (sampling, extrapolation) comes in preparation for the ALJ
hearing.
Up to that point, several factors tend to inhibit the full disclosure of in-
formation by the Medicare contractor and the QIC. These include:
1. Minimum Statutory Requirements. The statutes and the Program Integrity
Manual (PIM) suggest that the health care provider is entitled to a full ex-
planation of how the extrapolation was done. In practice, these explanations
typically omit critical information, but do provide superficially a narrative
regarding how the overpayment demand was obtained.
2. Focus on Medical Necessity. Historically, the bulk of appeals have been based
on arguments about medical necessity, but not on the viability of the under-
lying statistics supporting the sample and extrapolation.
3. Proprietary Work. Although none of the statistical work performed by either

the Medicare contractor or the QIC can be considered a proprietary trade
secret, in practice it appears the almost universally exhibited behavior makes
it appear so. In a practical sense, the details of how the statistics are compiled
never are revealed unless the ALJ orders it.
Once the ALJ stage of the appeal process begins, then the health care provider
may exercise their rights and attempt to force the revealing of more information.
In terms of the types of evidence that may be viewed, the law is flexi-
ble. For example, the “ALJ may receive evidence at the hearing even though the
evidence is not admissible in court” (42 CFR 405.1036(e))
If the health care provider has attempted to get discovery of the docu- Subpoena
ments, and filed a motion for this that has been granted by the ALJ but still did
not receive the information, then the ALJ may issue a subpoena to compel disclo-
sure. The ALJ also can based on their own judgment and without a request from
the health care provider issue a subpoena. (42 CFR 405.1036(f))

If the QIC or contractor (or other party) wishes to appeal the subpoena,
then it must argue based on a privilege such as undue burden, case preparation,
or confidentiality. If this occurs, then there can be an immediate appeal of the
subpoena to the MAC.
Only under circumstances that we have never encountered would these
reasons be enough to stop the subpoena. Here are the reasons:
5.1 Undue Burden

It is less of a burden for the contractor or QIC to provide a copy of the electronic
spreadsheet, than to do what now is the general practice. In every case we have
examined, the evidence regarding the statistics that is provided by the contractor
consists of paper print-outs of selected portions of the worksheet, but the electronic
copy of the spreadsheet itself is retained. There are two burdens that using a paper
version places upon the overall process:
1. First, the contractor must assign staff to take the time to define which ex-
cerpts from the spreadsheet will be handed over, and then to print out and
verify this information before passing it onwards;
2. Second, although this is not strictly relevant to the “undue burden” argu-
ment, nevertheless, by providing paper-only copies of excerpts of the spread-
sheet, a substantial burden is placed upon the health care provider who must
use this information to determine if the statistical analysis is valid. For ex-
ample, it usually is required manually to re-enter all of the supplied paper-
based data into a redundant electronic worksheet to be used by the health
care provider.
Not only is this process more expensive, it also creates the potential problem of
verification between the original contractor spreadsheet and the newly created
“copy”. All of this extra and costly work on both sides could be avoided if the
contractor simply would hand over the electronic spreadsheet immediately, in-
stead of waiting to be forced to do this by the ALJ.
5.2 Confidentiality
In these proceedings, concerns about confidentiality pertain primarily to informa-
tion regarding individual patients and not to the work processes of the contractor.

As a matter of course, great care is taken to ensure that all medical “name-linked”
records are treated with appropriate and documented security measures. Any CD
containing data usually is encrypted and given a password so that should it be
lost, only an authorized person could read its contents. In cases where the QIC or
contrtactor provide print-outs of the statistical materials, it is not uncommon to
see patients identified only by a number, and not by name.
In general, everyone on all sides recognizes the importance of protection
of medical information on patients.
On the other hand, concerns regarding confidentiality of contractor pro-
cesses should not be used as a smoke screen to hide the underlying statistical
method used to select the sample of cases analyzed and to calculate the estimated
overpayment demand.
What we have found is that the information held back by the contractor
and QIC most often concerns the “nuts and bolts” of the statistical calculations.
These include items such as:
• How the random numbers for the sample were generated and what exactly
were the numbers used;
• Which formulas were used, and where, and what exact cell references (on
the spreadsheet) were used as inputs and outputs of these formulas;
• The existence of any intermediate calculations, including previously undoc-

umented analyses of the data; and
• So-called “macro-data”, such as the editing history of the electronic records,

including a verification that the documents have not been modified at any
time after it was realized that an appeal was imminent.
Not one of these withheld items have even the slightest relation to providing con-
fidentiality of patient data. Instead, they all relate to transparency — a look at
exactly how various mathematical operations were performed.
As a consequence, confidentiality as the second reason to withhold a sub-
poena should not be valid, simply because this safeguard is designed to protect
information (and data) that is not at issue in a forensic review of the statistical
analysis.

5.3 Case Preparation

The third reason to withhold a subpoena is to protect information used in prepar-
ing the case. This general protection pays homage to the principle that the notes
of an attorney are kept out of bounds of the discovery process.
Here the issue revolves around whether there are any significant case
preparation notes of concern. Since the Medicare appeals process is generally not
adversarial in nature, it is unlikely that council for the contractor or QIC would be
involved, unless there were a charge of fraud or some other malfeasance.
As a consequence, case preparation rarely if ever stands in the way of
creating a subpoena concerning the seeking for a fuller explanation of how the
statistical work was done.
In sum, the reasons of undue burden, case preparation or confidentiality
should not stand in the way of the health care provider getting full disclosure of
all statistical work done by the contractor on their case.
What happens usually is quite different....
6 Evasive Actions by Contractors & QICs to Cover Up

Their Statistical Work
Based on our experience,12 the general response by the contractor many times is
not sufficient. That is, they tend not to provide sufficient information to the health
care provider enabling sufficient verification of their underlying statistical work.
The following are only a few of the problems typically found with con-
tractor provisioning of needed information. The list is not complete, but should
provide a flavor of what is to be expected.
The general pattern can be summarized as follows:
1. Provide only the minimal amount of information specified by the CFR, and
when there is an interpretation needed regarding what must be supplied,
err on the side of providing less rather than more information.
2. Do not provide details regarding the statistical procedures used except in the
most general and high-level sense, and then if asked for further information,
provide only the minimum amount possible.
12
But not based on any survey or systematic review of contractor provided information, which
in any case is not possible to obtain.

3. Focus more on asserting what statistical work was done rather than showing
what was done.
4. If forced to provide even greater detail regarding the statistics used, focus
on providing written explanations rather than providing electronic data that
can be studied by a neutral external expert.
5. If forced to provide spreadsheets and detailed information, attempt to pro-

vide print outs, so that the electronic copy of the calculations do not get out.
6. If required to provide electronic spreadsheets, attempt to “protect” various

parts of the spreadsheet so it may not be viewed in its entirety by the health
care provider.
Below are a few aspects of the associated problems encountered by the health care
provider in a typical case.
6.1 Incomplete Explanation

In more or less every case we have examined, the explanation given for the refund
demand is woefully inadequate, including the extrapolation of what is due back.
It is not uncommon to see entire parts of the analysis cut out altogether.
6.2 Misleading Explanations

Another common pattern we see is the use of extensive reference to complicated
statistical formulas, when they are either not actually used in the analysis, or if
they are, then there is zero explanation of how they were used.
In many cases, the auditor simply provides a photocopy of six or seven
key statistical formulas from a popular textbook.
The implication is that these formulas and their underlying valid statis-
tical methodology have been used in the estimation of how much is to be clawed
back. However, upon subsequent more detailed analysis, we have never seen
even one case in which all of the formulas were used.
The implication is clear. The auditor has put in these formulas as a type
of “window dressing” to lead the reader and trier of fact to believe that underlying
their refund demand lies a sophisticed statistical methodology.

In reality, it is rare that the judge can read or clearly understand these
mathematical notions, and the same can be said of the judge’s staff, or most if
not all of the attorneys involved. So the result is what would be expected: these
formulas receive little scrutiny, and accomplish their intended purpose of giving
the impression of a solid statistical analysis when in fact none may have been
performed.
6.3 Failure to Show Work

In some cases, the contractor provides a partial explanation, but omits to explain
critical steps in the analysis. For example, they may claim to be using certain
formula, but never show how the formula were used.
Claiming to use valid statistical formulas is not the same as actually showing
that you did it.
The contractor should be required to show their work. Merely providing
a description of what they claim was done is not sufficient. Many contractors
believe the following “explanation” is adequate:
“The extrapolated refund was calculated based on use of accepted sta-

tistical methodology.”
It is not possible to get behind the motivations for this, but in a number
of cases, we have found that the contractor was using the formulas improperly.
In several cases, fake or manufactured data early on in the process was inserted Manufactured
into critical formulas. The effect was to generate an extreme bias later on in the Data
analysis — a bias against the health care provider.
Only by forcing the contractor to show their work can such malfeasance
be exposed.
6.4 Skipping Required Steps

The PIM requires that no matter what approach is taken, the contractor must use
a valid statistical procedure.
Unfortunately, it has been even more common to see evidence indicating
the contractors have skipped entire steps in the procedures specified by the PIM.
For example, there is a pronounced proclivity to take liberties in the sampling,
particularly as regards the underlying error rates.

We find in numerous cases that instead of taking a probe sample to de-

termine the underlying error rate in the universe of claims, critical assumptions
are made regarding this error rate, even before even a single claim has been sub-
jected to analysis. This is a remarkable lapse of due process, and it completely and
without doubt invalidates the statistical analysis and conclusions derived from it.
In practice, only after the skipping of critical steps is identified, does it
become easy for the trier of fact to recognize these omissions. However, the con-
tractors appear to have developed the art of writing up a convincing explanation
for their work in such a way that on the first reading these flaws are not noticed
except by a trained statistical expert.
6.5 Providing Large Amounts of Irrelevant Information

We also have seen an unconscionable trend by some contractors to burden the
appeal with an excessive amount of completely irrelevant information. For exam-
ple, in one case, as part of their justification for the statistical procedure used, they
included more than 100 pages of a photocopied software manual!
We never learned what the response from the ALJ looked like, but can
only imagine. Was anyone supposed to actually read this material? Was it meant
to cover up the lack of a true explanation? One can only speculate as to the out-
come as the Judge realized this pile of irrelevant information had been inserted
into the record, thus requiring a careful but unnecessary yet time-consuming read-
ing.
fff
There are numerous places in the process, and within the confines of
statistical procedure, where an analysis can go wrong. Although this list is not
conclusive, here are the principal problems we have found in contractor work:
1. Misuse of Formulas. Instances in which although the correct formulas are em-
ployed, they are misused by feeding into them the wrong data. For example,
a formula may call for “Data A” but the contractor substitutes “Data B”.
2. Inadequate Sample Size. Statistical methodology provides many ways to work

through a problem, but one of the most fundamental errors is to use a sample
size that is not right. In the cases we have reviewed, the sample sizes have
been not slightly too small, but substantially too small. In one recent case, the

sample size was more than 2,000% too small, using as the base calculation
the data created by the contractor themselves.
3. Lack of a Probe Sample. We have yet to come across a case in which the contrac-
tor performed a valid probe sample, or even any type of probe sample at all.
Remember that the formalized statistical procedure of the “Probe Sample” is
completely different from the “Probe Review”, the latter of which is used for
determining only the overall advisability of a general audit, and not for any
work leading to the calculation of the overpayment demand. See Page 9. It
may surprise you that skipping this fundamental statistical procedure may
be so prevalent.
In no way may this type of work past muster, even by the “loosest” stan-
dards.
When a contractor engages in this type of behavior, it may not be atypi-
cal. From what we can gather, this behavior is a product of years of practice and
habit. One reason it may at this point appear anomalous is due to the renewed fo-
cus on the underlying statistical methodology. It appears that historically, almost
all of the focus during appeal litigation converged on the details of individual
patient cases and the complex problems associated with understanding medical
necessity.
7 eDiscovery in the Federal Rules of Civil Procedure

Since Medicare hearings are embedded in administrative law, they are not consid-
ered to be adversarial. There is no opposing party, and the ALJ is concerned only
with fair application of the CFR and PIM, as well as any other related rules.
As a consequence, the complexities associated with an adversarial hear-
ing generally are not present.
Simply for discussion purposes, lets make an assumption: Any eDiscov-
ery rules that are accepted in the Federal Rules of Civil Procedure may inform any
decisions made by the ALJ as regards the right of the health care provider to see
and inspect a full accounting of any statistical calculations made.
Here are a few facts about eDiscovery to keep in mind:
1. The general trend is that the amount of relevant information kept only in
electronic form is much greater than information and potential evidence
kept in hard (paper) copy.

2. eDiscovery allows the collection of not only the obvious information being
sought but also of information about the information, e.g., records showing
how and when electronic files were altered, and who did it. This information
sometimes is called “macro-data”.
3. Unless there is some specific reason, it is general practice for the party sup-
plying the eDiscovery information to hand over the discoverable data in the
same form in which it is kept, unless there is some hardship involved. Here, in
the Medicare audit world, this would mean producing the electronic spread-
sheets.
The transition from “paper-based” rules of discovery to “electronic dis-

covery” (eDiscovery) took effect at the end of 2006 after the Supreme Court ap-
proved the modification of the FRCP. One of the drivers for the reform of the
discovery rules was the rapidly rising cost of “document review”. It was many
times more efficient to “go electronic” instead of relying on the artifact of paper.
We review below a few of the principal changes.
Case Management. Electronically stored information is to be addressed in pre-

trial conferences, including claims of privilege. Rule
16(b)(5–6)
Duty of Disclosure. “Electronically stored information” is a category required in
initial disclosure, except for some data where there would be “undue costs” 26(a)(1)(B)
involved in producing it. If there are claims of privilege, these also are ad- 26(b)(2)(B)
dressed early on. 16(b)(5–6)
Interrogatories. Electronic data can be included as a business record for an in-

terrogatory. This is not particularly relevant for Medicare overpayment de- 33(d)
mand cases since interrogatories are not allowed in the administrative pro-
ceedings.
Form of eDocuments. Electronically stored information (not ”documents”) are

included in the definition of “documents”. The touchy issue of what “form” 34(a)
to produce the documents in is solved by simply stating the default form of
how the information “is ordinarily maintained”. Production is required in
only one form. 34(b)(ii–
iii)
Deleted Data. There is no sanction if data has been erased in the “routine, good-
faith operation of an electronic information system”. 37(f)

Subpoena. The subpoena rules were changed to include eDocuments. 45
Inadvertent Disclosure. A proposed Rule 502 would require further disclosure

of related information if there is an inadvertent disclosure of something. 502(a)
8 Application to Medicare Appeals

As mentioned earlier, there is no attempt here to insist on application of FRCP to
the administrative law at the heart of the Medicare Appeals process. Nevertheless,
based on the FRCP rules regarding eDocuments and the eDiscovery process, we
can suggest the following:
1. Case Management. It may be good practice to insist from the very beginning
that the Medicare contractor hand over all information related to how the
statistical calculations were done, and this includes not only the basics of the
explanation called for in the PIM, but also all related information including
eDocuments.
2. Insist on Electronically Stored Information. It might be well to point out that

by adopting the regular practice of simply printing out paper copies of se-
lected excerpts of the statistical calculation process, not only is the contrac-
tor withholding important information, but also it is substantially raising
costs, including costs for the contractor themselves, but also for the health
care provider who must expend additional and essentially unnecessary re-
sources in handling the paper-based information.
3. Avoid “Form Shifting”. It may be useful to argue that by shifting the form
of the electronically stored information into paper, it not only causes hard-
ship, but the “general practice” is to hand over information in that form in
which it is “ordinarily maintained”. It should also be argued that by chang-
ing the form of the information, the contractor is leaving out both significant
documentary information but also is making it possible to hide the accompa-
nying “macro data” such as the editing history of the (electronic) document
and relevant dates, e.g., to verify if the document has been changed at any
time after the initial overpayment demand.

4. Subpoena. As mentioned earlier, the ALJ has the power to issue the sub-
poena. The rules from the FRCP would imply that asking for eDocuments
now should be considered to be more or less “routine”.
fff
9 Closing
Medicare overpayment demands may pose a bankruptcy threat to a health care
provider. There is a standard process available for appeals. In that process, the
health care provider is entitled to review all details of how the Medicare contractor
reached their conclusion. Too often the litigation bogs down in complex and time-
consuming arguments about medical necessity, and ignores the appeal-worthy
issues regarding the statistical analysis used. There appears to be evidence of
widespread abuse of the statistical procedures specified in the PIM. By using a
vigorous eDiscovery strategy, the attorney for the health care provider may be able
to uncover a range of problems including misuse of process, inadequate sampling
and deceptive or misleading justifications. When brought to light this may result
in an even better outcome for the health care provider in their appeal.

About the Author

Edward M. Roche, Ph.D., J.D. is the Director of Scientific Intelligence for Barr-
aclough Ltd.. He is a member of the California Bar and the American Health
Lawyers Association.
Barraclough Ltd. is an advisory firm that provides litigation support,

including forensic statistics.
Barraclough Ltd.
New York, NY 10022
Tel: (646) 416-6592
Web: http://www.barracloughltd.com
http://www.medicare-audit-defense.com
Email: info@barracloughltd.com
For a free “eBook” copy of this document, visit the website.13
13To read an eBook you must have an eBook reader such as the Amazon Kindle, SONY eBook,
Barnes & Noble Nook or Apple iPad.

FEDERAL DISTRICT TERRA

COURT INCOGNITA
MEDICARE TERRA
APPEALS COUNCIL INCOGNITA
FINAL EXPERT
REVIEW OF E-
ADMINISTRATIVE REPORT
DISCOVERY
LAW JUDGE BASED ON E-
MATERIALS
DISCOVERY
REMAND
3
MORE
EXPERT
RECONSIDERATION DETAILED
REPORT ON
DISCUSSION
STATISTICAL
QIC OF
DEFICIENCIES
STATISTICS
REDETERMINATION INITIAL
DISCUSSION
CONTRACTOR OF RESULTS
Figure 1: eDiscovery of Statistical Methodology and Appeal of Medicare Over-

payment Demands.


Understanding Doctor'S Rights: Effective Use of E-Discovery: Edicare Verpayment Udits

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Understanding Doctor'S Rights: Effective Use of E-Discovery: Edicare Verpayment Udits

Uploaded by

Copyright:

Available Formats

M EDICARE O VERPAYMENT A UDITS

UNDERSTANDING DOCTOR’S RIGHTS:

Keywords: RAC; Recovery Audit Contractor; CMS; Medicare Fraud; Au-

∗ BarracloughLtd. is a New York State corporation that specializes in providing Research

Abbreviations & Sources

1 The Specter of a Medicare Audit

2 Types of Statistical Analysis Leading to the Overpay-

2.1 Initial Targeting of Health Care Provider

2.1.1 The Error Validation Review

more detailed look at a small number of claims they have submitted.

2.2 Sample Analysis

1. Selecting the provider or supplier;

2. Selecting the period to be reviewed;

4. Designing the sampling plan and selecting the sample;

6. Estimating the overpayment.

The contractor is given a most liberal flexibility in choosing which sta-

“If a particular probability sample design is properly executed, i.e., defin-

claims that have been identified and studied in detail.

2.3 The Extrapolation

2.4 The Revision Problem

3 The Structure of the Appeals Process

3.1 The Redetermination by the Medicare Contractor

The responses provided by the contractor vary according to the stage

3.2 The Reconsideration by a Third Party

• More details regarding the sampling process;

• Electronic spreadsheets showing the calculations;

• A fuller explanation of the statistical method employed; (some experts refer

3.2.2 QIC Results

3.3 Appeal to Administrative Law Judge (ALJ)

4. Documentation. The documentation of the statistical method provided to the

3.3.1 Additional Demands

3.3.2 The ALJ’s Options

3.4 Further Appeals

4 Health Care Provider Rights in Cases of Statistical Malfea-

“Sufficient documentation shall be kept so that the sampling frame

There is a similar stipulation regarding worksheets, which in all cases refers to

4.0.1 Due Process and Targeting

3. Proprietary Work. Although none of the statistical work performed by either

5.1 Undue Burden

• The existence of any intermediate calculations, including previously undoc-

• So-called “macro-data”, such as the editing history of the electronic records,

5.3 Case Preparation

6 Evasive Actions by Contractors & QICs to Cover Up

5. If forced to provide spreadsheets and detailed information, attempt to pro-

6. If required to provide electronic spreadsheets, attempt to “protect” various

6.1 Incomplete Explanation

6.2 Misleading Explanations

6.3 Failure to Show Work

“The extrapolated refund was calculated based on use of accepted sta-

6.4 Skipping Required Steps

We find in numerous cases that instead of taking a probe sample to de-

6.5 Providing Large Amounts of Irrelevant Information

2. Inadequate Sample Size. Statistical methodology provides many ways to work

7 eDiscovery in the Federal Rules of Civil Procedure

The transition from “paper-based” rules of discovery to “electronic dis-

Case Management. Electronically stored information is to be addressed in pre-

Interrogatories. Electronic data can be included as a business record for an in-

Form of eDocuments. Electronically stored information (not ”documents”) are

Subpoena. The subpoena rules were changed to include eDocuments. 45

Inadvertent Disclosure. A proposed Rule 502 would require further disclosure

8 Application to Medicare Appeals

2. Insist on Electronically Stored Information. It might be well to point out that