1

METHODS OF PUBLIC POLICY ANALYSIS

Policy analysis is "determining which of various alternative policies will most achieve a given set of goals in light of the relations between the policies and the goals". However, policy analysis can be divided into two major fields. Analysis of policy is analytical and descriptive²i.e., it attempts to explain policies and their development. Analysis for policy is prescriptive²i.e., it is involved with formulating policies and proposals (e.g., to improve social welfare). The area of interest and the purpose of analysis determine what type of analysis is conducted. A combination of policy analysis together with program evaluation would be defined as Policy studies. Policy analysis is methodologically diverse using both qualitative methods and quantitative methods, including case studies, survey research, statistical analysis, and model building among others. One common methodology is to define the problem and evaluation criteria; identify all alternatives; evaluate them; and recommend the best policy agenda per favor.

QUALITATIVE RESEARCH: Qualitative research is a method of inquiry employed in many different academic disciplines, traditionally in the social sciences, but also in market research and further contexts. Qualitative researchers aim to gather an indepth understanding of human behavior and the reasons that govern such behavior. The qualitative method investigates the why and how of decision making, not just what, where, when. Hence, smaller but focused samples are more often needed than large samples. In the conventional view, qualitative methods produce information only on the particular cases studied, and any more general conclusions are only propositions (informed assertions). Quantitative methods can then be used to seek empirical support for such research hypotheses. This view has been disputed by Oxford University professor Bent Flyvbjerg, who argues that qualitative methods and case study research may be used both for hypotheses-testing and for generalizing beyond the particular cases studied. Data collection Qualitative researchers may use different approaches in collecting data, such as the grounded theory practice, narratology, storytelling, classical ethnography, or shadowing. Qualitative methods are also loosely present in other methodological approaches, such as action research or actor-network theory. Forms of the data collected can include interviews and group discussions, observation and reflection field notes, various texts, pictures, and other materials. Qualitative research often categorizes data into patterns as the primary basis for organizing and reporting results. Qualitative researchers typically rely on the following methods for gathering information: Participant

The ways of participating and observing can vary widely from setting to setting. or setting. An example of applied ethnographic research. 4. Unstructured Interview. Historical research helps us in answering questions such as: Where have we come from. motivations and emotions. used by a researcher to understand how people communicate and develop symbolic meanings. Ethical Inquiry. It includes the study of ethics as related to obligation. observation and surveys. In participant observation researchers typically become members of a culture. including quantitative data. as perceived by the study population. review of records. 6. 2. Some distinctive qualitative methods are the use of focus groups and key informant interviews. the aim is for the researcher to gain a closer insight into the culture's practices. Survey items are piloted on study participants to test the reliability and validity of the items. The focus group technique involves a moderator facilitating a small group discussion between selected individuals on a particular topic. interviews. used for investigating cultures by collecting and describing data that is intended to help in the development of a theory. Non-participant Observation. identify ethics. it uses a variety of data sources. and adopt roles to conform to that setting. is conducted by field experts within the boundaries of a specific field of study or profession. In doing so. Phenomenology. Structured Interview. 7. describes the ³subjective reality´ of an event. examines the foundations for a science. Participant observation is a strategy of reflexive learning. analyses the beliefs and develops ways to specify how a knowledge base should change in light of new information. in order to clarify definitions. One traditional and specialized form of qualitative research is called cognitive testing or pilot testing which is used in the development of quantitative survey items. This method is also called ³ethnomethodology´ or "methodology of the people". not a single method of observing. duty. rights. group. it is the study of a phenomenon. Critical Social Research. and Analysis of documents and materials. allows one to discuss past and present events in the context of the present condition. based or ³grounded´ in the observations or data from which it was developed. It is argued that the researchers' ability to understand the experiences of the culture may be inhibited if they observe without participating. 5. choice etc. Historical Research. Reflexive Journals. Semistructured Interview. where are we. who are we now and where are we going? Grounded Theory is an inductive type of research. Philosophical Research. right and wrong. the best qualified individual in any field of study to use an intellectual analyses. . 8. and allows one to reflect and provide possible answers to current issues and problems. is the study of a particular culture and their understanding of the role of a particular disease in their cultural framework. or make a value judgment concerning an issue in their field. This is a particularly popular method in market research and testing new initiatives with users/workers.2 Observation. Foundational Research. an intellectual analysis of ethical problems. 3. Ethnographic Research. Field Notes. In the academic social sciences the most frequently used qualitative research approaches include the following: 1.

mathematical or computational techniques. Qualitative research. The researcher analyzes the data with the help of statistics. analysis and interpretation of observations for the purpose of discovering underlying meanings and patterns of relationships. Quantitative data is any data that is in numerical form such as statistics. etc. quantitative research is often contrasted with qualitative research which is the examination. The objective of quantitative research is to develop and employ mathematical models. which contrast qualitative research methods. Qualitative methods produce information only on the particular cases studied. narrow question and collects numerical data from participants to answer the question. and any more general conclusions are only hypotheses. Kuhn (1961. and less frequently in anthropology and history. The researcher looks for themes and describes the information in themes and patterns exclusive to that set of participants. and political science. it has been argued that the two go hand in hand. economics. Although a distinction is commonly drawn between qualitative and quantitative aspects of scientific investigation. Quantitative research is generally made using scientific methods. on the other hand. theories and/or hypotheses pertaining to phenomena. which can include: ‡ ‡ ‡ ‡ ‡ ‡ The generation of models. In the social sciences. this means that the quantitative researcher asks a specific. the term relates to empirical methods. though this use of the term differs in context. which built on the work of Ernst Heinrich Weber. percentages. p. based on analysis of the history of science. 162) concludes that ³large amounts of qualitative work have usually been prerequisite to fruitful quantification in the physical sciences´. Qualitative research is often used to gain a general sense of phenomena and to form theories that can be tested using further quantitative research. For example. Quantitative research is used widely in social sciences such as psychology. theories and hypotheses The development of instruments and methods for measurement Experimental control and manipulation of variables Collection of empirical data Modeling and analysis of data Evaluation of results In the social sciences particularly. Quantitative methods can be used to verify. Research in mathematical sciences such as physics is also 'quantitative' by definition. which of such hypotheses are true. For . asks broad questions and collects word data from participants. including classifications of types of phenomena and entities. Approaches to quantitative psychology were first modeled on quantitative approaches in the physical sciences by Gustav Fechner in his work on psychophysics. The process of measurement is central to quantitative research because it provides the fundamental connection between empirical observation and mathematical expression of quantitative relationships. in a manner that does not involve mathematical models. sociology. originating in both philosophical positivism and the history of statistics. quantitative research refers to the systematic empirical investigation of social phenomena via statistical. The researcher is hoping the numbers will yield an unbiased result that can be generalized to some larger population. In layman's terms. A comprehensive analysis of 1274 articles published in the top two American sociology journals between 1935 and 2005 found that roughly two thirds of these articles used quantitative methods.3 QUANTITATIVE RESEARCH: In the social sciences.

Associations may be examined between any combination of continuous and categorical variables using methods of statistics. the modern idea of quantitative processes have their roots in Auguste Comte's positivist framework. and when phenomena occurred. non-linear model. Measurement Views regarding the role of measurement in quantitative research are somewhat divergent. social sciences and biology. Measurement is often regarded as being only a means by which observations are expressed numerically in order to investigate causal relations or associations. as seen below: . In opinion surveys. Positivist scholars like Comte believed only scientific methods rather than previous spiritual explanations for human behavior could advance science. Statistical methods are used extensively within fields such as economics. In the field of health. Quantitatively based opinion surveys are widely used in the media. In the field of climate science. researchers compile and compare statistics such as temperature or atmospheric concentrations of carbon dioxide.4 instance. with statistics such as the proportion of respondents in favor of a position commonly reported. Kuhn argued that within quantitative research. where. However. Causal relationships are studied by manipulating factors thought to influence the phenomena of interest while controlling other variables relevant to the experimental outcomes. A fundamental principle in quantitative research is that correlation does not imply causation. why. in the social sciences qualitative research methods are often used to gain better understanding of such things as intentionality (from the speech response of the researchee) and meaning (why did this person/group say something and what did it mean to them?). and also finds applications within the physical sciences. Although quantitative investigation of the world has existed since people first began to record events or objects that had been counted. For example. Quantitative research using statistical methods starts with the collection of data. This principle follows from the fact that it is always possible a spurious relationship exists for variables between which covariance is found in some degree. controlling for other key variables such as exercise.this would require verification. how. Software packages such as SPSS and R are typically used for this purpose. for example. it has been argued that measurement often plays a more important role in quantitative research. respondents are asked a set of structured questions and their responses are tabulated. based on the hypothesis or theory. researchers might measure and study the relationship between dietary intake and measurable physiological effects such as weight loss. such as in statistical mechanics. or by using factor analysis. He argued that such abnormalities are interesting when done during the process of obtaining data. Use of statistics Statistics is the most widely used branch of mathematics in quantitative research outside of the physical sciences. Empirical relationships and associations are also frequently studied by using some form of General linear model. This is because accepting a theory based on results of quantitative data could prove to be a natural phenomenon. validation and recording before the analysis can take place. Positivism emphasized the use of the scientific method through observation to empirically test hypotheses explaining and predicting what. Usually a big sample of data is collected . the results that are shown can prove to be strange.

in which criteria are established for selecting cases from historical records for inclusion in the study. group. and that departure is often enough to start a search (Kuhn. Tree-ring width. In the case of tree-ring width. policies. an empirical inquiry that investigates a phenomenon within its real-life context. and their very neutrality makes them particularly sterile as a source of remedial suggestions. They provide a systematic way of looking at events. or event) stressing developmental factors in relation to context. case study methods involve an in-depth. Case studies may be descriptive or explanatory. different species in different places may show more or less sensitivity to. Quantitative research may involve the use of proxies as stand-ins for other quantities that cannot be directly measured. projects. decisions. events. This field is central to much quantitative research that is undertaken within the social sciences.D. in which criteria are established and cases fitting the criteria are included as they become available. or other systems that are studied holistically by one or more methods. for example. As a result the researcher may gain a sharpened understanding of why the instance happened as it did. 180). Thomas offers the following definition of case study: "Case studies are analyses of persons. In classical physics. periods. p.g. In contrast. The case study is common in social sciences and life sciences. When used in this way. 1961. can include quantitative evidence. Case study A case study is an intensive analysis of an individual unit (e. the theory and definitions which underpin measurement are generally deterministic in nature. They may be prospective. longitudinal (over a long period of time) examination of a single instance or event: a case. during the period of the instrumental record) to determine how much variation is captured. including whether both short and long term variation is revealed. it is likely to yield mere numbers. Case studies lend themselves to both generating and testing hypotheses.. Case study research means single and multiple case studies. or retrospective. is considered a reliable proxy of ambient environmental conditions such as the warmth of growing seasons or amount of rainfall. analyzing information. The latter type is used to explore causation in order to find underlying principles. Case studies should not be confused with qualitative research and they . and reporting the results. collecting data. institutions. Another suggestion is that case study should be defined as a research strategy. The case that is the subject of the inquiry will be an instance of a class of phenomena that provides an analytical frame ² an object ² within which the study is conducted and which the case illuminates and explicates. But numbers register the departure from theory with an authority and finesse that no qualitative technique can duplicate. say. Psychometrics is the field of study concerned with the theory and technique for measuring social and psychological attributes and phenomena. say) only reconstructs a certain amount of the variance of the original record.5 When measurement departs from theory. rainfall or temperature: when reconstructing a temperature record there is considerable skill in selecting proxies that are well correlated with the desired variable. tree-ring width and other climate proxies have been used to provide a semiquantitative record of average temperature in the Northern Hemisphere back to 1000 A. the proxy record (tree ring width. and what might become important to look at more extensively in future research. relies on multiple sources of evidence and benefits from the prior development of theoretical propositions. Although scientists cannot directly measure the temperature of past years. a person." Rather than using samples and following a rigid protocol (strict set of rules) to examine limited number of variables. The proxy may be calibrated (for example. probabilistic measurement models known as the Rasch model and Item response theory models are generally employed in the social sciences.

6 can be based on any mix of quantitative and qualitative evidence. unusual or particularly revealing set of circumstances. Case selection and structure of the case study An average. retrospective study. In clarifying lines of history and causation it is more useful to select subjects that offer an interesting. as opposed to random sampling. or typical. Survey research Survey research a research method involving the use of questionnaires and/or statistical surveys to gather data about people and their thoughts and behaviors. and thereby to offer reasoned lines of explanation based on this rich knowledge of setting and circumstances. It is thus possible to take many routes through this typology. Single-subject research provides the statistical framework for making inferences from quantitative case-study data. Or it may be chosen because of researchers' indepth local knowledge. those which are extreme. there is a distinction to be made between the subject and the object of the case study. One of its early successes was the development of the theory of two-step flow of . Alternatively. When selecting a subject for a case study. then processes are decided upon. a case may be selected as a key case. parallel or sequential. 2. theorybuilding or illustrative). Three types of cases may thus be distinguished: 1. Outlier cases (that is. with. nested study. Key cases Outlier cases Local knowledge cases Whatever the frame of reference for the choice of the subject of the case study (key. 3. the case study through which the theoretical focus. then approaches are delineated (theory-testing. This method was pioneered in the 1930s and 1940s by sociologist Paul Lazarsfeld. Thomas thus proposes a typology for the case study wherein purposes are first identified (evaluative or exploratory)." The case study is sometimes mistaken for the case method. Thus. researchers will therefore use information-oriented sampling. local knowledge). single. or an evaluative. case is often not the richest in information. The object is that theoretical focus ± the analytical frame. multiple. could be viewed and explicated. chosen because of the inherent interest of the case or the circumstances surrounding it. snapshot or diachronic. outlier. an exploratory. theory-building. A case selection that is based on representativeness will seldom be able to produce these kinds of insights. for example. approach and process in the case study. historical unity´ through which the theoretical focus of the study is being viewed. The typology thus offers many permutations for case study structure. the object. with a principal choice being between whether the study is to be single or multiple. deviant or atypical) reveal more information than the putatively representative case. situated between concrete data taking techniques and methodological paradigms. and whether it is nested. the lens. if a researcher were interested in US resistance to communist expansion as a theoretical focus. The initial use of the method was to examine the effects of the radio on political opinion formation of the United States. theorytesting. 2005): "The case study is a research approach. and choices also about whether the study is to be retrospective. but the two are not the same. where researchers have this local knowledge they are in a position to ³soak and poke´ as Fenno puts it. for example. The subject is the ³practical. decisions need to be made about purpose. then the Korean War might be taken to be the subject. Beyond decisions about case selection and the subject and object of the study. This is also supported and well-formulated in (Lamnek.

to suggest new hypotheses for future study. ." This should not be confused with the word statistic. using blocking to reduce the influence of confounding variables. In both types of studies. Instead. and the estimated experimental variability. The basic steps of a statistical experiment are: 1. alternative hypotheses. data are gathered and correlations between predictors and response are investigated. Statistical methods A common goal for a statistical research project is to investigate causality. An experimental study involves taking measurements of the system under study. 4. The difference between the two types lies in how the study is actually conducted. Such people have often gained this experience through working in any of a wide number of fields. organization. The word statistics. Further examining the data set in secondary analyses. Performing the experiment following the experimental protocol and analyzing the data following the experimental protocol. when referring to the scientific discipline. using the following information: preliminary estimates regarding the size of treatment effects. 2. There is also a discipline called mathematical statistics that studies statistics mathematically. and interpretation of data. analysis. The method was foundational for the inception of the Quantitative research tradition in sociology. Statisticians recommend that experiments compare (at least) one new treatment with a standard treatment or control. and in particular to draw a conclusion on the effect of changes in the values of predictors or independent variables on dependent variables or response. Statistics Statistics is the study of the collection. Design of experiments. the effect of differences of an independent variable (or variables) on the behavior of the dependent variable are observed.[4] whose plural is statistics ("this statistic seems wrong" or "these statistics are misleading"). an observational study does not involve experimental manipulation. There are two major types of causal statistical studies: experimental studies and observational studies. A statistician is someone who is particularly well versed in the ways of thinking necessary for the successful application of statistical analysis. In contrast. Planning the research. and randomized assignment of treatments to subjects to allow unbiased estimates of treatment effects and experimental error. Each can be very effective. manipulating the system. as in "Statistics is an art. to allow an unbiased estimate of the difference in treatment effects. Consideration of the selection of experimental subjects and the ethics of research is necessary. At this stage. the experimenters and statisticians write the experimental protocol that shall guide the performance of the experiment and that specifies the primary analysis of the experimental data. It deals with all aspects of this. including the planning of data collection in terms of the design of surveys and experiments. including finding the number of replicates of the study. and then taking additional measurements using the same procedure to determine if the manipulation has modified the values of the measurements. is singular. referring to a quantity (such as mean or median) calculated from a set of data. 3.7 communication.

blunder. The best illustration for a novice is the predicament encountered by a jury trial. However. The famous Hawthorne study examined changes to the working environment at the Hawthorne plant of the Western Electric Company. then modified the illumination in an area of the plant and checked if the changes in illumination affected productivity. However.. The null hypothesis. Key terms used in statistics Null hypothesis Interpretation of statistical information can often involve the development of a null hypothesis in that the assumption is that whatever is proposed as a cause has no effect on the variable being measured. While one cannot "prove" a null hypothesis one can test how close it is to being true with a power test. asserts that the defendant is innocent.8 5. Error Working from a null hypothesis. the study is heavily criticized today for errors in experimental procedures. Documenting and presenting the results of the study. H0. Those in the Hawthorne study became more productive not because the lighting was changed but because they were being observed. Error also refers to the extent to which individual observations in a sample differ from a central value. The researchers first measured the productivity in the plant." Measurement processes that generate statistical data are also subject to error. worker productivity) changed due to observation itself. Many of these errors are classified as random (noise) or systematic (bias). which tests for type II errors. Many statistical methods seek to minimize the mean-squared error. but merely that the evidence was insufficient to convict. So the jury does not necessarily accept H0 but fails to reject H0. two basic forms of error are recognized: ‡ Type I errors where the null hypothesis is falsely rejected giving a "false positive". specifically for the lack of a control group and blindness. ‡ Type II errors where the null hypothesis fails to be rejected and an actual difference between populations is missed giving a "false negative". The H0 (status quo) stands in opposition to H1 and is maintained unless H1 is supported by evidence ³beyond a reasonable doubt". The researchers were interested in determining whether increased illumination would increase the productivity of the assembly line workers. ³failure to reject H0" in this case does not imply innocence. such as the sample or population mean. The Hawthorne effect refers to finding that an outcome (in this case. It turned out that productivity indeed improved (under the experimental conditions). whereas the alternative hypothesis. The indictment comes because of suspicion of the guilt. Experiments on human behavior have special concerns. H1.g. and these are called "methods of least squares. but other important types of errors (e. asserts that the defendant is guilty. such as when an analyst reports incorrect units) can also be important. .

it is true that. the limits of the interval are yet-to-be-observed random variables. . such that the drug will be unlikely to help the patient in a noticeable way. Confidence intervals allow statisticians to express how closely the sample estimate matches the true value in the whole population. However. the interval would include the true (population) value 95% of the time. as the true value is not a random variable. Either the true value is or is not within the given interval. such a claim does not even make sense. Significance Statistics rarely give a simple Yes/No type answer to the question asked of them. if the sampling and analysis were repeated under the same conditions (yielding a different dataset). that is as a Bayesian probability.9 Interval estimation Most studies will only sample part of a population and so the results are not fully representative of the whole population. This does not imply that the probability that the true value is in the confidence interval is 95%. Any estimates obtained from the sample only approximate the population value. Often they are expressed as 95% confidence intervals. the probability is 95% that the yet-to-be-calculated interval will cover the true value: at this point. in a large study of a drug it may be shown that the drug has a statistically significant but very small beneficial effect. a 95% confidence interval for a value is a range where. One approach that does yield an interval that can be interpreted as having a given probability of containing the true value is to use a credible interval from Bayesian statistics: this approach depends on a different way of interpreting what is meant by "probability". Formally. From the frequentist perspective. Interpretation often comes down to the level of statistical significance applied to the numbers and often refers to the probability of a value accurately rejecting the null hypothesis (sometimes referred to as the p-value). Referring to statistical significance does not necessarily mean that the overall result is significant in real world terms. before any data are sampled and given a plan for how the confidence interval will be constructed. For example.

Sign up to vote on this title
UsefulNot useful