Learn how to look at some of the ways you might structure an evaluation and how to choose the way that best meets your needs.
Ex Post Facto Designs Pre-experimental designs are the simplest type of design because they do not include an adequate control group. A pre- and post-intervention design involves collecting information only on program participants.
This information is collected at least twice: However, findings using this design may be enough to indicate your program is making a difference depending on how rigorous the proof needs to be, proximity in time between the implementation of the program and the progress on outcomes, and the systematic elimination of other alternative explanations.
Not an authentic experimental design Design Evaluation designs not control for many extraneous Evaluation designs Subject to many threats to validity Typically conducted for exploratory purposes Usually convenient and financially feasible The three types of pre-experimental designs are: Pre-experimental Designs Image taken from: A good experimental design can show a casual relationship between participation in your program and key student outcomes.
The key to this design is that all eligible program participants are randomly assigned to the treatment or control group. When random assignment is used, it is assumed that the participants in both the control and treatment groups have similar attributes and characteristics.
The purpose of a true experimental design is to control bias. In a true experiment, differences in the dependent variables can be directly attributable to the changes in independent variable and not other variables.
Characteristics of Experimental Design Research controls manipulation of the intervention or treatment Participants are random assigned to groups Intervention or treatment occurs prior to observation of the dependent variable Strengths Causal relationships between variables can be found Limitations Limited external validity generalizability due to the controlled experimental environment Ethical concerns The image below provides a model of several experimental designs.
Experimental Designs Image taken from: A quasi-experimental design is very similar to an experimental design except it lacks random assignment. Depending on treatment and comparison group equivalency, evidence generated from these designs can be quite strong.
To conduct a quasi-experimental design, you will need to identify a suitable comparison group i.
Characteristics of a Comparison Group Members of a comparison group may receive other types of services or no services at all. A comparison group should be similar to the treatment group on key factors that can affect your outcomes.
You may have to collect data to try and control for potential differences as part of your statistical analyses. Strengths Enables experimentation when random assignment is not possible Avoids ethical issues caused by random assignment Limitations Does not control for extraneous variables that may influence findings The image below shows several examples of quasi-experimental designs.
Quasi-Experimental Designs Image taken from: The ability to produce a quality evaluation with such as design is directly related to the quality and quantity of data readily available.
The phenomenon of interest has already occurred at the time of observation or measurement. There is typically no control or comparison group.
Main weakness of design: Essentially, your analysis will be limited to the data that is available. You can investigate research questions that are inappropriate for experimental designs.
These designes are typically more logistically and financially feasible. You can pay more attention to context instead of seeking to control variables and the environment.
These designs are particularly effective when Krathwohl,p.Program evaluation is essential to public health. The Centers for Disease Control and Prevention sets standards for evaluation, develops evaluation tools and resources, and provides support for evaluation capacity-building.
Depending on your program’s objectives and the intended use(s) for the evaluation findings, these designs may be. The key to a good evaluation plan is the design of the study or studies to answer the evaluation questions. There are many possible research designs and plans. There are many possible research designs and plans.
EVALUATION MODELS, APPROACHES, AND DESIGNS— purposes. As with utilization-focused evaluation, the major focusing question is, “What are the information needs of .
It discusses in more detail the key steps to follow when planning and undertaking an evaluation and how to answer evaluation research questions using different evaluation research designs. It also discusses approaches to the interpretation and assimilation of evaluation evidence.