A Causal Relationship Is Determined Philosophy Essay

Internal Validity refers to the rough reality regarding implications of cause-effect and/or causal relationships. As such, internal validity is only pertinent in studies where a causal relationship is determined. It's not appropriate in a number of observational or descriptive studies. However, for studies that evaluate the effects of social programs and/or interventions, internal validity is possibly the primary consideration. In those perspectives, one would determine that the program or treatment made a difference – i.e., the improved test scores or reduced symptomology. Nonetheless, there may be a number of reasons, other than the program, why test scores shall improve or symptoms shall reduce. The vital question in internal validity is whether the observed changes shall be attributed to the program or intervention (i.e., the cause) and not to the other possible causes (like "substitute clarifications" for the result).


Figure 3.1 : Internal Validity

One of the most difficult thing to comprehend about internal validity is that it is only significant to the specific study in the question. That is, one may consider internal validity as a "zero generalizability" apprehension. All that internal validity means is that one has proof that what was done in the study (i.e., the program) caused what was observed (i.e., the result) to happen. It doesn't tell whether what was done for the program was what was required to do or whether what was observed was what was wanted to observe - those are construct validity concerns.

It is likely to have internal validity in a study and not have the construct validity. For example, a study where you look at the effects of a new computerized tutoring program on sociology performance in first grade students. Visualize that the tutoring is unique in that it has a heavy computer software component and think that's what will really work to improve sociology performance. Finally, visualize that you were wrong (hard, isn't it?) - it turns out that sociology performance did improve, and that it was because of something you did, but that it had nothing to do with the computer program. What caused the improvement was the individual attention that the adult tutor gave to the child - the computer program didn't make any difference. This study would have internal validity because something that you did affected something that you observed - you did cause something to happen. But the study would not have construct validity, specifically, the label "computer sociology program" does not accurately describe the actual cause (better described as "personal adult attention").

The key issue in internal validity is the causal one, as such, establishing a cause-effect relationship will be discussed first. Then we'll discuss different threats to internal validity - kinds of criticisms the critics will raise when one tries to conclude that the program caused the outcome. For convenience, the threats to validity are divided into three categories. The first involve the single group threats - criticisms that apply when you are only studying a single group that receives the program. The second consists of the multiple group threats - criticisms that are likely to be raised when you have several groups in the study (e.g., a program and a comparison group). Finally, social threats to internal validity will be discussed - threats that arise because social research is conducted in real-world human contexts where people will react to not only what affects them, but also to what is happening to others around them.

Establishing a Cause-Effect Relationship

Usually, there are three criteria that must be met prior to conclusion that one has evidence for a causal relationship:

Temporal Precedence - First, one should show that the cause happened before the effect. For instance, does inflation cause unemployment? It undoubtedly seems plausible that as inflation increases, more employers find that in order to meet costs they have to lay off employees. So it seems that inflation could, at least partially, be a cause for unemployment.


Figure 3.2: Cyclical Functions

But both inflation and employment rates are occurring together on an ongoing basis. Is it possible that fluctuations in employment can affect the inflation? If we increase the work force (i.e., lower unemployment) we may have more demand for goods, which would tend to drive up the prices (i.e., inflate them) at least until supply can catch up. So which is the cause and which the effect, inflation or unemployment? It turns out that in this kind of cyclical situation involving ongoing processes that interact that both may cause and, in turn, be affected by the other. This makes it very hard to establish a causal relationship in this situation.

Covariation of the Cause and Effect - Before one shows that he/she have a causal relationship, one has to show that he/she have some type of relationship. For example, consider the syllogism:

if X then Y

if not X then not Y

If you observe that whenever X is present, Y is also present, and whenever X is absent, Y is too, then you have demonstrated that there is a relationship between X and Y. I don't know about you, but sometimes it's not easy to think about X's and Y's. Let's put this same syllogism in program evaluation terms:

if program then outcome

if not program then not outcome

Or, in colloquial terms: if you give a program you observe the outcome but if you don't give the program you don't observe the outcome. This provides evidence that the program and outcome are related. Notice, however, that this syllogism doesn't not provide evidence that the program caused the outcome -- perhaps there was some other factor present with the program that caused the outcome, rather than the program. The relationships described so far are rather simple binary relationships. Sometimes we want to know whether different amounts of the program lead to different amounts of the outcome -- a continuous relationship:

if more of the program then more of the outcome

if less of the program then less of the outcome

No Plausible Alternative Explanations - Just because you show there's a relationship doesn't mean it's a causal one. It's possible that there is some other variable or factor that is causing the outcome. This is sometimes referred to as the "third variable" or "missing variable" problem and it's at the heart of the issue of internal validity. What are some of the possible plausible alternative explanations? Just go look at the threats to internal validity (see single group threats, multiple group threats or social threats) -- each one describes a type of alternative explanation.

In order for you to argue that you have demonstrated internal validity - that you have shown there's a causal relationship - you have to "rule out" the plausible alternative explanations. How do you do that? One of the major ways is with your research design. Let's consider a simple single group threat to internal validity, a history threat. Let's assume you measure your program group before they start the program (to establish a baseline), you give them the program, and then you measure their performance afterwards in a posttest. You see a marked improvement in their performance which you would like to infer is caused by your program. One of the plausible alternative explanations is that you have a history threat - it's not your program that caused the gain but some other specific historical event.

For instance, it's not your anti-smoking campaign that caused the reduction in smoking but rather the Surgeon General's latest report that happened to be issued between the time you gave your pretest and posttest. How do you rule this out with your research design? One of the simplest ways would be to incorporate the use of a control group - a group that is comparable to your program group with the only difference being that they didn't receive the program. But they did experience the Surgeon General's latest report. If you find that they didn't show a reduction in smoking even though they did experience the same Surgeon General report you have effectively "ruled out" the Surgeon General's report as a plausible alternative explanation for why you observed the smoking reduction.

In most applied social research that involves evaluation of the programs, temporal precedence is not a difficult criterion to meet as one administers the program before the measurement of the effects. And, establishment of covariation is relatively simple as one has some control over the program and can set things up so that he/she has some people who get it and some who don't (if X and if not X). Usually the most difficult criterion to meet is the third - ruling out alternative explanations for the observed effect. That is why research design is such a significant issue and why it is closely linked to the idea of internal validity.

Exhibit 3.5: Social Interaction Threats

The social threats to internal validity mean the social pressures in the research context that may lead to posttest differences that are not directly caused by the treatment itself. A number of these threats occur because the various groups (e.g., program and comparison), or key people involved in carrying out the research (e.g., managers and administrators, teachers and principals) are aware of each other's existence and of the role they play in the research project or are in contact with one another. Many of these threats can be minimized by isolating the two groups from each other, but this leads to other problems (e.g., it's hard to randomly assign and then isolate; this is likely to reduce generalizability or external validity). The foremost social interaction threats to internal validity are:http://www.socialresearchmethods.net/kb/Assets/images/intsoc1.gif

Diffusion or Imitation of Treatment - Happens when a comparison group learns about the program either directly or indirectly from program group participants. In a college context, children from different groups within the same college might share experiences during lunch hour. Or, comparison group students, seeing what the program group is getting, might set up their own experience to try to imitate that of the program group. In either case, if the diffusion of imitation affects the posttest performance of the comparison group, it can have an jeopardize your ability to assess whether your program is causing the outcome. Notice that this threat to validity tend to equalize the outcomes between groups, minimizing the chance of seeing a program effect even if there is one.

Compensatory Rivalry - Comparison group knows what the program group is getting and develops a competitive attitude with them. The students in the comparison group might see the special sociology tutoring program the program group is getting and feel jealous. This could lead them to deciding to compete with the program group "just to show them" how well they can do. Sometimes, in contexts like these, the participants are even encouraged by well-meaning teachers or administrators to compete with each other (while this might make educational sense as a motivation for the students in both groups to work harder, it works against our ability to see the effects of the program). If the rivalry between groups affects posttest performance, it could maker it more difficult to detect the effects of the program. As with diffusion and imitation, this threat generally works to in the direction of equalizing the posttest performance across groups, increasing the chance that you won't see a program effect, even if the program is effective.http://www.socialresearchmethods.net/kb/Assets/images/intsoc4.gifhttp://www.socialresearchmethods.net/kb/Assets/images/intsoc3.gif

Resentful Demoralization - This is almost the opposite of compensatory rivalry. Here, students in the comparison group know what the program group is getting. But here, instead of developing a rivalry, they get discouraged or angry and they give up (sometimes referred to as the "screw you" effect!). Unlike the previous two threats, this one is likely to exaggerate posttest differences between groups, making your program look even more effective than it actually is.

Compensatory Equalization of Treatment - This is the only threat of the four that primarily involves the people who help manage the research context rather than the participants themselves. When program and comparison group participants are aware of each other's conditions they may wish they were in the other group (depending on the perceived desirability of the program it could work either way). Often they or their parents or teachers will put pressure on the administrators to have them reassigned to the other group. The administrators may begin to feel that the allocation of goods to the groups is not "fair" and may be pressured to or independently undertake to compensate one group for the perceived advantage of the other. If the special sociology tutoring program was being done with state-of-the-art computers, you can bet that the parents of the children assigned to the traditional non-computerized comparison group will pressure the principal to "equalize" the situation. Perhaps the principal will give the comparison group some other good, or let them have access to the computers for other subjects. http://www.socialresearchmethods.net/kb/Assets/images/intsoc2.gif

As long as we engage in applied social research we have to deal with the realities of human interaction and its effect on the research process. The threats described here can often be minimized by constructing multiple groups that are not aware of each other (e.g., program group from one school, comparison group from another) or by training administrators in the importance of preserving group membership and not instituting equalizing programs.