Dependent t-tests

Dependent t-tests are a special brand of t-tests used when your two "groups" to be compared are actually just one group measured on two occasions. For example, you could use a dependent t-test to compare a group's score on the GRE before and after they have completed a test preparation course.

It's important to use the dependent t-test instead of the regular old independent t-test because the independent t-test makes the assumption of independent observations. If you've got more than one data point per person, then those data points are not independent; they are dependent. Thus the dependent t-test. The dependent t-test is also more powerful (more likely to produce a significant result) than the independent t-test because it is a within-subjects test.

Imagine a test-preparation course for the GRE. We want to know whether the course is effective. The following dataset presents the pre-course and post-course scores on the GRE for 20 people (I made these up). Open it in SPSS.

Look at the scores in the "pre" column and compare them with the scores in the "post" column. The "post" scores are higher, but only by a little.

Plotting Your Data the Between-Subjects (Wrong) Way

To get a graphical impression of how different the scores are, plot them using Graphs -> Legacy Dialogs -> Error Bar. In the Error Bar window (shown below), tell SPSS that the "Data in Chart Are" "Summaries of separate variables."

Move both "Pre" and "Post" into the "Error Bars" window and press "OK". This should produce the following plot:

As you can see, the POST scores are slightly higher than the PRE scores. If you were to treat these two groups as independent samples and compute an independent t-test, you would get t(38) = 0.30, p = .76 (not significant). Before you start concluding that the course had no effect, let's take a look at what happened to each person's score.

Plotting Repeated-Measures Data (the Right Way)

"Repeated measures" data occur when you collect more than one measure from a single person. In this case, you've got two scores from each person. What we want to do is create a plot showing how each person increased or decreased from the pre-test to the post-test. Select Graphs -> Legacy Dialogs -> Interactive -> Line. Click once on the "[pre]" item and then hold down the Ctrl key and click once on the "[post]" item. Both of those items should now be highlighted:

After both are highlighted, move the mouse pointer over them and watch as it turns into a glove. Click on the two highlighted items to "grab" them, then drag them over into the X-axis box:

A new window will appear called "Specify Labels":

The Values box is the label for the y-axis (dependent variable). In this case, your dependent variable is GRE Score, so you could type that in. The Categories box is the label for the x-axis. In this case, it is "Time." Click OK to proceed.

Finally, grab "id#" and drag it into the box labeled Style. A warning window will appear, asking if you want to convert id# to categories. Select Convert. The window should now look like this:

To create the plot, click OK. You should get something like this:

What does this plot show you? Each line represents a single person. In every case, the line goes upward slightly from pre-test to post-test. For some people, the increase was greater than for others. A regular independent t-test would not detect a significant increase because it compares the post-test mean to the pre-test mean. Rather than looking at the mean for the whole group, a dependent t-test tracks each individual and sees whether, in general, each score tends to increase or decrease over time. It is very unlikely that, just by chance, all 20 people would show an increase in their scores.

Putting the Figure into a Google Document

As we did with the figure for the 2-way ANOVA, to copy this graph into a Google Document, click once on the chart so that it has a border around it, then right-click on the chart and select "Export." Select "Selected" in the first box, "None (Graphics only)" as Document Type, and under Graphics Type I would recommend saving it in .PNG format, which tends to be good for line drawings. Save it to a location you'll be able to get to easily and use a name that you can remember. Then open a Google Document and select Insert -> Picture.

Generally, you would not want to use a line graph like this one in a psychology paper because it's very messy. In fact, it is unusual to see a figure for a dependent t-test because only two means are being compared and it is easy for people to visualize that in their heads. However, the interactive line graph above can be very instructive during early stages of interpretation because it can tell you whether trends are being upset by just a few people.

Paired-Samples T Test in SPSS

Because each participant in the above graph increased over time, we can be pretty confident that the GRE prep course had an effect. To test whether the effect was significantly stronger than we would expect by chance, we must determine the statistical significance of the change. Select Analyze -> Compare Means -> Paired-Samples T Test. First, select the variable "pre." You will see it appear in the "Current Selections" window, below:

Then select the variable "post." You will see it appear as the second "Current Selection":

The black right-arrow is now active, so you can press it to move those two variables into the "Paired Variables" window:

Then press OK to start the analysis.

The first output you get is about the two variables pre and post:

This shows you that the mean GRE score during the pre-test was 1073, and the mean score during the post-test was 1089. There were 20 scores at each time period, indicating that there were 20 participants. The standard deviation (SD) for each group was approximately 165.

Next, SPSS computes the correlation between the two variables:

I made these data up, so they're a lot "cleaner" than most data you'd run into in the real world. In the table above, the correlation between pre-test and post-test is exactly r = 1.00. That's because the rank order of the 20 people didn't change at all from pre- to post-test. The person who was #1 at the pre-test was still #1 at post-test, and #20 was still #20. Usually, you'll have some flipping around and get a correlation less than r = 1.00.

The really important results are in the third box:

First, you're presented with the mean difference between PRE and POST: 15.80. On average, people at post-test scored 15.8 points higher than the same people at pre-test.

Confidence Intervals

The 95% confidence interval tells you that, based on these data, there is a 95% probability that the true difference between pre-test and post-test is between 14.56 points and 17.04 points. Conversely, there is a 5% probability that the true difference between the means is either less than 14.56 points or greater than 17.04 points.

You can use confidence intervals to tell you right away whether a dependent t-test is significant at p < .05. If the 95% confidence interval crosses 0, then the test will not be significant at p < .05. The reason is that if the interval crossed zero, then one possible value of the difference would be 0 (no difference at all). But if the interval does not cross zero, then the p-value of the dependent t-test will be less than .05.

The results of the dependent t-test are given at the end:

From these, we see that t(19) = 26.687, p < .001. According to the dependent t-test, the post-test scores are significantly higher than the pre-test scores. The independent t-test would have led us to believe there was no difference. This comparison between the results of independent and dependent t-tests is very important: for dependent data, you will have much more statistical power (likelihood of finding an effect) if you use the appropriate statistical tests.

APA Style

To summarize the results above, you could write:

On average, scores at the post-test were 15.8 points higher than scores at the pre-test (95% CI: 14.6 to 17.0), dependent t(19) = 26.79, p < .001.

or

The average GRE score at the post-test was 1089 (SD = 165), while the average score at pre-testing was 1073 (SD = 165). Results from a dependent t-test indicate that this difference was significant, t(19) = 26.79, p < .001.