Psychology Classics On Amazon
Experimental Design Tutorial
David Webb (Owner, writer & host of All-About-Psychology.Com)
If you study psychology, it's almost inevitable that you will be expected to learn about research methods and statistics - and for very good reasons! For instance, in order to critically evaluate the findings of key psychology studies you need to be aware of how the data supporting the research came about; and you can't seriously hope to conduct your own research with confidence unless you have a clear idea how to design, execute and analyse your investigation.
When I first started supervising research dissertations and psychology projects I came across a large number of students who refused to consider doing anything other than interviews, whether such an approach was appropriate for their particular investigation or not. The most common reason for this was the incorrect assumption that qualitative research methods (text based) are much easier to understand and carry out proficiently than quantitative (number based) research methods.
Why Do So Many Psychology Students Think This?
To be fair it's understandable, words just seem more real and easier to relate to than numbers. A point that seems to be reinforced when students first encounter the experimental method with its strong statistical component and abstract terminology. Terms like extraneous variable, independent variable etc are enough to put most people off before they start.
The primary aim of this tutorial then, is to show you that the experimental method isn't nearly as complicated or as frightening as many people think. The following information formed part of an introductory research methods course which I co-wrote with Dr David Tyfa at the University of Huddersfield (UK). I hope you find it useful.
All the best
David Webb BSc (Hons), MSc
Something To Relate To
The first thing our students did on their introduction to research methods course was to complete a simple word search puzzle similar to the one shown above. Each student did the puzzle on a computer and when they had found all the words, the time it took them to complete the puzzle (in seconds) appeared on the screen.
Now what the students didn't realize at the time is that they were actually taking part in an experiment.
I (along with my teaching support team) took half the students into PC lab 1. My colleague Dr Tyfa (along with his teaching support team) took the other half of the students into PC lab 2.
When the students were doing the puzzle in PC lab 1 (my lab), the teaching team made their presence felt as much as possible, walking up and down each bank of computers, looking over the students shoulder, essentially being as conspicuous as possible.
In PC lab 2 (Dr Tyfa's lab) the teaching team did the opposite i.e. they remained as inconspicuous as possible and showed no interest in what the students were doing at all.
(Try to imagine yourself as one of the students doing the word search puzzle in either PC lab 1 or PC lab 2)
The students were completely unaware what was going on, as far as they were concerned the object of the exercise was to collect some data i.e. the time taken to complete the puzzle, which they could then practice entering into a statistical software package.
The true nature of the exercise wasn't revealed until the following week when I gave a lecture on the experimental method. Because what I was able to do now, was take key experimental design concepts and relate them to something the students had direct experience of i.e. the word search puzzle experiment.
Before you move on, take a look at the quick reference guide which outlines the major experimental design concepts that you need to be aware of. It would be a good idea to keep the quick reference guide page open as you continue the tutorial.
Imagine trying to understand the concepts listed on the experimental design quick reference guide without something to relate them to! I hope that having the word search experiment to draw upon, just like my students, you'll see as we explore these concept one by one, that they aren't that difficult to get your head around and actually make a great deal of sense.
Let's start with the logic of the experimental method.
If two groups of participants are equal in all respects save one and are not similar in respect of a behavior that is being measured, then the difference between them must be attributable to the one way in which they were different. (J.S Mill)
When we did the word search experiment we had two groups of participants, i.e. some in (PC lab 1) and some in (PC lab 2). Now for the time being, let’s accept that these two groups of participants were equal in all respects save one. This of course begs the question, what was the one way in which the two groups of participants were not equal?
The answer is level of teacher presence!
Remember, when the students were doing the puzzle in PC lab 1 (my lab), the teaching team made their presence felt as much as possible, walking up and down each bank of computers, looking over the students shoulder, essentially being as conspicuous as possible. In PC lab 2 (Dr Tyfa's lab) the teaching team did the opposite i.e. they remained as inconspicuous as possible i.e. showed no interest in what the students were doing at all.
Now if it turns out that the psychology students in PC lab 1 and PC lab 2 were not similar in respect of the behavior that was being measured (in our case the time taken to complete a word search puzzle), then according to the logic of the experimental method, this difference must be attributable to level of teacher presence. In other words
Level of teacher presence (the cause) made a difference in the time taken to complete the puzzle (the effect).
So the first thing to note about the experimental method is that it is fundamentally concerned with establishing cause and effect.
In addition to understanding the logic of the experimental method you’ll also need to understand the language that accompanies it. So let's take each concept listed on the Quick Reference Guide in turn and where possible, link it to the word search puzzle experiment.
The first thing listed on the quick reference guide is the Experimental Hypothesis, which you may see denoted as the H1. As the definition on the guide states it’s the prediction of the outcome of the experiment.
The hypothesis is the starting point in the process of experimentation, so when we were thinking about a simple experiment that our students could do, we needed an idea that we could test; and this is what we came up with.
High levels of teacher presence in PC lab 1 will cause students to perform a word search puzzle more quickly than students in PC lab 2.
Now because we’ve predicted the way in which behavior will change, (i.e. it will get faster,) we’ve actually formulated a One-Tailed Experimental Hypothesis.
If we’d simply stated that high levels of teacher presence will cause students in PC lab 1 to perform a word search puzzle differently than students in PC lab 2. We would have formulated a Two-Tailed Experimental Hypothesis because all we’re saying here is that the behavior of interest will change.
The decision to choose a one-tailed or a two-tailed hypothesis depends on how confident you feel in predicting the way that behavior will change. We were confident enough to go for a one-tailed experimental hypothesis because a large body of research evidence suggests that the presence of others will improve performance on simple tasks.
Another type of hypothesis you need to be aware of is the Null Hypothesis sometimes denoted as the H0. The null hypothesis simply states that any observed differences between groups were down to chance. The idea behind it, is that depending on the result of statistical testing at the end of the experiment you will either reject or retain the null hypothesis. Put simply the bigger the difference between groups, the less likely that the difference is down to chance and the more likely you are to reject the null hypothesis.
Continuing with the quick reference guide, let’s have a look at the Independent Variable (the IV). This is the one factor that is different between conditions. Now the IV is the thing that you as the experimenter manipulate in order to see if it causes a change in behavior. And as previously mentioned the one thing that differed between conditions in the word search experiment was level of teacher presence.
So what about the Dependent Variable (the DV)? The aspect of behavior being measured? Again as previously mentioned, in the word search experiment it was the time taken (in seconds) to complete the word search puzzle.
Hopefully you’re beginning to see that although the experimental method uses unfamiliar terms like the H1, H0, IV and DV the things they allude to are not actually that complicated.
Let’s go back to the logic of the experimental method and highlight the independent and dependent variables.
If two groups of participants are equal in all respects save one (The independent variable) and are not similar in respect of a behavior that is being measured (the dependent variable) then the difference between them must be attributable to the one way in which they were different (the independent variable)
Now, remember when I said lets accept for the time being that the two groups of participants doing the word search puzzle are equal in all respects save one. Well clearly this is impossible to maintain when dealing with human subjects for the simple fact that we are not clones, we’re all different and we bring our individual differences with us when we take part in an experiment. For instance, some of the students will have had more sleep than others and this may have affected their performance i.e. they didn’t finish the word search puzzle as quickly as they would have done if they'd had a proper night’s sleep.
Now this shouldn’t be a problem providing we are only talking about one or two people, and this kind of thing is referred to as a random error because things like number of hours sleep vary randomly in a population of people and it’s something that cannot be eliminated. It’s when random errors become constant errors that you’ll find yourself saying Houston we have a problem.
Remember the cornerstone of the scientific method is to establish cause and effect and we do this by manipulating the independent variable to see what effect it has on the dependent variable. Now imagine if it transpired that all the students in PC lab 2 had been to the same party the night before and only had on average 2 hours sleep? Whereas; all the students in PC lab 1 had fallen out with the person throwing the party so they didn’t go and so they ended up having a normal night’s sleep.
You now have no way of knowing whether students in PC lab 1 performed the word search puzzle more quickly because of level of teacher presence (the IV); or whether they performed the puzzle more quickly because they’d had more sleep and as such were more alert. In this example sleep has become a Confounding Variable. It’s confounded your results and the experiment is ruined. This somewhat frivolous example was used to introduce you to another integral aspect of the experimental method i.e. control.
In the experimental method we accept that random error exists. We also accept that random error produces extraneous variables. We must, therefore, always do our best to ensure that extraneous variables do not become confounding variables. The simplest way to do this is to randomly allocate people to different conditions. So in our sleep example for instance, random allocation would ensure that all the sleep deprived party going people didn't end up in the same group.
This leads us nicely into the 3 main types of experimental design because as you’ll see each has strengths and weaknesses in relation to control issues. Experimental design simply refers to the way in which participants are deployed during the experiment. Before we look at the 3 designs, however, I just want to quickly outline the difference between the experimental group and the control group. The experimental group is where you expect the predicted behavior change to occur (in our case PC Lab 1 i.e. high level of teacher presence) and the control group acts as the baseline so that this change can be assessed (PC Lab 2 i.e. low level of teacher presence). OK on to the 3 main types of experimental design.
Independent Subjects Design
Depending which book you read this design may also be referred to as independent groups, independent samples or between groups. The independent subjects design is the one we employed during the word search experiment. In this design participants are divided into entirely separate groups on the basis of random allocation, meaning that each participant has an equal chance of being allocated to either the experimental group or the control group.
Let’s say we have 60 participants, we could put the numbers 1 to 60 in a hat and get participants to pick a number. Participants who pick an even numbers would go into the experimental group and participants who pick an odd numbers would go into the control group.
Each participant will provide us with a single score. It’s then a case of looking at all the scores in the experimental group and comparing them with all the scores in the control group.
Repeated Measures Design
This is also referred to as the related samples design or the within groups design. In this design the participant performs in both the experimental condition and the control condition.
Each participant, therefore, provides us with two scores or paired data. It’s then a case of comparing all the paired data scores to see if there’s a difference between the conditions.
Now the main strength of this design over the independent subjects design is that you don’t have to worry as much about individual differences confounding the results of your experiment because if you think about it, each participant acts as their own control.
In other words they take their random variability i.e. number of hours sleep, personality etc. with them across the conditions so that they cancel each other out.
The danger of this type of design is that Order Effects will confound your results. Because each participant is being asked to do something twice it’s possible that they will perform better in the second condition because of practice or that they perform worse in the second condition because of fatigue or boredom.
Therefore, we must ensure that the order of the two conditions is counterbalanced.
Counterbalancing is a technique employed to ensure that half the subjects perform in the experimental condition first and the control group second. And half the subjects perform in the control group first and the experimental group second.
Counterbalancing does not get rid of order effects but it does makes sure that any possible confounding effects cancel each other out.
Independent subjects and repeated measures are the two most common types of design in experimentation and the thing to note about them is that their strength lies in the weakness of the other.
With independent subjects design you don’t have to worry about order effects because you aren’t getting people to perform twice; but in the repeated measures design you are this is can be a real problem. However, in the independent subjects design you have to worry about individual differences between conditions; but in the repeated measures design you don’t because people take their differences with them, thereby nullifying any possible effect.
Matched Pairs Design
In this design participants are arranged into pairs on the basis of a pre-test. For instance you might pair people up who have a similar IQ. Members of the pairs are then allocated randomly to either the experimental condition or the control condition. It’s then a case of comparing the paired data scores to see if there’s a difference between the conditions.
In Conclusion
This tutorial has covered a lot of material and it would be worth going through at least a couple more times. If some of the concepts still seem a little abstract just keep trying to relate them to word search experiment.
Please bear in mind that the tutorial only covers simple experiments where you only have 2 conditions i.e. the experimental group and the control group and where you only have one independent variable and one dependent variable. Experiments can be more complex than this you can have more than two conditions and you can have multiple independent and dependent variables but the logic remains the same.
Finally I'd like to conclude with a definition of an experiment because having completed the tutorial it should make more sense.
"An experiment is a study of cause and effect. It differs from simple observation in that it involves deliberate manipulation of one variable (the independent variable), while controlling other variables (extraneous variables) so that they do not affect the outcome, in order to discover the effect on another variable (the dependent variable)."
S. Heyes (1986).
Especially designed for Halloween loving psychology students, this Funny Paranormal Distribution T-Shirt is perfect for trick or treating or for a spooky costume party. Also ideal for anyone who likes research methods, math and statistics who will appreciate the humorous play on words. Sales help support All-About-Psychology.Com which has been providing free and comprehensive information and resources for psychology students and educators since 2008.
Recent Articles
-
Psychological Theories
Apr 25, 24 09:43 AM
I’ve set a lofty or an over ambitious goal of reading and gathering psychological theories or principles starting from prehistorical religion up until -
5 Factors to Consider When Caring for Elderly Parents At Home
Apr 25, 24 09:09 AM
Insightful blog post outlining five tips to prioritize as you care for your elderly parents. -
The Psychology of Dreams
Apr 23, 24 10:34 AM
Learn all about the psychology of dreams.
Please help support this website by visiting the All About Psychology Amazon Store to check out an awesome collection of psychology books, gifts and T-shirts.
 |
 |
Go To The Psychology Research Methods Page