by Andy Field
12.2 Participants
These used to be called ‘subjects’, but psychology is as vulnerable to political correctness as anything else! Give details of how many participants you used, and how many took part in each condition (if a between-groups design was used). Include information on demographic variables such as age and gender, and provide brief details of how you obtained the participants – for example, did they volunteer, and were they paid for taking part? Were they naive about the purpose of the experiment? Participants in psychology experiments are often ‘blind’ to the purposes of the study. However, you need to be careful about how you express this – reading some practical reports, with phrases such as ‘blind participants were used’ or ‘all participants were blind’, you could be forgiven for thinking that many participants in psychology experiments have profound eyesight difficulties!
Sometimes, you might use a special population of participants, such as colour-blind individuals, learning-disabled people, hyperactive children or spider-phobics. If so, further relevant details should be provided about them. For example, in the case of the spider phobics, give brief details of the criteria by which they are defined as ‘phobic’, such as their mean scores on some recognised phobia test.
As mentioned earlier, you need to think about what information is relevant and what’s not relevant. This may depend on the circumstances. Thus, if you are measuring people’s ability to estimate distances, the issue of ethnic grouping is irrelevant and need not be mentioned, but details of whether the individuals concerned had normal stereoscopic vision should be given. Conversely, if you are conducting a study on attitudes to racial prejudice, ethnic grouping might be highly relevant and hence details should be provided in this section.
12.3 Apparatus
* * *
This section is reasonably straightforward. You have to provide enough details of the apparatus that was used, for someone to be able to replicate the experiment. Avoid writing ‘shopping lists’ of apparatus – write in full English sentences. Instead of writing ‘Stopwatch. Questionnaire. Tape recorder’, write ‘The apparatus consisted of a stopwatch, a questionnaire, and a tape-recorder’.
In the case of questionnaires and other pencil-and-paper tests, if it’s widely used (such as Eysenck’s Personality Inventory and Witkin’s Embedded Figures Test, tests which are in common use amongst psychologists) give its name, a reference to where a full account of it can be obtained, and some justification for why you used that particular test. Briefly provide evidence that it’s a good measure. This might include reference to studies which have shown that the measure is reliable, valid and is appropriate for participants similar to the ones you are testing. (Many widely-used questionnaires and pencil and paper tests have been standardized on different sub-groups of people, so that there are sub-group norms for performance which differ according to the group under consideration).
If you are using a novel or home-brewed test (e.g. a questionnaire that you have devised yourself) then put the test or questionnaire in an appendix at the end of the report, and state in the apparatus section that it can be found there.
If the experiment involves presenting stimuli and/or recording responses on a computer, it is normally sufficient to give the generic type of computer used, plus a brief mention of what software was used. For example, ‘The stimuli were presented on the screen of an Apple Macintosh Quadra, using a program written in Supercard’. If the equipment is a little more out of the ordinary, give the name of the manufacturer and of the product. For example, ‘Electric shocks were administered using a ‘Sizzler Mk. II’ shock-generator (manufactured by Megajolt Ltd., Ohio)’.
Once again, consider the issue of relevance. If the experiment consists of completing a questionnaire, you probably need only mention the questionnaire itself: you can reasonably assume the reader realises that a pen or a pencil was required to fill in the questionnaire, and you might even be bold enough to let them assume that doing the questionnaire involved using a table, a seat and a floor. You certainly wouldn’t need to mention how the room was lit, unless it was something out of the ordinary. Details of the typeface and font size in which the questionnaire was written would also be superfluous. However, if you were doing a study of eye-movements during reading, the ambient lighting and typeface might well be influential factors, so they should be clearly specified. In short, only mention things that could reasonably be expected to have a bearing on the outcome of the study.
Sometimes you may see this section divided into two, ‘Apparatus’ and ‘Materials’. If so, ‘Apparatus’ is the place in which to describe things like computers and other hardware, whereas details of questionnaires, visual stimuli and the like should go in the ‘Materials’ section.
12.4 Procedure
This section gives details of how you carried out the experiment in practice. Suppose your experiment involves examining the effects of cerebral blood-flow on memory. Participants are given a memory test twice, once after being held upside down on a rope, and once after being allowed to stand upright. How were these two conditions administered? Were all participants exposed to them in the same order, or was order of presentation randomized? Were participants tested one at a time, or in groups? How long did they dangle upside-down on the rope? How long did they stand? How long was the interval between these two experiences? How long was the memory test, in terms of time and number of items? What instructions were given to the participants? If there were long and detailed instructions, consider putting them in an appendix and referring the reader to that. However, if the instructions were short and fairly simple, this may not be necessary: it may suffice to mention them briefly only in this section of the report. Here’s an example from a face-recognition experiment in which measurements are taken of the speed with which participants can make judgements about whether or not a face is familiar to them.
‘Participants were asked to press the letter ‘S’ on the keyboard if the face was familiar to them, and the letter ‘L’ if it was not. They were asked to respond as quickly but as accurately as possible’.
As with the ‘Apparatus’ section, provide details only if they are needed for replication purposes, and if they are potentially relevant to the outcome of the experiment.
12.5 Summary
* * *
The ‘Method’ section sub-divides into the ‘Design’, ‘Participants’, ‘Apparatus’ and ‘Procedure’ sections. Together, they give the reader enough details for them to replicate the study should they wish to do so. The ‘Design’ section outlines the formal structure of the study: it identifies the experiment’s independent and dependent variables (i.e. the things that you, as experimenter, manipulate and measure, respectively). It also tells the reader whether you used an independent-measures, repeated-measures or ‘mixed’ design. The ‘Participants’ section gives relevant details of the people who took part in your experiment. The ‘Apparatus’ section provides information on what equipment you used to run your experiment. Depending on the nature of your research, ‘equipment’ may include questionnaires and pencil and paper tests, as well as computers and the like. Provide enough details for replication purposes, but don’t include irrelevant and unimportant details. The ‘Procedure’ section describes how you actually carried out the study in practice: again, include only details that would be required for replication purposes.
12. 6Practical Tasks
* * *
Identify the problems with the following procedure section:
Subjects were tested one at a time. Each subject arrived at the time that their appointment had been set for, and was asked to sit down at the table and to fill in the first questionnaire, which measured their mood before the experimental manipulations were administered. The questionnaire was administered via an EverKrash™ 1 GHz PC, with a 17 inch Trashiba™ monitor screen. This was placed by the window of the test room. The room measured approximately 10 ft by 15 ft. After they had completed the first questionnaire, which took abou
t 10 minutes, half of the subjects listened to a tape of sad music, while the other half listened to a tape of dance music. The mood of all subjects was then re-tested with a second questionnaire, administered under the same conditions as the first. Subjects were then debriefed and thanked for their participation in the study. The results were then analysed on the same computer by the experimenter, using Excel.
Answers:
The writer should refer to ‘participants’ rather than ‘subjects’.
It’s not really necessary to know that the participants had made an appointment to take part in the experiment and had arrived on time.
You need to use your judgement about whether or not to give precise details of equipment used. In this particular case, we probably don’t need to know the make and model of the PC and its screen because the computer is merely a vehicle for administering the questionnaire: presumably using a different computer – or even pencil and paper – would have made little difference to this particular study.
The computer’s location in the room is irrelevant detail that can be dispensed with.
We do not need to know the room’s dimensions in this case: and if we did, they should be presented in metric units, rather than imperial.
We are told how long participants took to complete the first questionnaire, but other – more important – information is missing: for how long did the participants listen to the mood-inducing music? How long afterwards did they complete the second questionnaire?
More details of the music should be given, such as the name of the composer and the title of the piece of music, so that the reader could use the same music in their own study if they so wished. Otherwise the study is potentially unreplicable – what constitutes ‘sad’ music? Does ‘dance’ music refer to modern popular music with no tune and an irritatingly monotonous beat or to the exquisite creations of someone like Johann Strauss?
We probably don’t need to know that ‘subjects were then debriefed and thanked for their participation in the study’: this should occur as a matter of course.
We do not need to know the mechanics of how the experimenter analysed the results, such as which computer was used and which software.
Here’s a better version of the same procedure section:
Participants were tested one at a time. Each participant completed the first questionnaire, which was administered by computer. After they had completed the first questionnaire, which took about 10 minutes, half of the participants listened to a tape of sad music (Samuel Barber’s ‘Adagio’) for ten minutes, while the other half listened for the same amount of time to a tape of dance music (J. Strauss’ ‘The Blue Danube’ waltz). The mood of all participants was then re-tested with a second questionnaire, administered under the same conditions as the first.
13 Answering the Question ‘What Did I Find?’ The Results Section
* * *
In this section, you present the results of your study. At this point, you don’t go into their theoretical implications – that will be covered in the next section of the report, the Discussion (see Chapter 14). All you do here is tell the reader what you found.
There are two basic kinds of statistics that need to be described here: descriptive statistics (such as group means) and inferential statistics (the results of statistical tests). Before giving the descriptive statistics, you may also need to provide information on any treatments that you have applied to the data.
13.1 Tidying Up Your Data
Sometimes when you run a study, the data need some kind of pretreatment before you carry out the processes of obtaining descriptive and inferential statistics. Take the example of reaction time data: every now and again, a participant produces an abnormally long reaction time (compared to the rest of their reaction times). This is usually for uninteresting reasons: their concentration might have lapsed on that particular trial, or perhaps they hesitated because they momentarily forgot which button to press. If you retain these long reaction times in your data, they may distort your findings or add so much ‘noise’ to the data that any effects which are lurking in the data become impossible to detect. Therefore, it’s considered OK to remove these spuriously long scores. This has to be done in a fair, principled way – you can’t merely discard data that you don’t like the look of, or which don’t fit in with your expectations! (In the case of reaction time data, one way to do it is to remove all reaction times longer than a certain pre-specified duration; or, if each participant is providing several responses, you can find the mean and standard deviation of them and then discount all scores which fall more than, say, 2 standard deviations above the mean. So that you don’t systematically bias the results, you must apply the same procedure to the data from all of the groups in your study.)
Because it is easier to get long reaction times than short reaction times, reaction-time data tend to be negatively skewed, rather than normally distributed around their mean. (See Chapter 4). This poses problems for statistical analysis, so some workers apply a corrective transformation (such as taking the logarithm of the reaction time, rather than using the raw RT) to compensate for the skew and make the data look more normally distributed.
Another way in which data may be treated is that certain participants may be excluded for one reason or another: perhaps they didn’t provide a complete set of data, or were unable to do the set task well enough to provide meaningful data. (I used to run face recognition experiments on Open University students. They are the most enthusiastic and obliging participants I have ever worked with. Unfortunately, they are often not much use for any experiment involving recognizing famous faces, because they don’t get time to watch the telly or go to the movies! Therefore I sometimes had to exclude participants from my results because they simply didn’t know the celebrities whose faces I was asking them to recognize. To have retained them would have distorted the results, because their data would have been mixed up with data from participants who did know the celebrities in question, but couldn’t recognize them from the pictures that I was using in my study.)
Any treatments such as these that you have performed on your data should be described clearly in this section, as they are obviously essential for a would-be replicator to know about.
13.2 Descriptive Statistics
* * *
Descriptive statistics provide summaries of group performance (see Chapter 4). They include things like averages (means, medians and modes), measures of how scores are spread out around the average (things like ranges or standard deviations) and frequency data, if relevant. The idea is to give the reader a clear idea of what you found, by presenting a clear and succinct summary of all the data that you have collected in your experiment. Thus, in the case of the chocolate and exercise experiment mentioned earlier, you would probably have collected mood measurements from each of the participants in each of the experimental conditions. The reader of your report would be spared all these gory details, however. You might provide them merely with the mean mood rating for each condition, each mean being accompanied by some indication of the spread of scores around it. (The most frequently used measure of spread is the standard deviation, but the American Psychological Association now recommend giving the standard error of the mean. These two measures of spread are covered in Chapter 4.)
Data Presentation
You are aiming to enable the reader to get an immediate idea of what you found, so things like means need to be presented clearly. If you have just one or two means, it may be simplest to mention them within the text itself. If you have more than a couple, it may be clearer to present them in a table or graph. Which should you use? There are no hard and fast rules about this. Sometimes, means and standard deviations are best presented in a table. Other times, it might be clearer to display them in a graph. If you have only a few means, graphs generally are clearer. If you have a lot of means, or if you want to present additional information such as standard deviations, ranges, number of participants per condition, etc., a table m
ight be preferable. Whichever you decide to use, avoid duplication of information. If you have shown data in a table, don’t present it in a graph as well – pick one method of presentation or the other, depending on which is clearest to the reader. (See Chapter 4 for details on how to do graphs).
Tables and ‘figures’ (graphs and any other pictures) are numbered in order of presentation. Numbering is done separately for tables and figures. Thus the first table is ‘Table 1’, the second is ‘Table 2’, and so on. The first graph or picture is ‘Figure 1’, the second is ‘Figure 2’, and so on, even if a table has been presented in between the two figures. Figures and graphs are always referred to in the text by these labels. For example, you might write ‘Figure I shows the mean performance in each condition’.
This may seem like labouring a pretty simple point, but many students appear to be unable to do this properly! A common mistake is to write things like ‘Graph 1’, ‘Picture I’ or ‘the accompanying graph shows’.
13.3Inferential Statistics