How to Design and Report Experiments
Page 33
The precise length of each section will vary, depending on the nature of the study (some studies have more complicated procedures than others, for example, and so need a lengthier description in the ‘Procedure’ section). However a reader would expect to find all of these sections in the report, in this particular order:
• Title
• Abstract
• Introduction
• Method (sub-divided into the following sections:) Design
Participants
Apparatus and Materials
Procedure
• Results
• Discussion
• References
• Appendices (not always present).
These sections answer four basic questions:
‘Why?’ Why did I do this particular experiment? What did I expect to find out by doing it? This question is dealt with in the Introduction section.
‘How?’ How did I actually carry it out – what procedures and apparatus did I use? This question is covered in the Method section.
‘What?’ What did I find? What were my results? This information is provided in the Results section.
‘So What?’ What does it all mean? How do my results relate to previous research on the same topic, and what are their theoretical implications? What are my conclusions? These issues are all dealt with in the Discussion section.
What about all the other bits – the Title, Abstract and References? These have important functions too. The Title enables the reader to get a very quick idea of what the report is about. If they are still interested, they can read the Abstract, which is a summary that provides a brief outline of the main procedures, results and findings of the study. The Reference section allows the reader to obtain further information on the topic of the report, by providing full details of any previous work that the author has referred to. (It also enables the reader to check that the author hasn’t misquoted or misrepresented the work of others in the course of citing them!)
The overall length of the report will vary according to the precise nature of the study that’s being described, and on whether your tutor requires you to include Appendices. As a very rough guide, a journal article which describes only a single experiment usually comes out to no more than 20 pages or so of double-spaced manuscript, including references, tables and graphs: about 5000 words in all. The Introduction and Discussion account for about half of this total length.
In the following chapters, we are going to look in detail at what should be included in each of these sections. See also the checklist in Box 9.1 at the end of this chapter.
9.2 Title
* * *
Provide a succinct title of no more than about 15 words. If it were the title of a journal article, it would be informative enough to enable the reader to identify the paper from the journal’s index as something that they would be interested in reading: for example ‘Sex differences in problem solving behaviour’ rather than ‘Sex differences’ or ‘An experiment on problem solving’. The following sections will describe a report that could be entitled ‘The effects of nationality and age on sun-bed claiming behaviour’. fairly self-explanatory, isn’t it? Your title should be too.
9.3 Abstract
* * *
This is a brief summary (150 words maximum) of the report. (The American Psychological Association stipulate a 120-word limit, but we’ll settle for 150 for now). It gives the reader a quick idea of what you did, the main results, and their theoretical implications. It’s easiest to write this last, once you have written the rest of the report. Here’s the abstract to go with our study of sun-bed claiming behaviour: note that it’s only 119 words long, but it gives all the essentials of the study.
The effects of nationality (German, English or American) and age (‘young’, 20–30 years; or ‘old’, 60–70 years) were measured on latencies to claim sun-beds at an international resort. Ten males of each nationality (five for each age-group) were selected randomly and covertly filmed during the 30 minutes after the pool was opened in the morning. The speed with which each individual moved from the dining room to the sun-bed was recorded. Significant effects of nationality and age were found, but no interaction between them. Germans were faster than the British, who in turn were faster than the Americans. The young of all nationalities were faster than their older counterparts. It is concluded that national stereotypes have some basis in fact.
9.4 Introduction
* * *
This part of the report introduces the reader to the topic on which you are going to do your experiment, and provides a justification for why you did the experiment. You provide some background information about previous research in this area, and explain why your study was worth doing – how is it likely to add to our knowledge of this topic? Your experiment might aim to plug a gap in our knowledge, or clarify some issue which has arisen from previous research – perhaps previous experiments have produced inconsistent or conflicting results, or perhaps experiments have been done in two separate areas but no-one has thought of linking them together before.
Previous work is cited in a standardized way: in the text of the report, you refer to all previous work by means of the authors’ surnames followed by the date of publication (e.g. ‘Bonkers (1955)’, ‘Twitch and Cackle (1976)’ etc.); at the end of the report, there is a reference section which gives the full reference for each work mentioned in the text. See Chapter 15 for full details.
The final paragraph or so of the introduction should outline your proposed experiment, and state (in an informal way) what you predict your results will be, given your knowledge of previous research in this area. Here’s an abbreviated example of an introduction (in practice, you might include more information on previous studies and theories):
In recent years, there has been considerable interest in national stereotypes and the extent to which they are valid. Ever since Biggott (1967) reported that French shoppers were significantly more likely to push into a bus queue than were Swiss shoppers, studies have been performed that appear to show that reliable cross-cultural differences exist in what is considered ‘acceptable’ behaviour, even within the Western ‘developed’ nations. For example, Raciste, Morone and Kruelle (2000) recently presented evidence that people from Alsace are significantly more likely to consider dog-beating acceptable than are people from Labrador. Wikked and Hartless (2001)found that 95% of Welsh interviewed claimed that they had watched ritual poodle-drowning; in contrast, 68% of Swedes claimed to find this practice abhorrent.
One problem with all of these studies is that, with the exception of the original work by Biggott, they rely on responses to questionnaires: given that there is often some disparity between what people say they do and their actual behaviours (ThynKin, SeyYing and Doowing, 1978), the questionnaire studies may have overestimated the strength of these cultural variations.
One behaviour which has attracted considerable interest is sun-bed claiming: the establishment of priority of access to a sun-bed at a resort by means of placing a towel on it. While there have been previous studies of this phenomenon, they are either so old that cultural practices might have changed in the meantime (e.g. Buonaparte and Nelson’s (1805) study of sun-bed claiming behaviour on the Western European coast) or they have failed to use objective behavioural measurements (e.g. as in Krapp and Fewtile’s (1966) study, in which individuals of two countries were asked to give ratings of the acceptability of each other’s toenail-clipping behaviour). Also, previous studies have failed to take account of the age of the participants, and yet recent research has shown this to be an important variable in cross-cultural behavioural variation. For example, Kebbab, Burghur and Schnitzel (1995) have found that European young people are more pushy at supermarket checkouts than American young people, whereas the reverse is true for old people.
The present study therefore sets out to examine age and cultural differences in an overt behaviour (sun-bed claiming behaviour around a hotel pool) using a valid and o
bjective measure of performance: the speed with which individuals moved from one clearly-defined part of the hotel (the dining room) to the sun-bed. On the basis of previous research, it was predicted that there would be national differences in this behaviour which conformed to widely held national stereotypes–namely, that German tourists will be faster to claim sun-beds than American tourists, who in turn will be faster than the English. It was also expected that there would be some form of interaction between the age and nationality of participants, although the precise nature of that interaction is difficult to predict in advance.
9.5 Method
* * *
This tells the reader what you did in your experiment, in enough detail that they could replicate the study in all its important details. It breaks down into sub-sections.
Design
This gives details of the formal design of the experiment – such as whether it was an independent- or repeated-measures design (see Chapter 3). It identifies the independent and dependent variables in the study. Remember, the independent variable is what you manipulate, in your role as experimenter, and the dependent variable is what you measure. Here’s our ‘design’ section:
This study used a between-subjects design. There were two independent variables: nationality (with three levels: English, German or American) and age (with two levels: 20–30 years old or 60–70 years old).
The dependent variable was ‘sun-bed claiming speed’, defined as the time (in seconds) that it took a participant to run from the hotel dining-room to a sun-bed by the hotel swimming pool.
Participants
Give details of who took part in your experiment: provide details of their sex, age and any special characteristics of them that might be relevant to your particular experiment (e.g. handedness, bilingualism, etc.). State whether they were volunteers; whether they were paid for participating; how they were allocated to the different conditions of the study; and of course, how many there were per condition.
There were 30 participants (10 German, 10 English and 10 American), residents of the ‘Hotel Ripov’ during the first week of July 2000. Half of each nationality were between 20–30 years of age (M 26, SD 3.2), and the rest were 60–70 years old (M 64, SD 4.8). All were male, and free from any obvious physical or sensory impairments. Participants took part in the study unwittingly, and therefore remained completely naive about the aims and purpose of the study.
Apparatus
‘Apparatus’ in this context means things like stopwatches, computers, questionnaires, etc. Give important relevant details (e.g. brand-names and model numbers if the equipment is unusual), but omit trivial and unnecessary details like whether they used an HB pencil or a biro to fill in a questionnaire! Write this section in full English sentences, not as a ‘shopping-list’ of equipment.
Participants’ running speeds were measured with a hand-held stopwatch. A video-camera was used to film the participants’ behaviour: this was done so that inter-rater reliability checks could later be made on the accuracy with which running speed had been recorded, and also to provide a means of enabling the hotel staff to identify the participants and thus provide the experimenter with information about their nationality and age.
Procedure
Explain how you actually carried out the experiment in practice. Give details of exactly what was done to participants; what they had to do; the order in which tests were administered; and how long test sessions took.
From 7.30 to 8.00 a.m. each morning, the experimenter hid in a clump of bushes in a position that enabled him to see both the hotel’s swimming pool and the exit to the hotel’s dining room. As a person passed through the French windows of the dining room, the stopwatch was started. It was stopped when the person either placed their towel on a sun-bed (thus establishing ‘ownership’) or sat or lay on the sun-bed. This procedure was followed for one week. At the end of each day’s covert filming, the film was shown to the hotel manager, who identified the guest and provided information about the guest’s nationality and age. The first ten people of each of the predetermined permutations of nationality and age that were filmed, were chosen to be the experimental participants.
9.6 Results
* * *
This section falls into two parts, although they don’t have sub-headings. First, give descriptive statistics, such as means and standard deviation for each group or condition. Follow these with inferential statistics – the results of statistical tests used to decide whether any differences between groups or conditions were ‘real’ as opposed to merely due to chance. (See Chapters 5, 6 and 7). For the inferential statistics, state which test you used; the value of the test statistic; the number of degrees of freedom (where appropriate); and the significance level for this. Although generally you do not need to provide a justification of why you picked a particular test – it is usually self-evident – there are circumstances when you do (for example, explaining why you chose a non-parametric test, or saying why you picked a particular post-hoc test). One thing you should never include is the calculations for the test.
In most cases the information can be inserted into the text like this: ‘an independent-means t-test was performed. This showed that participants who had received 40 mg of the drug ‘Pukupp’ recalled significantly fewer words than those who had consumed 20 mg (t (29) = 3.65, p < .001)’. If you have lots of results, consider using tables or graphs to display them. However, don’t duplicate information unnecessarily: if the results are shown in a table, don’t also show them in a graph, or vice versa. Make sure that all tables and graphs are clearly labelled with self-explanatory titles and legends. A good rule of thumb is that the text of the results section should be intelligible to a naive reader without reference to any tables or graphs; and similarly, the tables and graphs should be understandable without reference to the text.
Don’t put raw data in this section – put them in an appendix at the end of the report if your tutor says you need to provide them. Explain in words what the descriptive and inferential statistics show, but don’t interpret them – that’s left until the next section. (So, for example, in this section we describe the data on sun-bed claiming speed that we have recorded, and say whether there were any significant differences between the ages and nationalities. Here, you wouldn’t speculate about why these differences had occurred, or relate these findings to previous data or theories on age and cultural differences in behaviour – all of that should be left until the Discussion).
Figure 9.1 shows the mean latency to claim a sun-bed (time from dining room to sun bed) for each permutation of nationality and age. (Note that the shorter the latency, the faster the participant). The error bars represent the 95% confidence interval for the mean. Inspection of Figure 9.1 suggests that the three nationalities differed in sun-bed claiming speed, with the Germans being fastest, the Americans slowest, and the English falling between these two extremes. There also appears to be some effect of age, with the younger participants of all nationalities being somewhat faster overall than their older counterparts.
A two-way independent-measures ANOVA (nationality: three levels, American, German and English; age: two levels, younger and older) was performed on these data. There was a significant main effect of nationality (F (2, 30) = 21.03, p < .0001, r = .36). Post-hoc tests revealed that, overall, the German tourists were faster to claim a sun-bed than were the English tourists, who in turn were faster than the Americans (Bonferroni tests, p < .05 for all tests). There was also a significant main effect of age (F (1, 30) = 14.88, p < .01, r = .28): regardless of nationality, younger tourists were faster to claim a sun-bed than were older tourists. From Figure 9.1, it appears that the effects of age were more marked for the Germans and English than they were for the Americans. However, the ANOVA failed to support this interpretation, revealing no significant interaction between age and nationality (F (2, 30) = 2.34, ns, r = .04).
Figure 9.1 Effects of nationality and age on sun-bed claiming behaviour
9.7 Discussion
* * *
Start by briefly restating the main results, in words. Say whether or not they support your experimental hypothesis (as stated at the beginning of the Introduction). Then relate your findings to those of previous studies: do your results support previous work, refute it, or force a re-evaluation of earlier findings? If your results are at variance with previous work, why do you think this has happened? What theoretical implications does this have? Basically, you are assessing your experiment’s contribution to knowledge in this area of psychology. What faults or limitations did your study have? Do these seriously affect confidence in your findings? How might they be remedied in future work? Suggest possible worthwhile future experiments in this area. Finish by summarizing the main conclusions that can be drawn from your study.
Clear effects of nationality and age on sun-bed claiming behaviour were found in this study: German tourists were faster than English tourists to claim a sun-bed, and the English were in turn faster than the Americans. For all nationalities, younger tourists were faster to claim a sun-bed than were older tourists.