How to Design and Report Experiments
Page 41
American Psychological Association (2001). Publication manual of the American Psychological Association (5th edition). Washington, DC: APA Books.
Primary and Secondary References
Primary references are those that you have read yourself. Secondary references are references that you haven’t actually seen yourself; you have only read a description of them in someone else’s work. Ideally, you should aim to use primary references. This is generally preferable to obtaining information second-hand, via another author who might be providing a misleading, distorted or biased account of the original authors’ results and conclusions. (We’re not saying that secondary sources deliberately set out to mislead, but mistakes or misunderstandings can happen).
There are a few problems with using primary references. Firstly, the reference might be hard to read, for stylistic or technical reasons (it’s generally easier to read an account of Piaget’s work by someone other than Piaget, for example). Secondly, you might get bogged down in the details, in a way that you might not with someone else’s potted account of that research. Thirdly, you may not be able to get hold of the original article or book – a variant of Sod’s Law is that the articles that appear to be most interesting to you generally seem to be in the most obscure and inaccessible journals!
Because of these problems, you may be stuck with using secondary references, at least to some extent. There are two ways of referencing these. One is to lie, and pretend you have read the primary reference. The problem with this is that your tutor will probably know you are lying, especially if the primary reference is an obscure one. A good example of this was when one of our students referenced an unpublished conference paper – implausible to begin with, but even more implausible given that the student would have been about six years old when the conference took place!
The honest way to deal with secondary references is to acknowledge their origins. This is done as follows. Suppose we want to refer to work by Tardive (1995), which we have only read about in a later article by Gubbins (2000). In the text, we would write ‘It has been claimed that schizophrenia can be cured by blood-letting (Tardive, 1995, as cited in Gubbins, 2000)’. Alternatively, we might write ‘Tardive (1995, as cited in Gubbins, 2000) claimed that schizophrenia can be cured by blood-letting’. This makes it clear to the reader that Tardive’s work is being talked about, but that you are taking Gubbins’ word for it. Secondary references do not appear in the reference section: there, you would include a full reference for Gubbins (2000), but not for Tardive (1995).
15. 4 Appendices
* * *
Students often get confused about what should be placed in an appendix, and what should go in the main text of the report. To some extent, this will depend on the personal preferences of your tutor, but as a rough guide, the following should normally be relegated to one or more appendices:
(a) Statistical calculations and the output from computer statistical packages
If you calculated a t-test on your results, by hand, the working-out would all go in the appendix. The value of ‘t’ that was obtained, the degrees of freedom and the probability value would go in the ‘Results’ section. If you used a computer to do the hard work, then the print-out from the computer program would go in an appendix, and the value of ‘t’, etc., would go in the Results section.
In the ‘Results’ section of the report, one thing you shouldn’t do is to describe results which are buried in an appendix! If a result warrants discussion, then it belongs in the ‘Results’ section itself.
(b) Raw data
You might be required to supply the raw data (e.g. individual participants’ scores) for your study. If so, this should go in an appendix. But, as a general rule, don’t include the raw data unless your tutor specifically requests you to do so.
(c) Questionnaires and other pencil-and-paper tests
If these are home-made or otherwise unusual, you could supply them in an appendix. If they are widely-used, you wouldn’t bother; as discussed earlier, it would suffice to provide a reference in the ‘Apparatus’ section to where the test can be found.
(d) Detailed instructions to participants
Sometimes you might have provided lengthy and detailed instructions to participants. These might have been read out to the participant, or given to them on a sheet of paper or on a computer screen. These instructions could be placed in an appendix.
(e) Stimulus materials
If you had run a verbal memory experiment, you might want to include an appendix which contained the word lists or text passages used as stimuli. Had you run a face-recognition experiment which involved deciding whether or not faces were famous, you could include a list of the celebrities whose faces were used. If your experiment involved remembering a set of abstract geometric shapes, you might include pictures of all of these in an appendix.
Finally, if you include one or more appendices, then acknowledge their existence in the body of the report. For example, in the ‘Results’ section, you might write ‘Table 1 shows the mean score for each condition. (The raw data for these means can be found in Appendix 1)’.
15.5 Practical Tasks
* * *
Here’s a list of publications I might have used in a paper. Put them into a reference list, in the correct order and in proper APA format.
Gordon Runt wrote ‘My Life Amongst the Weeds’. This is a chapter in a book, published by Weekling Press, in Ohio, Baltimore, on the 21st November 2000. The book’s title is ‘Pushing up Daisies’, and it was edited by Norma Lee Breffless. The page numbers for Runt’s contribution were 356–393. He emailed me to say ‘it’s an awfully heavy book to carry around’ (Runt, personal communication October 31st 1999).
I haven’t seen Sid and Terry O’Coopatian’s new book for 2001, ‘Work – Why Bother?’, published by Aardvark Publishing Co., Ltd., Bognor. However, I have read Napindi Evenning’s summary of their ideas in her article ‘The idolatry of idleness’, in the December 2001 edition of the magazine ‘Work-shy’. It was on pages 1–5 in volume 26.
Max Zillofashel and his colleagues Den Tistry, Tief Urtin and Saul Ted Peenat produced two works in 1996. The first was a journal article entitled ‘A participant study of dental phobias’, in volume 86 of ‘The Journal of Irreproducible Results’. It was on pages 335–338. The second appeared in the 26th volume of ‘The Journal of Oral Psychiatry’, on pages 334–336. It was called ‘My life without teeth: a participant study’, and it appeared in June 1998. It had an additional four authors: Don Utt, Jerry Attrick, Beena Stripper and Paul Mateefout.
A seminal work on how alcohol makes everyone look more attractive was published by Vera Sloshed, Reilly Plasted and Ada Nuff in 1994: this was entitled ‘The effect of alcohol on perception of objects’, and it appeared in the weekly magazine ‘Cirrhosis’, on the 6th September. It was on pages 113–117. Sloshed and Nuff reprised this success with an article in ‘Liver Damage’ two years later: pages 10–22 of volume 86 of this esteemed publication were devoted to ‘My life on a slave barge: the ‘morning after effect’ of excessive alcohol consumption’.
In December 2000, Wilma Leggfaloff edited a book called ‘Recurrent Anxieties’. It was published by Nosedrip Press, in Oxford. Armin Payne wrote a chapter covering pages 25–40 in this book. Its title was ‘No, no no! Not again!’
Chester Minitt had a prolific year in 1983: he wrote three works. Two of them were journal articles: ‘Towards a theory of time estimation’ and ‘No time like the present’. These were pages 25–26 and 22–28 in the first and third parts of volume 66 of the ‘Journal of Clockwatching Behaviour’. His other work that year was a book entitled ‘Here today, gone tomorrow’, published by Doomsday Press (Mozambique) Ltd., but that was in collaboration with Ivor Stopwatch, who was first author.
Answers:
Minitt, C. (1983a). Towards a theory of time estimation. Journal of Clockwatching Behaviour, 66 (1), 25–26.
Minitt, C. (1983b) No time like the present. Journal
of Clockwatching Behaviour, 66 (3), 22–28.
Napindi Evenning, N. (2001, December). The idolatry of idleness. Work-shy, 26, 1–5.
Payne, A. (2000). No, no no! Not again! In W. Leggfaloff (Ed.) Recurrent Anxieties (pp. 25–40). Oxford: Nosedrip.
Runt, G. (2000). My life amongst the weeds. In N.L. Breffiess (Ed.) Pushing up daisies (pp. 356–393). Ohio: Weekling.
Sloshed, V. & Nuff, A. (1996). My life on a slave barge: the ‘morning after effect’ of excessive alcohol consumption. Liver Damage, 86, 10–22.
Sloshed, V., Plasted, R. & Nuff, A. (1994, September 6) The effect of alcohol on perception of objects. Cirrhosis, 113–117.
Stopwatch, I. & Minitt, C. (1983) Here today, gone tomorrow. Mozambique: Doomsday.
Zillofashel, M. (1996). A participant study of dental phobias. Journal of Irreproducible Results, 86, 335–338.
Zillofashel, M., Tistry, D., Urtin, T., Peenat, S.T., Utt, D., Attrick, J. et al. (1998) My life without teeth: a participant study. Journal of Oral Psychiatry, 26, 334–336.
Note: Runt’s personal communication does not appear in this list. Neither does Sid and Terry O’Coopatian’s book, because that was a secondary reference. Therefore, I’ve only included the reference that I have read, that mentioned it (i.e. Evenning’s article).
16 Example of an Experimental Write-Up
* * *
Here is an unpublished study carried out by Jayne Grimes and Graham Hole. It looked at the effects of age and training on children’s ability to resist being influenced by an interviewer’s distortions of their accounts of events. Comments on what has been written and why, are given throughout.
[A succinct but informative title, giving the reader a clear idea of the study’s topic:]
‘Empowering the child witness to correct interviewer’s distortions of their testimony.’
16.1 Abstract
[The purpose of our study:]
An experiment based on the work of Roberts and Lamb (1998) was performed to determine whether children could be trained to correct an interviewer’s distortions of their description of a staged event.
[An outline of what we did:]
Forty children, in two age groups (mean ages of 6 years 7 months, and 9 years 6 months) saw a staged event, and were then interviewed three weeks later. Before the interview, half of the children in each group were trained to correct any distortions made by the interviewer.
[A summary of what we found, and our conclusions:]
Older children recalled significantly more correct information than did younger children, and trained children corrected significantly more of the interviewer’s distortions than did untrained children. No relationship was found between the amount of correct information remembered and the number of distortions corrected by the child.
16.2 Introduction
* * *
[A description of previous research in this field, as a way of providing a rationale for our own study:]
Considerable research has surrounded the limitations of adults and children in providing eyewitness testimony, and explored the role of both social and cognitive factors influencing the quality of their recall. In particular, research has demonstrated the susceptibility of eyewitnesses to misleading post-event information. Since Loftus’ early studies (e.g. Loftus, 1979) it has been known that adult witnesses’ descriptions can be corrupted by false information supplied to them in the guise of questions. Witnesses may subsequently report the false information, rather than their original memories of the event.
These effects have also been demonstrated in children (e.g. reviews in Ceci & Bruck, 1993; Dent & Flin, 1992; Spencer & Flin, 1993), and it has been suggested that children are especially prone to ‘interrogative suggestibility’ (Gudjonsson, 1992) of this kind.
Several reasons have been advanced for why children might be more suggestible than adults. Children might have greater difficulty in distinguishing between real and imagined events (Johnson & Foley 1984; Lindsay, Johnson & Kwon, 1991) or they might be more liable than adults to conform to the perceived authority of the interviewer (e.g. Goodman, Sharma, Thomas & Considine, 1995), especially given that they may tend to assume that the adult knows the answers to the questions being asked.
There is some evidence that younger children are more susceptible than older children to misleading information and leading questions (Ceci & Bruck, 1993; Goodman & Reed, 1986; Shrimpton, Oates & Hayes, 1998). Younger children do not remember as much as older children when recalling an event during the free recall phase of an interview (Dent & Stephenson, 1979), and so might be more likely to incorporate into their accounts information derived from the interviewer’s subsequent directed questioning.
Some researchers (e.g. Bjorklund, Cassel, Bjorklund, Brown, Park, Ernst & Owen, 2000; King & Yuille, 1987) have suggested that the child witness is susceptible to leading questions due to the perceived superior status of the interviewing adult. As Bjorklund et al. put it, ‘Because young children view adults as omnipotent, cooperative, and truthful conversation partners, in their attempts to be socially compliant, they agree with information they believe adults want to hear’ (page 422).
[Homing in now, on studies that are more specifically relevant to the one that we are reporting:]
It is claimed that the child effectively conforms to the suggestions put to them. This suggests that if the child were given practice in not conforming to the interviewer’s suggestions, and was made to feel that they had permission to contradict the interviewer, the influence of the interviewer’s authority on the accuracy of the child’s recall might be minimised.
Goodman et al. (1995) conducted a study that investigated the accuracy of four year olds’ recall of a play event in which they had just participated. Half of the children were interviewed by an unfamiliar interviewer, and the rest were interviewed by their own mothers. The mothers and strangers were either biased interviewers (having been provided by the experimenter with inaccurate information about the play event) or unbiased interviewers (provided with no prior information). Children gave less accurate accounts of the event to the unfamiliar interviewers than to their mothers.
Dent (1982) compared the effectiveness of experienced and inexperienced interviewers in eliciting information from children. Each interviewer questioned five children about a staged incident. Not only did the experienced interviewers elicit more correct information from the child but they also obtained more incorrect information from them. Interestingly, it was the interviewers with a strong preconceived idea about the incident who elicited the greatest amount of incorrect information. In the Goodman et al. study, the biased group of interviewers (which contained both mothers and strangers) had been given some information about the event the child was participating in; however, this information was not always accurate. The results also indicated that the children were more susceptible to the leading questions by the strangers than they were their mothers, perhaps because of the former’s greater authority.
Goodman and Rudy (1991, cited in Spencer & Flin, 1993) conducted a series of experiments that assessed how suggestible children were when they had participated in a situation that was both stressful and personally significant, such as receiving an inoculation or providing a blood sample. Children as young as four were able to resist the suggestive questions from the interviewer that concerned potentially abusive actions made by the adult conducting the medical procedure. Even after a delay of about a year, the children were unlikely to make false accounts of the events as a response to the suggestive questions. These results indicate that in some circumstances, children are able to resist being influenced by leading questions and the authority of the interviewer.
However, misleading information can come in a variety of forms. One is prior to the child providing their account of the event, for example by the use of leading questions. There is, however, another form that does not appear to have been very widely researched: the effects of interviewers’ distortions of the child’s acc
ount of events during paraphrasing. Distorted paraphrasing may happen unintentionally during interviews, if the interviewer holds erroneous preconceptions about the topics on which information is sought from the child; in courtroom situations, it may also happen deliberately, if the interrogator is seeking to cast doubt on the accuracy of the witness’ testimony.
Roberts and Lamb (1998) reviewed transcripts of children interviewed in relation to suspected sexual abuse. While it was found that few of the interviewers used leading questions, some distorted the child’s account of the events that had happened. Although this was infrequent, when it did occur the children tended not to correct the interviewer.
[Next, we provide a specific rationale for doing our study – to ‘plug a gap’ in our knowledge of this area, inspired by a suggestion from previous researchers in this field. We briefly outline what we are going to do, and we finish with a clear but informally-stated description of what we expect to find: effects of age and training on children’s ability to contradict the interviewer’s distortions of their testimony:]
Roberts and Lamb suggested that with training and practice, children might be empowered to correct interviewers. The present study is a test of this suggestion. The primary aims were to examine the effects on children’s recall of misinformation introduced by the interviewer’s distorted paraphrasing of their testimony, and to see whether training would empower the child to correct the interviewer’s distortions.