How to Design and Report Experiments

by Andy Field

  For most first-year psychology practical reports, providing a rationale is sometimes a little artificial: you are unlikely to be doing an experiment that hasn’t been done before. However, the rationale described in this section is probably the one that applies best: you will probably have replicated some well-established psychological phenomenon, perhaps with a slightly novel twist.

  To Resolve Some Anomaly that has Arisen in Previous Research on this Topic

  Suppose that Dr. Bullworker has found that exercise improves mood, whereas Dr. Flabby has found the opposite. What has produced this discrepancy in their findings? Perhaps neither has considered the effects of chocolate consumption as an influence on mood: in Bullworker’s study, the exercisers ate chocolate before exercising, while in Flabby’s study they didn’t. There is a need for another study to be performed, that systematically manipulates those factors that differed between the Bullworker and Flabby studies.

  11.2How to Describe Previous Research and its Findings

  So one function of the Introduction is to explain to the reader why you are performing your experiment. To explain what the issues are, you need to provide background information about relevant previous research and theory in this particular area. You must confine discussion to material that is directly relevant to your study. So, for example, if your project was on short-term memory problems in the elderly, the relevant stuff is previous research on that topic, and not on memory in general, or the elderly in general. Try to keep the description of the background material focused. A common mistake is to be too unselective: in this case, for example, some students would ramble on about memory in general and end up with a mini-essay that could have been entitled ‘everything that I could find out about memory from a basic textbook and pack into three sides of A4’.

  In referring to previous research, you need to describe it at an appropriate level of detail. For most studies that you cite, you will probably need only to give a bare outline of the procedures used and the results that were obtained. The reader usually doesn’t need to know how many participants were involved, what statistical tests were performed by the previous authors, precise details of the procedures used, and so on. If they want all of that information, they’ll just have to go to the original source to read it for themselves. (That’s one of the reasons why you provide detailed references to any work cited).

  Here are examples of the level of detail that will usually suffice:

  ‘Pike, Kemp, Towell and Phillips (1997) found that moving faces can be recognized more accurately than faces seen in multiple static views that provide comparable amounts of perspective information about the face. This suggests that motion per se may provide important information for face recognition, perhaps by aiding in the derivation of 3D structure.’

  ‘Rensink et al.’s (1997) demonstrations of ‘change blindness’ show that observers often fail to notice quite marked changes that are made between successive views of a scene.’

  We don’t need to tell the reader any details of how these results were obtained, unless those details are important to our argument. Only if we thought that these workers’ testing procedures contained a serious flaw, would we consider providing a little more detail on those procedures. So, for example: ‘Rhubarb’s (2000) conclusion was based on data from only five participants, and subsequent studies (e.g. Custard, 2001), using larger numbers of participants, have failed to replicate his findings’. Here, the number of participants is a potentially important issue, and we would mention it; if Rhubarb had used enough participants for this not to affect his conclusions, then we wouldn’t bother discussing it.

  Only if a previous study is central to your own research (as, for example, when your experiment is a direct replication of a previous study, or uses a complicated procedure that was used in previous research) should you go into any great detail – and even then, you should focus on what the reader needs to know to understand what you are doing, or to get a grasp of the issues involved in this area of research. As a rough guide, if you have spent more than half a page of A4 on describing a single study, then you are probably going into too much detail.

  One aid to writing the rationale for your study is to imagine that you are at a party, and that you have trapped someone in a corner of the room. This person is a psychologist, but not a specialist in the area of research within which you have performed your study. They make the fatal mistake of asking you what you are researching: you explain that you have just completed an experiment on how many words people can remember while their head is immersed in a bowl of condensed soup. The psychologist then compounds their error by saying: ‘Really? How interesting. Why did you do that experiment? What does that tell us about memory?’ You answer their question, by explaining what previous studies on soup immersion and memory have found, and what remains to be discovered. Unless you are seriously lacking in social skills, (in which case you’re probably a psychology lecturer and this analogy won’t help anyway) you would not respond to their question by telling them everything you knew about memory or psychology in general. For fear of them making a feeble excuse and leaving you alone in the corner of the room, or committing hara kiri on the carpet in front of you, you also wouldn’t go into too much detail about the previous studies in the area, or ramble on for hours about the theoretical issues.

  11.3Outlining Your Own Experiment

  * * *

  Once you have given the reader a fair idea of what the issues are, and why you have decided it is worth doing your experiment, it is time to outline what it is you propose to do. You only need to give a fairly general outline, since all of the minutiae are going to be described in the following sections. Suppose we wanted to resolve the discrepancy between Bullworker and Flabby’s studies on the effects of exercise on mood. We might have four group of participants: chocolate-guzzling exercisers, chocolate-guzzling non-exercisers, chocolate-deprived exercisers, and chocolate-deprived non-exercisers. We could measure each participant’s mood after a week of whichever of these four treatments they had been assigned to. If our hypothesis is that mood is affected by the combination of chocolate and exercise, we might predict that the chocolatey exercisers should be the happiest of all, followed by the chocolatey couch potatoes and the non-chocolatey exercisers. The non-chocolatey couch potatoes should be the most miserable of all.

  Something like the previous paragraph is all the level of detail that you need to write at this stage – all the particulars of how the groups will be treated and measured are deferred until later in the report. Explain briefly why you expect these particular outcomes, from a theoretical point of view. For example, here we would probably have already discussed the effects of exercise and chocolate on endorphin levels in the brain, and so this would be the motivation for our predicted outcomes.

  It’s a little dishonest, but sometimes outcomes are ‘predicted’ after the results have been analysed. With the benefit of hindsight, you can often see what you should have expected in the first place; it’s then easy to rewrite the Introduction to make your ‘predictions’ fit the data that you actually obtained. This accounts for why the Introductions of journal articles sometimes seem amazingly prescient about what the results are likely to be – it’s because they were known to the author at the time of writing the Introduction!

  11.4Providing Predictions About the Experiment’s Outcome

  * * *

  One thing that A-Level and Access to Higher Education students often do is to make formal statements of the ‘experimental’ (or ‘alternative’) and ‘null’ hypotheses at the end of the Introduction section. In other words, they write something like this:

  ‘The experimental hypothesis is that there will be a statistically significant difference between the four groups. The null hypothesis is that there will be no statistically significant difference between the four groups.’

  Real psychologists never write this kind of stuff in their Introductions. Just outline the study’s predictions in a clear
but informal way, like this:

  ‘If chocolate consumption and exercise interact in their effects on mood, then it is predicted that the four groups will differ in their moods as follows: the chocolate-consuming exercisers should produce the highest mood ratings, the non-chocolate consuming non-exercisers should give the lowest ratings, and the other two groups should produce mood ratings somewhere between these two extremes.’ Don’t bother stating the null hypothesis – because it’s invariably that there is no statistical difference between the experimental conditions, it goes without saying and hence is completely superfluous.

  Essentially, what you are trying to do at the end of the Introduction is to produce a cliff-hanger: you are leaving the reader on the edge of their chair, desperate to read on and discover what results you actually obtained. First though, you have to tell them exactly how you carried out the study.

  11 .5Summary

  The Introduction’s main purpose is to explain to the reader the reasons why you did your study. What has been done in this area before? Why is your study worth doing?

  Introductions normally have three main parts: a description of relevant previous research and theories; a brief outline of your experiment; and an informal summary of what you expect to find and what your findings might imply theoretically.

  Reasons for doing a study include all or some of the following: to test a theory; replicate or extend previous findings; or to try to resolve anomalous or contradictory findings from previous work.

  In referring to previous research, make sure you stick to studies which are directly relevant to your own, and avoid going into too much detail about them.

  The Introduction ends with a statement of what you expect to find: write this in everyday English, rather than the stilted formal exposition of hypotheses beloved of A-level tutors!

  11.6Practical Tasks

  * * *

  Suppose you wanted to write the Introduction to the following experiment. Which of the following numbered points would you include in the Introduction of a report entitled ‘Realism as a Factor Affecting Observational Learning in Children’? Which would you leave out, and why?

  Bandura, Ross and Ross (1963) showed that children imitated aggressive behaviours performed by an adult towards an inflatable doll called ‘Bobo’.

  It has been inferred from this research that children will copy cartoon characters performing aggressive acts, and that therefore cartoon violence is undesirable. However, Bandura’s studies used a human actor: perhaps children may not be so susceptible to imitating cartoon characters behaving aggressively, since they know that these are not real people.

  In the present study, the extent to which the actor looked human was varied. There were two conditions. In the ‘human’ condition, children saw a male actor behaving aggressively towards a Bobo doll. The ‘rabbit’ condition was identical, except that a man dressed in a rabbit suit performed the aggressive acts.

  Twenty children recruited from a local primary school took part in this experiment. Permission for them to take part was obtained from their parents. The children’s mean age was 5 years and 6 months. Ten were allocated randomly to the ‘human’ condition, and the rest were allocated to the ‘rabbit’ condition.

  The principles of operant conditioning were originally worked out using mainly rats and pigeons, but they have been applied to humans as well. For example, the behaviour of gamblers has been explained as being due to them operating on a variable-ratio schedule of reinforcement.

  Piaget (1963) claimed that children pass through several stages in their development. At around the age of six or seven (the age of the children in the present experiment) they enter the stage of ‘concrete operations’, when their thinking is dominated by physical appearances.

  Even children less than a year old can imitate previously seen actions: for example they may copy an adult who pokes out their tongue (Meltzoff and Moore, 1977).

  Psychologists have studied children’s development for over a hundred years, but still have much to learn about it.

  The experimental hypothesis is that children in the ‘human’ condition will be significantly more likely to imitate the aggressor against the doll than will children in the ‘rabbit’ condition, as measured by an independent t-test comparing the two conditions. The null hypothesis is that there will be no difference between the two conditions, in terms of their propensity to copy the aggressor’s acts.

  Developmental psychology investigates how cognitive, emotional and social behaviour develop during infancy and childhood, as a result of the interaction between genetics and experience.

  Answers:

  Include: it’s a description of a study that is relevant to the topic of the current research.

  Include: this provides a rationale for the current study. It shows that there is a problem with the conclusions that have been drawn from previous research, and hints at what kind of study needs to be done to address this problem.

  Include: it gives a brief overview of the experiment that is the subject of this report. It is only an outline: fuller details will be given later in the report.

  Omit: the level of detail about the participants is more appropriate to the ‘Method’ section than the ‘Introduction’. All we need to know at this stage is that 5 year-old children took part in two conditions in the experiment.

  Omit: the present study is on social learning, not learning in general. This study on operant conditioning appears to have no relevance to the current experiment.

  Omit: a discussion of theories of social learning or social development might be appropriate, but there seems no good reason to discuss theories of cognitive development here.

  Omit: this study is about imitation, but has apparently only superficial relevance to the present study, which is on much older children and involves a different kind of imitation.

  Omit: this is such a superficial and axiomatic statement that it’s just a waste of space.

  Omit (most of it, anyway!): the first part of the first line is worth including. In this section of the report, there’s no need to state which statistical test will be used; that can be left until the ‘Results’ section. The second sentence, about the ‘null hypothesis’, can be dispensed with: because it merely says the opposite of the ‘experimental’ hypothesis, it goes without saying.

  Omit: you don’t need to define ‘Developmental psychology’ for the reader – you can assume they know that much!

  12 Answering the Question ‘How?’ The Method Section

  * * *

  The idea of this section is to provide the reader with a clear idea of what you did and how you did it. The standard by which this section will be judged is: would it be possible to replicate this study, in all its important aspects, solely on the basis of the information that is provided in this section? The tricky bit is to walk the tightrope between providing too much information (including irrelevant information) and too little.

  This section is sub-divided into several sections. The ‘Design’ section provides an overview of the overall structure of your experiment; the ‘Participants’ section provides the necessary information about the people who took part in your experiment; the ‘Apparatus’ section gives details of any equipment used (including things like questionnaires and other tests); and the ‘Procedure’ tells the reader how the study was carried out in practice.

  12.1 Design

  * * *

  This section provides an overview of the formal design of the study. What were the ‘independent variables’ in the study? (Independent variables are the factors that you, the experimenter, manipulate: see Chapter 2).

  How many conditions were there? Was it a repeated-measures design (with each participant taking part in all of the conditions of the study), an independent-measures design (in which participants performed in one and only one condition), or a more complex, ‘mixed design’ (using a combination of repeated-measures and independent-measures variables)? (See Chapter 3). What was measur
ed – in other words, what were the dependent variables?

  This may all seem rather arcane, so here’s a concrete example that will hopefully clarify what’s required. Suppose you are interested in factors affecting how well factory workers can spot black cornflakes on the cornflake production-line. You might devise an experiment in which you examine the effects of age and time of day on workers’ ability to detect the dodgy cornflakes. You obtain groups of young, medium and old workers. For each of these age-groups, you have a group of morning-shift workers, and a group of evening-shift workers. You measure the number of black cornflakes that each worker detects during a four-hour shift. (You cunningly arrange it so that, unknown to the workers, the same number of black cornflakes is presented during each worker’s shift).

  The formal design section for this experiment would look something like this:

  ‘There were two independent variables: age of worker (with three levels: young (21–30 years), medium (31–40 years) and old (41–50 years)) and time of shift (with two levels: morning or evening). The experiment used a wholly between-groups design, with each worker assigned to only one of the permutations of age and shift. The dependent variable was the number of black cornflakes detected within a shift (out of a maximum of 20).’