species known to the testers.
Species used to test
diagnostic characters must
be carefully chosen to
represent a range of states
of those characters (e.g.
pairs of species that have
different types of hairs,
different thorn shape, etc.).
and
was
X
X
Some approaches to testing
? How do you Y
able 9.1
Examples of questions
to ask
What species is this?
What do you like/dislike
about the illustration?
What do you not
understand?
Which do you read fastest
and why?
Which do you prefer and
why?
What is the difference
between species
species
know they are those
species?
On this specimen, would
you say character
a, b or c (for example,
would you say the leaves
are simple, pinnate or
bipinnate)?
T
Example of a task
Recognize the species
illustrated in a range of
illustrations.
Read and comment on
species descriptions or
other written components
of the guide.
Comparison of pairs of
species that differ in few
but significant ways,
asking testers to note
differences between the
species.
Examination of single
specimens to describe
character state
represented.
What are you testing?
Illustrations that help to
confirm identification.
Comprehensible text
Diagnostic characters
that are observable,
understandable and help
to reach an accurate
identification
Testing the field guide 221
acilitators observe testers for signs
acilitators observe testers for signs of
Depending upon the testers,
facilitators can ask them to report
verbally their conclusions and
points of difficulty in the keys, or to
write their answers on a form.
F
of confidence, worry or confusion.
Interview testers afterwards to
understand their reasons for
successful or unsuccessful
completion of the task.
Testers work individually or in groups to
identify species or answer questions.
In this case, it is best if facilitators do
not interact with testers while they are
performing the tasks as it is important
to see if the task can be completed
using the tools available in the guide.
F
confidence, worry or confusion.
Observation is particularly important in
the usability test as it is the most
complex of the tasks that can be set
and involves using the whole guide, so
careful attention must be paid to the
parts of the guide used.
Interview testers afterwards to
understand their reasons for successful
or unsuccessful completion of the task.
-up of the whole
-up of the whole
-up for observers
eys prepared using different
Specimens for comparison
or marked species in the
field.
K
structures and formats.
To ensure that the testers
are using the keys and not
their own knowledge, the
selected species must be
previously unknown to the
testers.
A mock
guide.
Specimens for
identification or marked
species in the field.
A mock
guide (as close as
possible to the final product).
Photocopies of the
mock
to record comments.
X
What is this species?
Which key format enables
you to reach a correct
identification and to feel
confident of your
identification?
What is this species?
How did you decide that?
What did you like/dislike
about the process of
identification and why?
In this guide, which species
might be suitable for annual
fodder?
This can be presented as a
quiz, including questions
suchas ‘why should species
not be used for medicine
in winter’, which requires
testers to use indexes,
possibly linking local to
scientific names, locate the
species description and
find the correct information.
Use two or more key
formats to identify a range
of species chosen for
testing a range of
diagnostic characters.
Identify a range of
species using the guide,
or correctly conclude that
the species is not in the
guide.
Find information
contained in the guide,
such as a range of
species appropriate to
a chosen purpose.
Anna Lawrence
Usable keys
Usability
Source:
222 Plant Identification
testers rather than the other way round. The basic process is much the same as for the usability test carried out in the field. However, particular attention needs to be paid to the quality of the specimens in this case (see the section on ‘Preparing materials for testing’).
Steps in a usability test
The first step is to consider the purpose for which the guide is intended and to decide on a task that exemplifies this purpose. If the purpose of the guide is to enable people to identify particular trees, then the task for the usability test will be identification. If the purpose is to help users confirm identity in order to then find information about a species, then the task could be to find some specific information about a plant or about its use (for example, ‘use the guide to find out how to prepare chamomile in a tea’, or
‘use the guide to find which species are suitable for animal fodder in the dry season’).
Setting the task involves thinking through the various ways in which people might use the different components of the guide to successfully complete the task. It may be helpful at this stage to think back to your initial consultation and to remind yourself what kind of questions the guide is designed to answer (see Chapter 3).
Having made a decision about the task, the next step is to select the species to be identified using the guide. This should be done in relation to each potential user who will be involved in the usability testing. It is important that the range of species to be identified covers various possibilities. For example, of six species selected to be identified, two might be well known to the testers, two less well known and closely related (so that they have to be examined particularly carefully to be separated), another rare and currently flowering, with another presented in a vegetative state so that other characters have to
be tested.
Once the species have been selected, then the user and the specimen need to be brought together. There are various options for this:
•
Take the user to the plants (a field test): plan a route through the forest/field that takes the respondent to various pre-selected plants. These may be labelled – for example, each with a number – so that the tester has no hints as to the names, but the facilitators can cross-check on a list of correct identifications against each number. This could be tested by numbering ten random plants in a forest and seeing whether the ones that are in the guide are accurately picked out by users, and those that are not in the guide are all noted as being not in the book. Another approach involves asking users to find plants in real life situations using the guide. For instance, a field guide promoting local medicines may aim to allow people to locate the plants (perhaps prior to using them) and allow no confusion with non-medicinal plants (that are not in the book). This involves choosing an area where some of the target species are present, perhaps along a path to make the test easier, and asking users to locate any of, for example, five species from the guide that they find. In this case, plants are numbered.
•
Take the plants to the respondent (a simulated test): bring plant specimens to the place where the interview will be held. This requires care because the identification of plants in the field often depends upon characteristics, such as the bark, latex or habit of the plant, that cannot be observed in small specimens (for example, branches); but it is the next best thing if a field trip is impossible.
Testing the field guide 223
Decisions now need to be made on what has to be observed and noted down during the usability test. It is important to remember that each potential user participating in the test will need to have one observer watching them throughout the task. What you are looking for is confidence that leads to a successful search and the points at which the search breaks down or becomes difficult. You should look for signs of hesitation, annoyance, revisiting the same part of the book more than once, starting over, etc. These must be noted down as they will form the basis for discussion after the task has been completed.
The next decision to be made is how the observation will be recorded. This can be as a set of notes or on pre-prepared forms (see Table 9.2) in which tables can be used that list all of the components of the guide, with columns for entering what people did and for your own comments and queries that you want to take up with users at the end of the test. Preparing the tables, which are essentially a description of the guide and its constituent parts, can take some time. An example is included in Table 9.2, and can be adapted to specific situations.
You may find it much faster and safer (in terms of recording all that the user does and says) to note sequentially what is being done, said and observed by the user.
Nevertheless, if the tables are not completed, you run the risk of losing (or not recording) other information. For example, it is important to know which components of the guide are being used and which are not. If a table is not made for the usability tests, then the interviewer may forget to ask important questions about the components that are not being employed. We therefore strongly recommend that the tables are used during, or immediately after, each test has been completed as a means of recording the data. This will be invaluable when it comes to analysing the results.
You need to pilot the usability test, preferably with a member of the potential users –
but if that is too difficult, certainly among yourselves. Then revise the test in order to overcome any confusion. For example, in Ghana, a theoretical test protocol was devised for comparing various formats with the same species content. However, early tests using two or three invigilators showed that the tests took too long, and so were simplified. In the early tests, we asked participants whether they had enjoyed themselves and how the tests might be improved. Many people found the tests interesting, but said that if the test was shorter they would have got just as much interest out if it and would not have become so tired at the end. Based on this, the tests were shortened and lasted about 1–1.5 hours of actual testing (excluding transport to the site) per participant, rather than twice as long in the original design. Other minor changes to the wording of questions and to the size of the forms were made in this early phase.
ORGANIZING WORKSHOPS FOR TESTING
Planning the methodology will help you to decide:
•
whether the team conducting the test needs training;
•
how to allocate the resources;
•
how to choose suitable target users to invite for testing;
•
the most appropriate dynamics for creating the best possible group interaction for the tests.
224 Plant Identification
Table 9.2 Completed usability form in the case of a specific guide to medicinal plants in Paraíba, Brazil
Note: The ‘Used?’ column indicates the order in which the tester used components.
Source: Ana Paula Ferreira and Anna Lawrence, results of testing usability of Agra (1996)
Testing the field guide 225
All activities must be planned, however simple they might seem. Who is to lead the activity, what material will be needed and, especially, what the aim is and why that particular technique is being used – all of this must be decided beforehand. One technique is suitable to help a group relax and another to wake them up after a long presentation.
When there are activities to do in the field in order to test material, it is better to do these in the morning and leave the afternoon session for theoretical discussions, seminars, debates, presentations, reading, etc. If there are long presentations, it is also advisable to hold them in the morning session as people are more awake and alert then, and keep the afternoon for more dynamic activities, such as group work or debates.
Planning meetings should be held in advance to allow enough time for materials to be produced, for a suitable methodological approach to be designed, and to issue invitations to those individuals who are to take part in the testing workshop as a sample of the user group in order to ensure their presence and participation. It is advisable that the first meeting should be held at least two months before the testing, and a second one between a month and a fortnight before.
It is also necessary to hold a meeting with the entire test team the day before the workshop. It is vital that all those involved know the aim of the test, and the materials to be used and tested. It is also important to check all of the materials, the set-up and the facilities available, and to be able to resolve unforeseen hitches and make methodological changes should team members or members of the target user group, for instance, be missing.
The time dedicated to each activity should be calculated carefully since there may be a variety of different tests to be performed. It is worth remembering that the individuals taking part in the testing are not necessarily used to identifying plants or animals, or to the discipline of recording each step involved in the process, or to being critical of another person’s work. A workshop like this will usually take a whole day.
Remember that the testers have made themselves available voluntarily, whether out of self-interest or not, and that they are giving up their time to take part. The organizers must do everything possible to encourage them to take part, making the logistics as easy as possible for them.
Location, facilities and materials
The venue where the workshop is to take place should be easy to get to, and if this is not the case, transport should be provided for the benefit of the participants and the team. If the workshop is held in an area where the participants do not live, all board and lodging should be covered by the test organizers.
As far as possible, the facilities at the venue should offer a reasonable standard of comfort and be well enough equipped for the activities that are to take place at the workshop. All of the material to be
tested and associated methodological material must be ready for use so that participants are not kept waiting unnecessarily.
Conduct of the tests
We recommend that the team, or at least one team member, arrives at the venue half an hour early to welcome the participants. A punctual start to proceedings is important.
The workshop should begin by welcoming all participants and outlining the aim of the activities that are to take place. All of those present should be introduced to each
226 Plant Identification
BOX 9.1 HOW TO RUN A USABILITY TEST
•
Aim: to test the efficiency and accuracy of the guide in helping users to identify species or find out information about the species contained in the guide. Essentially, there are two tasks: find the species in the guide and confirm that this identification is correct using the information contained in the description of the species, based on the important features of the species.
•
Location: an open area containing all or some of the species included in the guide, with some in flower or fruiting, or a communal area to which specimens have been brought.
•
Material for the participants: mock-up of complete guide; numbered plants or specimens to be identified, or list of questions to be answered by using the guide.
•
Material for the team: photocopied mock-up of complete guide; list of answers to each question or species to be identified; evaluation forms for the observer (see
‘Documentation: Instructions and forms for the test’).
•
Logistical guidelines for the activities: it is helpful to have two facilitators for each group of testers. If you have more than five testers, consider dividing them into more groups, each with two facilitators. One will present the activity and provide guidance, and the other will observe and record responses, analyses, comments, other observations, problems, etc.
Each tester group should have a complete printed guide and should identify two or three designated species. Each group should be allotted different species, which will facilitate testing of many of the species in the guide. Encourage different group members to identify each species in turn, with assistance from the rest of the group.
•
Method:
1
Start by making people feel comfortable and by explaining what you are going to do and why. They should not feel that they are being tested, although this is hard to avoid at some level. Say something quite simple covering:
–
Plant Identification Page 37