On the basis of these experiences, Eurostat is now considering adding a question on life satisfaction in the core part of the EU-SILC questionnaire on a yearly basis in the near future, while every 6 years an ad hoc module with around 20 variables on the topic will supplement this information. This will provide a remarkable resource to the research and policy community: a harmonized cross-country data set with a sufficiently large sample size to estimate the relationship of subjective well-being to a host of individual and geographic characteristics over time. Eurostat’s plan for annual measurement is an important step that will help to establish a time series for more than 30 countries.
There have also been advances in including subjective well-being in time-use surveys. This is particularly important to improve our understanding of experiential well-being, since it enables the examination of the link between people’s activities, daily circumstances, social contact, and feelings. Some countries (the United States, France, Poland, Luxembourg, the United Kingdom, and Canada) have incorporated some version of experiential well-being into their time-use surveys. For example, in the United States a governmental agency, the Bureau of Labor Statistics, regularly conducts a time-use survey of over 12,000 individuals (the American Time Use Survey, ATUS). In 2010, 2012, and 2013, a well-being module was included in ATUS that sampled three time-use episodes for each person and asked a number of experiential well-being questions. Unfortunately, the module was limited to three rounds even though a strong recommendation to continue collection of these experiential well-being data was issued by a National Academy of Sciences panel (Stone and Mackie, 2015).
Table 7.1. Data Collections on Subjective Well-Being Undertaken by National Statistical Offices in OECD Countries
Source: Exton, C., V. Siegerink, and C. Smith (2018), “Measuring subjective well-being in national statistics: Taking stock of recent OECD activities,” forthcoming, OECD Publishing, Paris. StatLink 2 http://dx.doi.org/10.1787/888933839753.
The investment of NSOs in measuring subjective well-being is extremely important and should continue (see sidebar, “The Experience of the UK Office for National Statistics in Measuring Subjective Well-Being”). As with other indicators such as GDP, subjective well-being data is at its most valuable when one can observe and compare trends, which requires tracking data over long periods. Continued methodological progress would be facilitated by the collection and dissemination of long time series in large, high-quality data sets. Collection of such data will also facilitate the generation of policy-relevant insights. Researchers can help support this process by ensuring that analyses of the data that do exist are carried out and disseminated, demonstrating their usefulness; conversely, NSOs can help by ensuring that micro-data are available in a timely manner. Nevertheless, the methodological and conceptual issues raised later in this document must be taken into account when interpreting the data collected. To take one example, Deaton (2012) and Deaton and Stone (2016) suggest that tracking of subjective well-being data can be disrupted by a change in the questionnaire design or by the provision of inappropriate cues, and that such factors do not always average out.
THE EXPERIENCE OF THE UK OFFICE FOR NATIONAL STATISTICS IN MEASURING SUBJECTIVE WELL-BEING
In November 2010, supported by the then Prime Minister David Cameron, the UK Office for National Statistics (ONS) launched the Measuring National Wellbeing (MNW) program with the aim of establishing “an accepted and trusted set of National Statistics which help people to understand and monitor national well-being.” Alongside a six-month national debate that asked people “what matters” to them, ONS began its program by developing and adding four questions on subjective well-being (called “personal” well-being, in the ONS initiative) to one of its largest household surveys (the Annual Population Survey). Some reflections on ONS experiences and lessons learned are the following:
• A robust set of personal well-being questions was developed. ONS undertook extensive work to develop a robust and credible set of four questions to capture personal well-being and introduced them into the UK’s largest household survey. Challenges faced in the process included investigations into interview mode effects, different response scales, question placement, and cognitive interviewing. These questions were then added to the Annual Population Survey, whose large sample size provided the opportunity to analyze personal well-being alongside numerous other variables while also minimizing survey cost. The findings from ONS testing and development were used as best practice in informing the OECD Guidelines on Measuring Subjective Well-Being.
• Experience from asking these questions has been positive. The four ONS personal well-being questions take only 75 seconds to ask and complete. Since survey space is at a premium, they are an efficient use of both time and space. Feedback from interviewers has been positive, with many reporting that the questions provide an opportunity to build a rapport with respondents; by providing a focus on how people feel about their lives, the questions have also helped avoid refusals.
• Demand for personal well-being data continues to grow. Since their introduction into the Annual Population Survey in April 2011, ONS subjective well-being questions have been used in over 20 government surveys covering areas such as crime, household wealth, and visits to the natural environment. Researchers have used these data to improve understanding of the relationship between personal well-being and a range of other outcomes.
• Personal well-being does not tell the whole story. The national debate managed by ONS reinforced the wealth of factors that people consider as important to their well-being, and added legitimacy to the Measuring National Wellbeing program. ONS developed a suite of 41 measures of national well-being, including both subjective and objective measures across a range of domains, including for example “Our Relationships,” “Health,” “the Economy” and “the Natural Environment.” ONS also recognized that presentation would be critical to acceptance of the measures and understanding of the wider program, and developed the National Wellbeing Wheel to respond to this challenge; the Wheel was recently replaced by a new interactive dashboard, accessible by the increasing number of users relying on mobile devices, which provides “live” updates as new estimates become available for each indicator.1 While most media attention still falls on measures of personal well-being, a growing number of policy studies have used the domains of the National Wellbeing Wheel as a framework to structure approaches to policy evaluation and improvement.
• The What Works Centre for Wellbeing was established in late 2014. Since its introduction, the Centre, which is dedicated to bridging the gap between evidence and policy, has helped ensure that high-quality evidence is available to support policy-making, giving a focus to attempts to improve well-being across the United Kingdom. ONS seconded its head of personal well-being for a period of two years to help establish the Centre and cement links between evidence and policy.
• Policy use of subjective well-being is increasing. Estimates of personal well-being, within the framework of wider measures of national well-being, helped policy-makers understand how their decisions affect people’s life. Examples of policy use of personal well-being data include: the Public Health Outcomes Framework, which monitors the four measures of personal well-being as part of its vision to improve and protect the nation’s health and well-being; the presentation of personal well-being results by occupation, to support young people in making a more informed choice about their career; and the employment of a well-being valuation approach in attempts to monetize the human cost of crime.
• There is no appetite for a single index of national well-being. ONS is frequently asked to consider a single measure to summarize progress and place well-being on the same footing as GDP. While the advantages of a single indicator (particularly in aiding communication) are recognized, ONS has no intention of producing a single index of well-being: too many conceptual and methodological hurdles are, as yet, unresolved to allow progress in that direction.
1. www.ons.g
ov.uk/visualisations/dvc364/dashboard/index.html.
Source: Text provided by Jil Matheson.
Global Reports and Tools
Several global analyses of social and economic progress beyond (or in addition to) GDP have been published and widely disseminated since 2009, including the Legatum Institute’s Prosperity Index (O’Donnell et al., 2014); the World Happiness Report,2 released annually since 2012 by the UN Sustainable Development Solutions Network (Helliwell, Layard, and Sachs, 2018); the US National Academy of Science Report on Measuring Subjective Well-Being (Stone and Mackie, 2015); and the OECD’s How’s Life? series and Better Life Index (OECD, 2015a). While all these projects include sections on subjective well-being, each takes a different approach to the analysis and comparison of well-being across countries. This diversity of approaches and initiatives helps to advance our understanding of subjective well-being and how it can be used.
Two particularly important documents—the OECD Guidelines on Measuring Subjective Well-Being (OECD, 2013) and the US National Academy of Science Report on Measuring Subjective Well-Being—lay out the current experience in collecting data on subjective well-being and provide a focal point for a growing consensus around methodology. We view these documents as “required reading” for policy-makers and researchers working with subjective well-being measures because they carefully consider many of the various conceptual and methodological issues that are only briefly touched upon below.
Furthermore, a US National Institute on Aging–supported conference on time-use and experiential well-being was held in 2015 to take stock of progress since the publication of the day reconstruction method (DRM) and to identify remaining challenges. A report of the conference proceedings (Stone and Smith, 2015) outlines many issues and questions that remain about the DRM, despite its use in dozens of research studies. Two other documents—the Legatum Institute report and the World Happiness Report—speak more to the policy uses of subjective well-being measures and are discussed in a later section of this chapter.
Improvements in Methodology
The Stiglitz, Sen, and Fitoussi (2009) report identified some of the methodological and interpretive issues that caused concern about using subjective well-being measures. Since the publication of the report, solutions have been presented and explored for many of those issues. A short summary of the issues is provided in Table 7.2 alongside the most promising solutions. While a deep examination of these issues is important to improving the measurement of subjective well-being, it is equally important to avoid setting a uniquely high standard for subjective well-being in contrast to other indicators, such as income, consumption, or wealth inequality, which, as shown in the other chapters in this volume, can be quite difficult to calculate or are similarly derived from self-reported measures that are equally sensitive to the survey vehicle used (for example, the length of the recall period used for expenditure diaries can have dramatic effects on consumption estimates, Beegle et al., 2011) or may have other issues related to self-reports more generally.
Some of the methodological issues detailed in Table 7.2 can be partially addressed by careful standardization of questionnaires, which may reduce framing and potential context effects.3 For this reason, the continued collection of standardized questions across countries is needed. Following the OECD Guidelines is a good way to ensure that questions are standardized, as they represent the state of the art for question formulation and survey administration. Eurostat’s 2013 EU-SILC ad hoc module on subjective well-being followed the OECD Guidelines, and was based on its recommended questionnaire. The EU-SILC and its ad hoc modules have a legal basis, with a common list of variables, concepts, classifications, and survey requirements translated in all EU languages. The legislation is accompanied by EU-SILC methodological guidelines, including the recommended questionnaire, translated in all languages. Importantly, the legislation requires that all EU countries contribute data to this effort.
To illustrate the importance of the concerns over systematically different response styles and bias, we provide a more detailed discussion here. If different groups of people show systematic differences in how they interpret subjective well-being questions or use response scales (for example, due to some common characteristic such as language or culture), then simply comparing the level of subjective well-being between these groups can yield misleading conclusions. The extent to which this is a problem will depend, in part, on the question the data are being used to answer. In some cases, this will not matter if the main focus of interest is whether the change in a variable produced a change in subjective well-being within a specific population, rather than direct comparisons between groups of individuals.
However, in simple descriptive analyses where levels of subjective well-being are compared across groups, such as gender or occupations, or across countries, then systematic differences in question interpretation or response styles between groups has the potential to cause bias. For example, if the elderly understand or respond to a subjective well-being question in a way that is systematically different to younger people, or if richer people have a response style that is somehow different to that found among the poor, then we might over- or under-estimate differences in subjective well-being between these groups. In order to have more concrete ideas about the extent to which this may be a problem, we should have a better idea of why such differences may exist in the first place, and have some theoretical justification for a concern with systematic differences in how subjective well-being questions are interpreted and answered.
Concerns about systematic biases, in particular their potential interaction with context effects, are not solved through the use of panel or longitudinal data if the goal is to compare levels of well-being between groups at a single point in time. A salient example of this is provided by Deaton (2012) on context effects, with its implications further refined in Deaton and Stone (2016) using subjective well-being data collected by the Gallup Organization. In this work, Deaton found evidence of a large impact of a set of political questions placed prior to an evaluative well-being question (the Cantril ladder), an effect that was larger than that of the 2008 recession. This effect was driven by respondents who reported feeling that the country was going in the wrong direction, which exerted a strong downward bias on their answers to the subsequent life evaluation question. Importantly, these context effects varied by race or ethnicity. For example, the negative treatment effect of the political polling questions (compared with a control without such questions) was smaller for blacks than for whites. This meant that, while whites in the control group on average reported life evaluations almost 0.2 scale points higher than blacks (on a 0–10 scale), in the treatment group there was almost no difference between the two groups (less than 0.03 scale points). This contrasts with results for gender, age, and income, where the size of the context effects remained stable across different population groups.
The finding that context effects can work differently for different populations complicates the interpretation of group differences in subjective well-being and requires more extensive study. Since NSOs are very unlikely to ask political polling questions, it will be important to understand whether other lead-in questions can also produce a significant shift in responses. For example, Lee et al. (2016) found that asking a self-rated health question immediately before a subjective well-being question prompted a stronger correlation between the answers compared with a situation when the question ordering was reversed. This effect was driven by a subsample of respondents who reported one or more chronic health conditions: among those without chronic health conditions, question ordering did not produce a significant difference in the size of the correlation.
Taken together, these studies reinforce the importance of question ordering for both survey design and data comparability. Unless explicitly tested through split-sample methods, the effects of question ordering will tend to remain hidden from view.
While systematic group differences in response styles are in
sufficiently addressed in the literature, several advances have been made. These include the use of vignettes (Crane et al., 2016; although see Grol-Prokopcsyk et al., 2015, and OECD, 2013, for concerns about these methods) when analyzing data from migrants (Senik, 2014; Exton, Smith, and Vandendriessche, 2015), and using individual fixed effects models with panel data. However, in general we view these concerns as unresolved and recommend continuing research.
Another continuing area of methodological development pertinent for experiential subjective well-being research is the use of real-time and near-real-time data capture—for example, with momentary data recording such as ecological momentary assessment (EMA), which is based on the administration of brief questionnaires in real time in people’s everyday lives (Stone and Shiffman, 1994; Shiffman, Stone, and Hufford, 2008), daily diaries, and day reconstruction methods (Kahneman et al., 2004). These are important techniques because of their potential to assess experiential subjective well-being with less retrospective bias than measures using (relatively) long recall periods that ask about fluctuating levels of emotions and pain. Long recall periods conflate actual memories of experiences with broad beliefs that do not necessarily accurately reflect experience. However, from a pragmatic perspective of data collection in national surveys, momentary methods can be unwieldy, burdensome, expensive, and impractical in some cases (e.g., while people are driving, or when they are engaged in activities that cannot be interrupted), yielding selection effects. As result, methods that ask about the prior day have become the standard; these include simple overall questions about the day before (as used in the UK ONS survey and the Gallup World Poll); so-called hybrid measures based on the DRM—which capture some details about the day, for example, the number of hours engaged in various activities (Christodoulou, Schneider, and Stone, 2014; Miret et al., 2012)—and DRM surveys (addressing the entire day or sections of the day, as done by the Survey on Health, Ageing and Retirement in Europe [SHARE]).
For Good Measure Page 28