On the Benefits and Limitations of Using Metrics in Global Health

Review by Lauren Wallace
October 11, 2016

Metrics: What Counts in Global Health

by Vincanne Adams
Published 2016

Duke University Press

buy on Amazon

There has long been a tension between sociocultural anthropology and more quantitative fields such as demography and medicine (Christman and Maretzki, 1982; Schepher-Hughes 1990; Kertzer and Fricke 1997). While some of this history results from differences in methodological and theoretical orientations, it also reflects fundamental contrasts in disciplinary worldviews. The basis for the differences between quantitative and qualitative forms of evidence and methods, and the possibilities for a productive union between them, have long been discussed by anthropologists. Metrics: What Counts in Global Health, a compilation of works edited by Vincanne Adams, builds on this concern by undertaking a detailed examination of the production and uses of metrics in global health practice.

One of the goals of this volume is to enable readers to think productively about the benefits and limitations of metrics. Adams defines metrics as “technologies of counting, but specifically technologies of counting that inform global knowledge” and are “imagined to offer uniform and standardized conversations about how best to intervene, how best to conceptualize health and disease, how best to both count and be accountable, and how best to pay for it all” (p.6). Simply put, global health metrics are standard, quantitative ways of measuring wellbeing and the impact of health interventions.

Adams and the other authors in the volume extend previous anthropological examinations of the problems of quantitative forms of evidence, their contribution to omission and erasure, and their effects “beyond the numbers” by exploring how metrics are a form of politics in their own right (Schepher-Hughes 1997; Bledsoe 2002). They argue that the impacts of metrics, and the ways that they are produced and circulated can be understood in terms of a new kind of “global sovereign”. In the first chapter of the volume, Adams defines global sovereignty as “a flexible assemblage of data production that like Mbembe’s fetish, orchestrates biopolitical health interventions so that they work within capitalism’s terms and limits…” (p.45). Adams and colleagues argue that this type of sovereignty is a political imperative tied to lingering forms of neoliberal financial arrangements, as well as types of development aid that both reaffirm and transcend state boundaries.

The introductory chapter traces the history of metrics from the colonial period through to the post-colonial focus on the need for development aid. Adams argues that the interdependence of health and economics, as well as the search for a universal metric, are constants between these time periods. She provides a detailed account of the development of some of the key metrics in postwar international health, including the QUALY (Quality Adjusted Life Year) and the DALY (Disability Adjusted Life Year), which are both ways to quantify the relationship between health and economic value. Next, she describes the history and uses of the new gold standard in global health metrics – the randomized control trial (RCT). She concludes with a brief exploration of alternative forms of accounting in global health, such as stories. She suggests that rather than opening up a space for rethinking evidence, these narratives often work to reaffirm the legitimacy of quantitative indicators.

This introductory piece artfully provides the reader with an understanding of the role that history has played in the rise of metrics, while drawing an outline of some of the broad themes in the forthcoming chapters. These themes include the difficulty of producing good data, and the creation of “silos of exclusion” (p.36), in which data and methods that do not meet the gold standard are rendered invisible. In addition, Adams explores the “fiscalization” of health, and life, (p.29) both in terms of the ways that the production of metrics is expensive, and in the ways they are linked to profit generation via public-private partnerships.

The volume is divided into four sections: Getting Good Numbers, Metrics Politics, Metrics Economics, and Storied Metrics. The first two chapters of the book, both case studies of maternal health in sub-Saharan Africa, focus on the ways that numbers can place political outcomes over health. Wendland’s discussion of “wobbly” estimates of maternal mortality in Malawi provides counterevidence for the idea that statistics are accurate and neutral. In reviewing how maternal mortality metrics are made into evidence, she shows that although the products of the equations used are assumed to be facts, they hide uncertainty. Wendland demonstrates how maternal mortality statistics are important politically, and are treated as fetishes even though they omit critical information and downplay the root causes of problems. Oni-Orison’s account of the Healthy Mothers Healthy Babies Program in Nigeria deepens the linkage between politics, sovereignty and metrics further by arguing that these numbers are not just used to advance politics and goals “but they also carry the political clout to determine who will get reelected to office, who will be promoted to chief medical director of a hospital, and who will win a government contract” (p.85). Where numbers are conduits for securing local sovereignty through development aid, health programs become woven into politics and governance. Here, Oni-Orison illustrates how the politics of record keeping can get in the way of good medical practice, with real consequences for the treatment of patients.

Part II, Metrics Politics extends the analysis of the interlinkages between metrics, governance and sovereignty by exploring how global health measurement becomes the foundation for regimes of governance. In the first chapter, Marlee Tichenor describes the fascinating case of the health workers’ data retention strike in Senegal. Tichenor’s analysis illustrates the way that metrics have outcomes for citizenship, since a lack of data can undermine the relationship between nations and their status as participants in systems of global health governance. While Tichenor focuses on the relationship between data and global citizenship, in her chapter, “Native Sovereignty by the Numbers,” Molly Hales describes the part that health metrics play in indigenous sovereignty. She shows that the demands for quantitative data placed on the Healing Our Families program in Alaska has undermined local forms of treatment and knowledge.

In Part III, Metrics Economics, the authors further explore the so-called “fiscalization” of health and life that Adams introduces in Chapter 1. Drawing on her work in Washington and Sierre Leone, Susan Erikson’s discussion of market logics provides an overview of the troubling links between moneymaking and global health problems. Erikson argues that more reflexivity about new fiscal arrangements in global health is required, since we do not yet fully understand their impacts. In the second and final chapter in Part III, Lily Walkover provides an overview of the ways donor money shapes the metrics of accountability at Hesperian Health Guides. She argues that the involvement of donor support shapes the type of work that non-profits do and in turn makes them complicit in the very systems they criticize.

Picking up threads from Hales and Walkover’s discussions, the final chapters in Storied Metrics examine more directly the resistance anthropologists and other qualitatively-trained individuals working in global health run up against in working with metrics systems. First, Carolyn Smith- Morris examines how ethnographers work on RCT research with United States veterans. Building on the theme of omission and erasure introduced in previous chapters, Smith-Morris highlights how clinical accounts of “fidelity” fail to capture the contextual data that are important in veterans’ lives. She suggests that social scientists who work alongside RCTs can ensure that more of this contextual data is reported. However, she argues that the inclusion of qualitative data is insufficient where decision-making power rests with clinicians. Next, in his account of Konbit Sante, a medical humanitarian NGO in northern Haiti, Pierre Minn describes the strategies used by staff to communicate the relevance of their program to funders. Minn illustrates the way that Konbit Sante’s strategies are a form of resistance against a limited set of indicators being used to evaluate global health interventions.

Finally, in a very brief concluding chapter, Adams explores future directions for social scientists’ meaningful engagements with global health metrics. She argues that discussing metrics in ways that undo their claims is essential. However, she suggests that subscribing to a critically applied medical anthropology, which also describes new models worth pursuing, is equally important. Adams contends that many anthropologists and other social scientists working in global health have taken up the challenge of including ethnographic evidence in statistical and RCT metrics. She provides a few succinct examples of this type of work. For instance, she describes how sufferers of “orphan diseases” have begun to advocate for the inclusion of small sample sizes in the production of metrics. In closing, she suggests that while many ways of rethinking the paradigm of global health research have been proposed, these alternative approaches are still underutilized today.

The epilogue is successful in assuring readers about the multiple forms of resistance leveled at metrics by anthropologists and others. However, a few more in-depth descriptions of social scientists’ optimistic engagements with statistical and RCT metrics would be useful, particularly for applied anthropologists and other global health practitioners. For example, Adams briefly mentions the groundbreaking model used by Partners in Health, however, she could elaborate more on the ways the model is particularly useful in thinking about our engagement with metrics.

Overall, this volume is insightful, engaging and impressive. Adams and colleagues succeed in exploring in detail the limitations of global health metrics and their impacts on what Adams terms “global health efficacy” (p.6). One of the major contributions of this volume is the way it shows how the production, circulation, and impacts of metrics can be understood in terms of the “global sovereign” (p.45). It is clear from the work of Adams and her contributors that as metrics become the organizing principles for global health, they create new kinds of identities and social relationships that have real effects for health workers, nation-states, and local, regional and national governments. I highly recommend this enlightening and ethnographically rich book. It is a must read for both medical anthropologists and global health practitioners, and would make an excellent addition to the reading list for graduate classes in medical anthropology or global health.

References Cited

Bledsoe, Caroline. Contingent Lives: Fertility, Time and Aging in West Africa. Chicago: University of Chicago Press, 2002.

Chrisman, N., and T.W. Maretzki, eds. Clinically Applied Anthropology: Anthropologists in Health Science Settings. London: D. Reidel: 1982.

Hahn, Robert and Marcia Inhorn, eds. Anthropology and Public Health: Bridging Differences in Culture and Society, Second Edition. New York: Oxford University Press, 2009.

Kertzer, David I., and Thomas E. Fricke, eds. Anthropological Demography: Towards a New Synthesis. Chicago: University of Chicago Press, 1997.

Schepher-Hughes, Nancy. “Three Propositions for a Critically Applied Medical Anthropology.” Social Science and Medicine 30, no. 2(1990): 189-97.

Schepher-Hughes. “Demography Without Numbers.” In Anthropological Demography: Toward a New Synthesis, edited by David I. Kertzer and Tom E. Fricke, 201-222. Chicago: University of Chicago Press, 1997.

Lauren Wallace is a PhD Candidate and Vanier Scholar at McMaster University. She is a medical anthropologist who studies changes in family planning and family formation in northern Ghana.

© 2016 Lauren Wallace
Creative Commons License

Download as PDF