What Works in Development?: Thinking Big and Thinking Small - Softcover

Softcover

ISBN 10: 0815702825 ISBN 13: 9780815702825

Publisher: Brookings Institution Press, 2009

Synopsis

What Works in Development? brings together leading experts to address one of the most basic yet vexing issues in development: what do we really know about what works— and what doesn't—in fighting global poverty?
The contributors, including many of the world's most respected economic development analysts, focus on the ongoing debate over which paths to development truly maximize results. Should we emphasize a big-picture approach—focusing on the role of institutions, macroeconomic policies, growth strategies, and other country-level factors? Or is a more grassroots approach the way to go, with the focus on particular microeconomic interventions such as conditional cash transfers, bed nets, and other microlevel improvements in service delivery on the ground? The book attempts to find a consensus on which approach is likely to be more effective.
Contributors include Nana Ashraf (Harvard Business School), Abhijit Banerjee (MIT), Nancy Birdsall (Center for Global Development), Anne Case (Princeton University), Jessica Cohen (Brookings),William Easterly (NYU and Brookings),Alaka Halla (Innovations for Poverty Action), Ricardo Hausman (Harvard University), Simon Johnson (MIT), Peter Klenow (Stanford University), Michael Kremer (Harvard), Ross Levine (Brown University), Sendhil Mullainathan (Harvard), Ben Olken (MIT), Lant Pritchett (Harvard), Martin Ravallion (World Bank), Dani Rodrik (Harvard), Paul Romer (Stanford University), and DavidWeil (Brown).

About this title

About the Author

Jessica Cohen is a development economic research fellow with the Global Economy and Development program at the Brookings Institution. William Easterly, professor of economics at New York University and a nonresident senior fellow at the Brookings Institution, is the author of The White Man's Burden:Why theWest's Efforts to Aid the Rest Have Done So Much Ill and So Little Good (Penguin, 2006).

Excerpt. © Reprinted by permission. All rights reserved.

What Works in Development?

Thinking Big and Thinking Small

Brookings Institution Press

Copyright © 2009 Brookings Institution Press
All right reserved.
ISBN: 978-0-8157-0282-5

Chapter One

Introduction: Thinking Big versus Thinking Small

JESSICA COHEN AND WILLIAM EASTERLY

The starting point for the contributions to this volume, and the conference for which they were prepared, is that there is no consensus on "what works" for growth and development. The ultimate goal of development research-a plausible demonstration of what has worked in the past and what might work in the future-remains elusive. As Martin Ravallion points out in his comment on chapter 2, we are beyond "policy rules" such as the Washington Consensus, and "thinking big" on development and growth is in crisis. The "big" triggers for economic growth have not been shown to work, either because they in fact did not work or because it was impossible to demonstrate their impact persuasively.

As a result, many in development have turned to "thinking small." For the most part-but not exclusively-the focus has shifted from macro- to micropolicy questions. This type of research commonly seeks the most effective method for delivering public goods such as education and vaccines. A growing methodology for analyzing micropolicy questions is randomized controlled trials (also known as Randomized Evaluations, REs). Much of this volume is about the merits and drawbacks of REs in elucidating what works in development. The specific arguments-we say more on them later in this chapter-revolve around several nagging questions. What kind of development policy research yields "hard" evidence? Are some types of evidence "harder" than others? Is there a trade-off between the scope of the questions researchers ask and the quality of the evidence they generate? What questions and what quality of evidence matter most for development policy and aid effectiveness? In exploring these issues, it is essential to first ask how the crisis in thinking big transpired and whether thinking small is indeed a solution.

The Collapse of "Thinking Big"

The failure of thinking big-elsewhere described as "the panaceas that failed"-has been widely acknowledged. Many would agree with Arnold Harberger that "there aren't too many policies that we can say with certainty ... affect growth." Some, like those behind the Barcelona Development Agenda, would go even further: "There is no single set of policies that can be guaranteed to ignite sustained growth." Even the universally revered dean of growth theory, Robert Solow, believes that "in real life it is very hard to move the permanent growth rate; and when it happens ... the source can be a bit mysterious even after the fact."

Where did this pessimism come from? As both Abhijit Banerjee, and William Easterly in his comment on Banerjeee, discuss later in the volume, several contributing factors readily come to mind: despite concerted attempts, macroeconomists were unable to deliver higher growth; the credibility of the growth regression literature waned; extremely volatile growth rates could not be explained; and growth analysis neglected to do enough long-run regressions.

The Failure of Big Pushes to Raise Growth

Three unsuccessful pushes are particularly notable:

1. The early big push in foreign aid (especially in the most aid-intensive continent, Africa).

2. Structural adjustment (also known as the Washington Consensus) in the 1980s and 1990s.

3. "Shock therapy" in the former Communist countries.

All of these episodes are far from natural experiments, of course, with adverse selection posing a severe problem for the interpretation of policy impact. However, all three had such poor outcomes that the counterfactual-that growth would have been even worse without the macroeconomic intervention-was hardly plausible.

The Failure of the Growth Regression Literature

The pessimism surrounding big pushes intensified as the credibility of the cross-country growth literature declined, with its endless claims for some new "key to growth" (regularly found to be "significant") and probably well-deserved reputation for rampant data mining. As the Easterly comment on Banerjee notes, the number of variables claimed to be significant right-hand-side (RHS) determinants approached 145, which is probably an undercount. Having a long list of possible controls to play with, researchers found it easy enough to arrive at significant results, and using the abundant heuristic biases that make it possible to see patterns in randomness, convinced themselves that the significant results were from the "right" specification, and that the others (usually unreported) were from the "wrong" ones.

The growth literature was also criticized for its inability to address causality. In the absence of clear evidence that growth outcomes can be attributed to specific levers, development research has severely limited utility for policy. This deficiency was probably due to the infeasibility of instrumenting for multiple RHS variables. Any such attempts usually relied on the Arellano-Bond or Arellano-Bover dynamic panel techniques, which (essentially using lagged RHS variables as instruments) became a kind of magical machine churning out causal econometric results. Unfortunately, the identifying assumptions were so implausible as to leave most outside observers unconvinced. This left the causality question unresolved.

Not to overstate the inevitability of the collapse of growth knowledge, neither data mining nor causality was a completely hopeless cause in aggregate regressions. Data mining can be held in check with a well-known methodology: estimating slight variants of the original specification that are as plausible as the original; or, better yet, adding new data that were unavailable at the time of the original estimation to the exact specification. As far as causality was concerned, occasionally there would be a reasonably plausible instrument for a RHS variable of particular interest.

Both remedies can be explored in the hotly debated literature on the effect of aid on growth. Since it is hard for researchers to hold themselves aloof from the strong vested interests in and political biases for or against aid, there was enormous scope for data mining in aid and growth regressions. One very simple test of data mining is to add new data that were not available at the time of the original regression specification and see if the results still hold. The famous result of Craig Burnside and David Dollar that "aid raises growth in a good policy environment" did not pass this test.

As for causality, one promising instrument for aid was population size, because of the quirk that the aid donor bureaucracy does not fully increase aid dollars one for one with recipient population size. Log of population is thus an excellent predictor of aid/gross domestic product (GDP), thankfully unrelated to the economic motivations for aid, and has been used in many studies. Another original strategy for identifying the impact of aid on growth has been to instrument aid from the Organization of Petroleum Exporting Countries (OPEC) to their poor Muslim allies with the interaction between oil price and a Muslim dummy variable. This approach uncovered a short-term effect of aid on output, but also a zero effect on medium-term growth. The generalizability problems of such identification strategies are much like those of REs. Does small-population-induced aid have the same effect as other aid? Does intra-Muslim aid have the same effect as aid from the United Kingdom to Africa? Another problem, which REs typically do not have, is serious doubt about the excludability of the instrument in cross-country growth regressions.

But even when establishing causality was not possible, it would have been equally extreme to say that strong partial correlations would or should have no effect on priors about causal policy effects. Researchers were also probably influenced by the established body of theory, which tended to predict causal effects of policies on growth without much reason to think that growth would feed back into policies.

The Volatility of Growth Rates

By and large, the growth literature has also failed to establish even robust partial correlations between growth and country characteristics. One reason is that growth rates are extremely volatile while country characteristics are persistent. A crude way of showing growth volatility is to do a random effects regression on a panel of annual per capita growth rates between 1960 and 2005. This reveals that only 8 percent of the cross-time, cross-country variation in growth is due to permanent country effects; the other 92 percent is transitory. The annual standard error of the pure time-varying component of growth is an amazing 5.06 percentage points. In the latest successive decades, 1985-95 and 1995-2005, there is virtually zero correlation between a country's performance in one decade and its performance in the next (hence virtually 100 percent mean reversion). This is very bad news when almost all of the plausible determinants of growth are relatively permanent country factors. It was like trying to explain differences in this week's batting averages of baseball players by differences in long-run fundamentals such as training regimens (or steroids!). The noise-to-signal ratio is so high with both weekly batting averages and decade growth rates that any such attempt is largely futile.

Long-Run versus Short-Run Development and Growth Literature

When asked about the impact of the French Revolution, Zhou En-Lai reportedly said it was "too soon to tell." So with growth performance. A very long-run average of growth rates is needed to lower the noise-to-signal ratio enough to have something interesting to relate. To put it another way, growth analysis has suffered from what Daniel Kahneman and Amos Tversky sarcastically call the Law of Small Numbers problem: reading too much into growth differences over one or two decades when they were mostly transitory. The obvious answer was to go to more long-run analysis, which is in fact the direction the macro literature took, doing regressions for log levels of per capita income as a function of long-run characteristics such as institutions.

The lack of persistence of growth rates made data mining easy-and easy to catch. As new growth observations came in, almost uncorrelated with past observations, the data miners would keep finding new variables that "explained" growth. As yet more new data came along, country factors would be the same, but the growth rates would get scrambled again, and the results would vanish. In an amusing nonacademic example, countries in which people ate fast were found to grow more rapidly over 2001-08 than countries in which people ate slowly. Fast-Food USA has been growing faster than slow-eating Japan. The culture on eating presumably is persistent, so this result would not have held in the old days of rapid Japanese and slow U.S. growth, and any slow-eating French boom would make the result disappear again.

Those who reject such chicanery currently hold the field in empirical growth research. This position itself may not be sustainable, however. For the consumers of academic research, just saying "it's too soon to tell" about a matter as visible as economic growth differences is almost impossible to accept. So the nearly universal debunking of growth knowledge has not stopped attempts to explain growth.

The latest attempts appear to embrace a theory on the order of

Growth (country i, period t) = Coefficient (country i, period t) * Policy (country i, period t).

This equation finally fits the data very well! Alas, tautological and nonfalsifiable theories are not usually allowed. Lest we appear to exaggerate, consider a statement by the World Bank Growth Commission in the aftermath of $4 million worth of conferences and consultants: "It is hard to know how the economy will respond to a policy, and the right answer in the present moment may not apply in the future." In chapter 2 of this volume, Dani Rodrik also seems to flirt with this extreme at times, although he avoids nonfalsifiable tautology by laying claim to independent knowledge with growth diagnostics exercises that would give insights into the coefficient (i,t)'s.

Ricardo Hausmann in chapter 6 notes the complexity of public policy and institutions and suggests that the search for which policy is "the answer" only makes sense if there was a "central planner" implementing policies, which is no more feasible for public policy than for private goods markets. He argues for decentralized public policymaking to address such complexity.

The unpopular but well-justified focus on the long term can also generate insights into development from long-run stylized facts. A long tradition in development is to establish robust stylized facts in levels that guide development thinking-examples are the positive correlation between democracy and per capita income (which has stimulated thinking about causal channels in both directions), the negative correlation between per capita income and fertility (often interpreted causally as "development is the best contraceptive"), the relationship between life expectancy and income, and how that relationship (the "Preston curve") has shifted over time (suggesting that major changes in health technology can improve health without income growth), and so on. The neoclassical production function model of development has come into question because of its violation of many stylized facts about development (such as capital flows, brain drain, and the failure of absolute convergence). New growth models have also been guided by macro stylized facts. For example, idea models that stress R&D efforts as a determinant of growth have run afoul of the stationarity of growth rates and the nonstationarity of R&D efforts, although they would fit the very long run of world development as a whole. Another stylized fact in many analyses is the surprising significance of very long-run history for determining today's outcomes, which may lend support to some growth theory models with increasing returns and sensitivity to initial conditions. As David Weil notes, commenting on chapter 4, scientific experiments are not the only means of learning about development; historians do not do experiments, but most economists think that they learn something from historians (including economic historians such as Stanley Engerman and Kenneth Sokoloff analyzing why North America is richer than South).

So thinking big is not dead. However, sixty-year-old hopes that thinking big would translate into clear guidance on how to move immediately into rapid growth and development have been repeatedly disappointed.

The New Promise of "Thinking Small"

The macro literature is not alone in lacking decisive evidence. REs, "natural experiments," and other methodologies prioritizing transparency and clean identification became popular because of the great vacuum of microevidence on development projects. As Lant Pritchett has eloquently put it, nearly all World Bank discussion of policies or project design had the character of "ignorant armies clashing by night." Despite the heated debate among advocates of various activities, they rarely presented any firm evidence or considered the likely impact of those actions. As far as we know, there was never any definitive evidence that would inform decisions of funding one instrument versus another (such as vaccinations versus public education about hygiene to improve health, or textbook reform versus teacher training to improve educational quality).

Even a World Bank handbook was quick to note that "despite the billions of dollars spent on development assistance each year, there is still very little known about the actual impact of projects on the poor." The RE literature made a clear case for basing aid policy on evidence rather than prejudice and special interests. This methodology holds tremendous promise for improving aid effectiveness (and cost-effectiveness) by helping policymakers, donors, and nongovernmental organizations (NGOs) choose between a nearly infinite range of development program possibilities. While REs have some drawbacks-and doing them well is often an art-they have the undeniable strength of transparency and usability. A simple comparison of means between treatment and control groups can persuasively illustrate the impact of many different types of development policies and NGO programs.

(Continues...)

Excerpted from What Works in Development? Copyright © 2009 by Brookings Institution Press. Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

Edition details

• Publisher: Brookings Institution Press
• Publication date: 2009
• Language: English
• ISBN 10: 0815702825
• ISBN 13: 9780815702825
• Binding: Paperback
• Number of pages: 257
• Editor: Cohen Jessica