A.  The Issue

The World Bank has been publishing its Doing Business report annually, from the first released in September 2003 (and titled Doing Business in 2004) until the one released in October 2019 (and titled Doing Business 2020).  It has always been a controversial report, criticized for a number of different reasons.  But it has also been one that gained a good deal of attention – from the news media, from investors, and from at least certain segments of the public.  And because it received a good deal of attention (indeed more than any other World Bank report), governments paid attention to what it said, especially about them.

It is not my intention here to review those criticisms.  They are numerous.  Rather, in this post I will present a different approach to how the results could have been presented, which would have been more informative and which might have quieted some (but not all) of the criticisms.

One feature of the report, which was also the most widely discussed aspect of the report, was its ranking of countries in terms of their (assessed) environment for doing business. Because the Doing Business reports were widely cited, countries paid attention to those rankings and by various methods sought to improve their rankings.  These were often positive and productive, with countries seeking to simplify business regulation so that businesses could operate more effectively.  While some criticized this as simplistic, it is difficult to see a rationale, for example, justifying that in Venezuela (in 2020), to start a business would require completing 20 different procedures normally requiring (given the times for review and other requirements) an estimated 230 days.  In contrast, in New Zealand, starting a business involves only one procedure and can be completed in half a day.  Of course, few new businesses in Venezuela actually take 230 days to get started.  Either they are simply ignored (with bribes then paid to the police to leave them alone despite their violation of the regulations), or bribes are paid to obtain the licenses without going through the complex processes.

Given the prominence of the report, some countries undertook to improve their rankings through less positive means.  Sometimes the system could be gamed in various ways (e.g. to enact some “reform”, but then to limit its application narrowly so as to fit the letter, but not the spirit, of what was being assessed).  Or sometimes pressure could be applied on those submitting the data to slant it in a favorable direction.

And sometimes a country would try to apply political pressure on World Bank management in order to secure more favorable treatment.  A recently released independent investigation (commissioned by the World Bank Board, and undertaken by the law firm WilmerHale), found that there was such pressure (or at least perceived such pressure) brought by China on Bank management to ensure its ranking in the 2018 report would not fall.  At the instigation of the president’s offiice, and then overseen by the then #2 at the World Bank (Kristalina Georgieva – now the head of the IMF), Bank staff were directed to re-examine the ratings that had been assigned to China, and indeed re-examine the methodology more broadly, to see whether with some set of changes China’s ranking would not fall.  According to the WilmerHale report, several different approaches were considered and tried until one was found which would lead to the desired outcome.  In the almost final draft of the report, just before its planned publication, China’s ranking would have fallen from #78 in 2017 to #85 in 2018 – a fall of 7 places.  But with the last minute changes overseen by Georgieva, China’s ranking in 2018 became #78 – the same as in 2017.  And that was what was then published.

B.  An Alternative Approach

This focus on rankings is misguided, although not surprising given how the results are presented.  A country may well have enacted significant measures improving its business environment, but if countries a bit below it in the previous year’s rankings did even more, then the country could see a fall in its ranking despite the reforms it had undertaken.  And that fall in ranking would then often be interpreted in the news media (as well as by others) as if the business environment had deteriorated.  In this example, it had not.  Rather, it just did not improve as much as others.  But this nuance could easily be missed, and often was.

A different presentation in the Doing Business reports could have addressed this, and might have made the report a bit less controversial.  Rankings, by their nature, are a zero-sum game, where a rise in the ranking of one country means a fall in the ranking of another.  While this is needed in a sports league, where a tiny difference in the seasonal won/loss record can determine the ranking and hence who goes to the playoffs, it is not the same for the business environment.  Investors want to know what is good and what is not so good in an absolute sense, and small differences in rankings are of no great consequence.  And while some might argue that countries are competing in a global market for investor interest, there are far more important issues for any investor than some small difference in rankings.  For example, China and Malta were ranked similarly for several years in the mid-2010s, but whether one was a bit above or a bit below the other would be basically irrelevant to an investor.

An alternative presentation of the findings, based on comparison to an absolute rather than relative scale, would have been more meaningful as well as less controversial.  Specifically, countries could have been compared not against each other in every given year to see if their rankings had moved up or down, but rather to what the set of scores were (and consequent rankings were) in some base year.  That is, one would see whether their business environment had gotten better or worse in terms of the base year set of scores and consequent rankings (what one could call that base year’s “ladder” of scores).  The Doing Business project had that data, reported the underlying scores, and used those scores to produce its rankings.  But there was no systematic presentation of how the business environments may have changed over time, with this then compared to what the rankings were in some base period.

There were, I should note, figures provided in the Doing Business reports on the one-year changes in scores for each individual country, but few paid much attention to them.  They were for one-year changes only, and did not show the more meaningful cumulative changes over a number of years.  Nor did they then show how such changes would affect the country’s position on an understandable scale, such as what the ratings were across countries in a base year.

In this new approach, the comparison would be to an absolute scale (the scores and consequent rankings in the base year), not a relative ranking in each future period.  The issue is analogous to different types of poverty measures.  Sometimes poverty in a country is measured as the bottom 10% of the population (ranked by income).  This is a useful measure for certain things (such as how to target various social programs), but will of course always show 10% of the population as being “poor” regardless of how effective the social programs may have been.  In contrast, one could have an absolute measure of poverty (for many years the World Bank used the measure of $1 per day of income per person), and one could then track how many people were moving out of poverty (or not) by this measure.  Similarly, with the relative ranking of the 190 countries covered in the Doing Business reports, one will always find a mean ranking of 95, regardless of what countries may have done to improve their business environment.  It is, obviously, simply a relative measure, and not terribly useful in conveying what has been happening to the business environment in any given country.  Yet everyone focuses on it.

Any comparisons over time of these scores must also be over periods where the methodological approach used (precisely what is measured, and what weights are assigned to those measures) has not changed.  Otherwise one is comparing apples to oranges, where changes in the scores may reflect the methodological changes and not necessarily changes in the business environment of the countries.  Such changes in the methodological specifics have been criticized by some, as such changes in methodology will, in itself, lead to changes in rankings.  But one should expect periodic changes in the methodological approach used, as experience is gained and more is learned.

C.  The Results

The 2016 to 2020 period was one where the methodological specifics used in the Doing Business reports did not change, and hence one can make a meaningful comparison of scores over this period.  The data needed are what they specifically call the “Doing Business Scores”, which are the absolute values for the indices used to measure the business environment.  They range from 100 (for the best possible) to 0 (for the worst).  These scores have (at least until now) been made publicly available in the online Doing Business database.  I used these to illustrate what could be done to present the Doing Business results based on an absolute, rather than relative, scale.

The results are presented in a table at the end of this post, and readers might want to take a brief look at that table now to see its structure.  The base year is 2016, and the calculations are based on the changes in the country’s Doing Business Scores over the period from 2016 to 2020.  No country scores 100, and the top-ranked country (New Zealand) had an overall score of 87.1 in 2016 and slightly less at 86.8 in 2020.  I re-normalized the scores to set the top score (New Zealand’s 87.1 in 2016) to 100, with the rest then scaled in proportion to that.

The scores are thus shown as a proportion to what the best overall score was in 2016.  And this was done not just for the 2016 scores but also for the 2020 scores.  Importantly, the 2020 scores are not taken as a proportion of the best score for any country in 2020, but rather as a proportion of what the best score was in 2016.  Those scores are shown in the last two columns of the table, first for 2016 (as a proportion of what the best score was in 2016 – New Zealand at 87.1), and then for 2020 (again as a proportion of what the best score was in 2016 – New Zealand’s 87.1).  Note that on this scaling, New Zealand in 2016 would be 100.00, while in 2020 its rating would fall very slightly to 99.66 (as the Doing Business Score for New Zealand fell slightly from 87.1 in 2016 to 86.8 in 2020, and 86.8 is 99.66% of the 2016 score of 87.1).

The first two columns in the table show the rankings of countries, as the Doing Business reports have them now, for 2016 and then for 2020.  The sequence in the table is according to the ranking in 2016.  The third (middle) column then shows where a country would have ranked in 2016, had they had then the business environment that they had in 2020.  These are shown as fractional “rankings”, where, for example, if a country’s score in 2020 was halfway between the scores of countries ranked #10 and #11 in 2016, then that country would have a “rank” of 10.5.  That is, that country would have ranked between those ranked #10 and #11 in 2016, had they had a business environment in 2016 that they in fact had in 2020.

This now provides an absolute measure of country performance over time.  It is a comparison to where they would have ranked in 2016 had they had then what their policies later were.  This will then provide a more accurate account of what has been happening in the business environment in the country.  Take the case of Germany, for example.  It ranked #16 in 2016 and then lower at #22 in 2020.  Many would interpret this as a business environment that had deteriorated over the period.  But that is in fact not the case.  The absolute score was 91.27 in 2016 and a slightly improved 91.50 in 2020.  The business environment became slightly better (as assessed) between those two years, and it would have moved up a bit to a “rank” of 15.3 if it had had in 2016 the business environment it had in 2020.  But because a number of countries below Germany in 2016 saw a larger improvement in their business environment by 2020 than that of Germany, the relative ranking of Germany fell to #22.

Some of the differences could be large.  El Salvador, for example, ranked #80 in 2016 and fell to #91 in the traditional (relative) ranking for 2020.  But its business environment in fact improved significantly over the period, where the business environment it had in 2020 would have placed it at a ranking of 70.0 in 2016.  That is, it would have seen an improvement by 10 positions between the two periods rather than a deterioration of 11.  But other countries moved around as well, changing the relative ranking even though in absolute terms the business environment in El Salvador became significantly better.

D.  The Politics

One can understand how politicians in some country could grow frustrated if, after a period where they pushed through substantial (and possibly politically costly) reforms that aimed to improve their business environment, they then found that their Doing Business ranking nonetheless declined.  One might try to explain that other countries were also changing, and that despite their improvement other countries improved even more and hence pushed them down in rank.  This can follow when the focus is on relative rankings.  But this is not likely to be very convincing, as what the politicians see reported in the news media is the relative rankings.

A calculation of how the ratings would have changed in absolute terms would not suffer from this problem.  Countries would be credited for what they accomplished, not based on how what they did compares to what other countries might have done.  If all countries improved, then one would see that reflected when an absolute scale is used.  But when the focus is on relative rankings, any move up in rank must be matched by someone else moving down.  It is a zero-sum game.

Such a change in approach might have made the Doing Business reports politically more palatable.  It might also have reduced the pressure from countries such as China.  Throughout the period of 2016 to 2020, China was taking actions which, in terms of the Doing Business measures, were leading to consistently higher absolute scores each year, including for 2018.  With an absolute scale, such as that proposed here, one would have seen those year-by-year improvements in China’s position relative to where it would have been in a base year ranking (2016 for example).  The improvements were relatively modest in 2017 and 2018, and then much more substantial in 2019 and 2020.  But despite that (modest) improvement in 2018, China’s relative ranking in that year would have fallen 7 places to #84 from the #78 rank in the published 2017 report.  China found this disconcerting, and the WilmerHale report describes how Bank staff were then pressured to come up with a way to keep China’s (relative) ranking no worse than the #78 position it had in 2017.  And they then did.

At this point, however, a move to a presentation in terms of an absolute scale is too late for the Doing Business project as it has operated up to now.  Following the release of the WilmerHale report, the World Bank announced on September 16 that it would no longer produce the Doing Business report at all.  Whatever it might do next, if anything, will likely be very different.