Benford’s Law and COVID-19 Reporting

Abstract
Trust in the reported data of contagious diseases in real time is important for policymakers. Media and politicians have cast doubt on Chinese reported data on COVID-19cases. We find Chinese confirmed infections match the distribution expected in Benford’sLaw and are similar to that seen in the U.S. and Italy and thus find no evidence of manipulation.Policy makers in the rest of the world should trust the Chinese data and formulatepolicy accordingly.
Highlights:
We find no evidence of manipulation of Chinese COVID-19 data using Benford’s Law.
Models on the trade-off between growth and deaths can be calibrated with Chinese data.
Future Chinese data post-quarantine should guide policy in other countries.

1 Chinese Reporting on the Coronavirus
Contrary to popular speculation, we find no evidence that the Chinese massaged their COVID-19 statistics. We use a statistical fraud detection technique, Benford’s (1938) Law, to assess theveracity of the statistics. This empirical finding is important because China was affected first.Policies to combat the global pandemic are informed by its response. Skepticism about theChinese data may result – and may already have resulted – in poor policy choices or result inthe public not accepting policy decisions to the detriment of society.
The media frequently claim the Chinese government has understated the numbers ofthose affected.1 Politicians echo these claims with President Trump declaring the reporteddeath toll and infections seemed “a little bit on the light side”. The on-going doubts over thecredibility of its published data are problematic as it impacts subsequent policy choices bycountries that saw epidemics later. Recent papers that rely on Chinese data for calibration andanalysis include: Models of economic activity and the trade-off with deaths such as Atkeson(2020), Jones, Philippon, and Venkateswaran (2020) and Alvarez, Argente, and Lippi (2020);Fang, Wang, and Yang (2020) predict the effect of movement restrictions on the spread of thedisease; the infection model produced by Imperial College London Ferguson et al. (2020) thatinformed U.K. government policy.2 Since the policy choice of many countries has been toundertake social distancing, travel bans, and lockdowns patterned after the successful choicesmade by China,3 it is important that policy makers know the data is reliable.
Lack of confidence in the Chinese data may have led to a slower response in Europe to theemergent pandemic. Chinese provinces neighboring Hubei province, the Chinese epicentre,imposed movement controls, quarantines and checks on January 23rd at a time when thenumber of confirmed cases in Hubei was 444 and the number of deaths was 17. Most majorcities in Hubei, including the provincial capital Wuhan, were also locked down at this time.4In comparison Italy, Europe’s initial pandemic hotspot, reached 445 cases on February 26th and17 deaths the following day. Italy imposed a regional quarantine on 50,000 people on February 22nd. Italy did not, however, impose a full regional lockdown until March 7th. It even tookuntil March 9th for the lockdown to be extended to the entire nation. Similarly, restrictionson international travel were too mild. France made travel advisories against travelling to partof Northern Italy and merely requiring self-quarantine for those returning from these areason February 23rd. Austria and Switzerland waited until March 10th and March 8th to imposecontrols on cross-border movement. By February 26th, Hubei had seen cases rise to 65,187 anddeaths to 2,615.
Chinese data from February had already shown the effectiveness of the quarantine measuresin slowing the progress of the disease, both within Hubei and in other Chinese provincessee Figure 1. The daily percentage change in the number of cases fell from over 100% to 30%within a week. Within two weeks it had fallen to 10%. By the end of February, the dailypercentage change in the number of confirmed cases in China had fallen to 1%. Yet, governmentsin Europe chose to delay their responses or to engage in far less aggressive measures.The popular media was skeptical about the effectiveness of quarantines5 and of the decline inChinese cases.
This skepticism about politically motivated manipulation of Chinese state statistics isdeeply rooted. Anecdotal and academic evidence point to lower level officials manipulatingdata to meet targets. In 2007, Chinese Premier Li Kejiang called all GDP measures “man-madeand therefore not reliable” when discussing data on Liaoning province.8 Lyu, Wang, Zhang,and Zhang (2018) find evidence that regional growth rates are manipulated to meet growthtargets. Specifically, in the case of the SARS outbreak in 2002-3, criticism of the Chinese responsesurfaced. The World Health Organization (WHO) suspected that China underreportedthe number of cases (see Parry, 2003; Ashraf, 2003).
Some data manipulation clearly took place early in the epidemic.10 The number of casesreported by the Wuhan authorities was “frozen” at 41 during the Hubei provincial ChinesePeople’s Political Consultative Conference and the Wuhan People’s Congress (Lianghui) betweenJanuary 12th and 17th, 2020. John Mackenzie, a member of the WHO emergency committeetold the Financial Times on February 5th that China must have been withholding informationon new cases from the WHO.11 By January 20th, Chinese President Xi Jinping issuedinstructions to all levels of government to put people’s safety first and take effective measuresto combat the epidemic. We will test the hypothesis that local governments ignored thisadmonition and chose to manipulate their COVID-19 case numbers.
2 Benford’s Law
Benford’s Law is used to detect fraud or flaws in data collection based on the distributionof the first digits of observed data. A Benford distribution of first digits arises naturallyfor processes that are exponential with multiple changes of magnitude, Michalski and Stoltz(2013). The spread of COVID-19 demonstrates exponential growth and changes of magnitude.
The frequency with which the first digit is “1” is 30.1%, the first digit is “2” is 17.6% etc,declining to the first digit being “9” only 4.6% of the time. Since it takes a 100% increase to gofrom “1” to “2” and a mere 11.1% increase to go from “9” to “1”, this logarithmic distributionmakes sense, as long as the data series is sufficiently large and has a number of changes ofmagnitude. See Table 1

The use of Benford’s Law to detect fraud has been widely demonstrated in economics and accounting (Varian, 1972). In economics, Benford’s Law has been used to determine if economic statistics have been manipulated by governments: Nye and Moul (2007), Gonzales-Garcia and Pastor (2009), Rauch, Göttsche, Brähler, and Engel (2011), Michalski and Stoltz (2013) and Holz (2014). In accountancy, Nigrini (1996) examined taxpayer records and used it to detect tax evasion. It has also been used in epidemiology, Idrovo, Fernández-Niño, Bojórquez-Chapela, and Moreno-Montoya (2011) examined the reported weekly number of confirmed cases from 35 countries during the A(H1N1) pandemic in 2009.

3 Fraud Detection Suggests No Data Manipulation
We compile data from the Johns Hopkins University Corona Virus Research Center, for Chineseprovinces and U.S. states.12 and for Italian provinces we use data from the daily Dipartimentodella Protezione Civile bulletins. We use the daily number of cases/deaths/recoveriesannounced for the analysis. We focus on periods when the number of national cases grows byat least 10% as after this the data series no longer follow an exponential path and the distributionof first digits is not likely to follow Benford’s Law (Table 2). The spread and scale of casesand deaths are shown in Figure 2 - 7 for China, U.S. and Italy and in Figure 8 for recoveriesin China. For confirmed cases the data series have multiple changes in magnitude for mostChinese provinces, U.S. states and Italian regions. Deaths do not show these multiple changesin magnitude for Chinese provinces and only to a limited extent in the U.S. and Italy. Chineseprovincial recoveries do show multiple changes in magnitude. More than two changesof magnitude are desirable for a series to follow Benford’s Law, which is why we focus on thenumber of confirmed cases.
The number of confirmed cases understates the true number of infections as China wasunable to test those who did not present at hospitals. However, the lack of tests is a problem inboth Italy and the U.S. The number of cases detected by health authorities in countries lackingtesting capacity is still valuable, see Harris (2020).

Figure 9 shows that for Chinese provinces, U.S. states and Italian regions the number ofconfirmed cases the distribution of the first digits shows a decline from 1 to 9 in line with the expected distribution of Benford’s Law. Figure 10 show the distribution of deaths is out of linewith Benford for Chinese provinces, but is reasonably close for the U.S. and Italy. Figure 11shows for Chinese provinces the number of recoveries appears close to Benford’s Law.

Tests of significance for Benford’s Law require that the "true" distribution should followthe Benford distribution. Our null hypothesis is that the observed distribution follows thetheoretical (Benford) distribution. The most common test is the Chi-Square test of Goodnessof Fit:

Table 3 displays our results. The Chi-Square test does not support a Benford distribution except for Italian regions’ number of confirmed cases, where the null hypothesis is rejectedat the 10% level. However, as noted in other papers on Benford’s Law, the Chi-Square test isextremely sensitive for large sample sizes and tends to reject statistical significance even forsmall differences. The Kuiper test does not reject the null hypothesis that the distribution isBenford for China, U.S. and Italy for confirmed cases. For China and Italy, the d and m testsalso do not reject the null. For the U.S. the m test rejects the null at the 5% level and the dtest rejects at the null at the 10% level. These tests of significance are supportive of the viewthat for confirmed cases, the distribution of the first digits follows Benford’s Law for all threecountries.

In general, for deaths most of the tests reject the null hypothesis at the 1% level althoughfor Italian deaths the Kuiper test does not reject the null and the d and m tests reject the nullat the 5% level. For US deaths the m test rejects the null at the 5% level. The US and Italyare closer to Benford than China probably because they have a higher number of deaths and awider geographical spread of deaths. Finally, Chinese recoveries show the Kuiper test rejectsthe null at the 1% level, the d rejects the null at the 10% level and m rejects the null at the 5%level.
4 Conclusion
China’s distribution of first digits for confirmed cases is in line with Benford’s Law. Thus wereject the hypothesis that the Chinese data has been manipulated. It also matches the distributionfound in the United States and Italy. An advantage of Benford’s Law is the inherentdifficulty of coordinating a distortion of figures in real time on a panel basis. It is possible tocreate data series that fit Benford’s Law (Diekmann, 2007). To manipulate the Chinese datain this fashion would would require someone to coordinate daily announcements across allprovinces while accurately forecasting future infection rates. This is improbable.
As China is at least a month ahead of Europe and six weeks ahead of the United States,its data should be used not only for calibration of models to inform policy measures to slowinfection, but also for guidance in the lifting of stay-at-home orders.