Original Article

Comparison of the Effective Reproduction Number (Rt) Estimation Methods of COVID-19 Using Simulation Data Based on Available Data from Iran, USA, UK, India, and Brazil

Background

The epidemiology of the contagious disease has changed over time, which refers to improved clinical interventions, as well as emerging threats, such as antimicrobial resistance. These diseases lead to widespread mortality worldwide each year.^1-3 One of the most dangerous contagious diseases of the coronavirus family is coronavirus disease 2019 (COVID-19), the outbreak of which has occurred since late 2019. The transmission rate of this disease is very high and is spreading rapidly in the world.^4-6 Estimation of the epidemiological features of contagious diseases, including COVID-19, is critical to assessing its prevalence in terms of transmission, predicting future outbreaks, and the effectiveness of disease control strategies.^7-9 Determination of effective reproduction number (R_t) is very important in monitoring and determining the epidemic situation of contagious diseases, as it determines the transmission rate of the disease.^10,11 The average number of secondary people infected by primary people is called the basic reproduction number (R0).^12,13 To determine the basic reproduction number (R0), it should be noted that the target population is susceptible to disease.^14,15 Due to the fact that in this period, the population in the mentioned countries is not completely susceptible, instead of estimating R0, we estimated R_t. To this end, several studies have been conducted, and the estimation of this index is based on existing methods. Attack rate (AR), exponential growth (EG), maximum likelihood (ML), time-dependent (TD), and sequential Bayesian (SB) methods available in susceptible-infectious-removed (SIR) and susceptible-exposed-infectious-removed (SEIR) multi-component models are the most important and most practical of these methods.^16,17 The important point here is that in most studies, randomly or arbitrarily, one or more methods have been used to determine R0 and R_t.^18-23 The purpose of comparing the methods studied in different countries is to check which method is known as the most accurate method in different data conditions of different countries. Another reason for this comparison is to compare the R_t values of different countries to check in which countries the epidemic situation occurred between June and October 2021. Moreover, it aims to investigate whether this reduction was more noticeable in countries that had a more comprehensive vaccination program in this period.

Considering the fact that there is a relatively large difference between the estimated values of R_t using these methods, as well as the difficulty of choosing the most accurate R_t, one of the challenging issues in these studies is whether the chosen method was the best one, and the estimated value of R_t was the most accurate. The answer to this question definitely requires a comprehensive and accurate study to use simulated data as well as real data, compare all these methods, and determine the most accurate method. Therefore, in the present study, in addition to a complete comparison of R_t values of different methods in different countries, simulated data have been used to determine the best and most accurate method using different scenarios. On the other hand, by using this index and choosing the best method, as well as achieving the exact effective reproduction number, it will be possible to detect the epidemic status of this virus.

Methods

The present study used five methods, namely AR, EG, ML, TD, and SB to estimate R_t. These methods are available in R0 statistical package, and to implement each of them, the required data were extracted from similar studies conducted over the same period in that country.²⁴ To this end, information including the time of onset, peak, and end of the epidemic was needed, which was determined using available data and sensitivity analysis. Moreover, it was necessary to accurately determine the generation time distribution, for which, according to similar studies, gamma distribution with different parameters was used.^25-29 In addition to this information, AR values based on similar studies were extracted for each of the studied countries and placed in the AR method.^30-33 Finally, after determining R_t using default and optimal approaches in different countries, these methods were compared using simulated data based on different scenarios. In the default approach, the same period of time was considered the length of the epidemic for all countries. However, in the optimal approach, using the appropriate commands in the used package, according to the number of daily items and available data, it was possible to consider a separate epidemic length in the analysis for each country.

Data

This study used two data sets. Actual COVID-19 data from Iran, the United States, the United Kingdom, India, and Brazil were collected on a daily basis from the Worldometers site, and other data, as mentioned before, were extracted from similar studies. These data are related to the period from the beginning of June to the end of September 2021, in other words, these data are related to the time when the new Omicron variant had not been yet identified in the world. The reason for selecting these countries was to compare Iran with four countries with the highest outbreak in this four-month period. Other data are simulated data based on different scenarios described in detail below.

Statistical analysis

As mentioned before, the applied statistical models in this study are AR, EG, ML, TD, and SB models, which are available in R0 statistical package. In this study, precise programming in R software was also used to estimate R_t, as well as simulate data, and then compare different models in addition to R0, EpiEstim, and EpiCurve packages.

Models

Attack rate method (AR)

In equation (1), AR is the attack rate, and S₀ presents the initial percentage of the susceptible population.^18,34

Exponential growth method (EG)

In equation (2), M is the Moment-generating function of generation time distribution. r is also an estimated parameter by Poisson regression.³⁵

Maximum likelihood method (ML)

In equation (3), let N₀, N₁,…, N_t identify incident cases over sequential time, and w_i is related to the GT distribution. μ_t is also related to the Poisson distribution parameter obtained by maximizing

$ll\left(R\right)={\displaystyle \sum _{t=1}^T\log \left(\frac{e^{-{\mu }_t}{\mu }_t^{N_t}}{N_t\text{!}}\right)}$ .^36,37

Time-dependent method (TD)

In equation (4), R_j is the effective reproduction number for the j^th person obtained from

$R_j={\displaystyle \sum _{i=1}^n\frac{N_{i}w\left(t_i-t_j\right)}{{\displaystyle {\sum }_{i\ne k}{N}_{i}w\left(t_i-t_k\right)}}}$ .³⁸

Sequential Bayesian method (SB)

In equation (5), N₀, N₁,…, N_t + 1 follows the Poisson distribution. This equation is completely different from the previous one since in order to estimate the effective reproduction number (R_t), classical inference logic is used in to . In contrast, Bayesian inference is used in . In the above equation, P( R │ N_i) is the posterior probability distribution, L(R;N_i)signifies the likelihood function, and P(R) presents the prior probability distribution, which is determined based on the posterior probability distribution of the previous days. Here, the value of R is estimated based on the maximum of the posterior probability distribution function.^21,39

Simulation study

In order to compare the studied methods, as well as to achieve the most accurate method in estimating the effective reproduction number (R_t), we designed a simulation study based on different scenarios. In this study, in order to increase the similarity of the simulated data to the real data, the data of five countries (Iran, USA, UK, India, and Brazil) were used to design different scenarios. These scenarios were designed considering the generation time distribution (GT), as well as the distribution of new cases, according to the dispersion status.

The gamma distribution (4.55, 3.30) as the GT distribution and the epidemic interval of 60 days with a peak at day 40 were used in scenarios 1-8; the gamma distribution (4.70, 2.90) and the epidemic interval of 54 days with a peak at day 36 were used in scenarios 9-16; the gamma distribution (5.00, 2.24) and the epidemic interval of 20 days with a peak at day 10 were used in scenarios 17-24; the gamma distribution (6.00, 3.80) and the epidemic interval of 40 days with a peak at day 21 were used in scenarios 25-32; and in scenarios 33-40, the gamma distribution (3.97, 3.29) and the epidemic interval of 41 days with a peak at day 30 were utilized. In each of these 40 scenarios, the negative binomial distribution or Poisson distribution was employed as the distribution of new items, respectively. Therefore, the epidemic started with one case at time t = 0, and then, for each case, the secondary cases were generated based on these two distributions in different scenarios. Moreover, for each of these scenarios, four values were used for R_t (1, 1.5, 2, and 3). In total, 10 000 epidemics with more than 50 cases were simulated for each scenario. Finally, epidemic data were collected daily, as well as cumulatively for 7 days, and the methods were compared by calculating relative bias and MSE; furthermore, a method was selected as the superior one that in addition to its low relative bias value in estimating R_t, it had the lowest MSE.

Results

Application

According to the purpose of the study, first of all, R_t values were estimated for all five countries using different methods. To this end, default and optimal approaches were used. As expected, in the optimal approach, there is no significant difference in the estimated values of R_t among the different methods. In this period, it can be said that the highest R_t belongs to the UK and USA. The estimated R_t values based on the TD method showed that the R_t values for the UK (1.33; 95% CI: 1.14-1.52) and USA (1.25; 95% CI: 1.12-1.38) are substantially higher than those in other countries, such as Iran (1.07; 95% CI: 0.95-1.19), India (0.99; 95% CI: 0.89-1.08), and Brazil (0.98; 95% CI: 0.84-1.14) from June to October 2021. On the other hand, according to the estimated R_t values greater than 1, it can be said that during this period, an epidemic situation has been occurred in these two countries, as well as Iran (Table 1). However, there are still differences among the different methods of estimating R_t, and the reader may be hesitant to choose the most accurate indicator in the R_t estimation. To answer this question and choose the most accurate method, we used the simulation study, the results of which are presented in Table 2.

In addition to estimating R_t over a determined period of time (beginning of June to the end of September 2021), these values were also estimated with the corresponding weekly confidence interval, and the results for all five countries are presented in . According to the R_t trend in these graphs, it can be said that in all countries, in general, there is a non-linear trend for this interval. However, according to the results of these graphs, the epidemic interval of each country can be determined approximately.

Close

Figure 1. Estimated Rt with 95% CI in Iran, the USA, the UK, India, and Brazil using a non-parametric model

compares different methods for predicting incidence. As can be seen, the SB method, unlike other methods, underestimates or overestimates the incidence rate for all five countries. On the other hand, it can be said that despite the closeness of the estimation in the three methods, the estimated values in the TD method are much closer to the observed values. However, in order to determine the most accurate method, and whether the TD is really the best method for estimating R_t, a simulation study was designed, the results of which can be seen in the next section.

Close

Figure 2. Observed and model predicted incidence for each method in five countries

Comparison of methods

Various scenarios were designed to compare the studied methods. These scenarios are designed according to the desired distributions, as well as different values of R_t. The reason for designing these scenarios is dealing with different conditions contained in the COVID-19 data for a more accurate comparison of the used methods. The results of this simulation study can be seen in Table 2. In this study, the values of MSE and relative bias were determined for each method in different scenarios, and then, the method with the lowest value of MSE was selected as the best method in estimating the R_t of population. In a total of 40 scenarios, the lowest values of MSE for TD, SB, ML, EG, and AR methods were observed in 15, 6, 12, 1, and 6 cases, respectively. Therefore, considering the results of this study, it can be said that in all scenarios, there was the lowest amount of MSE in the TD method. The next method is the ML, which had the lowest MSE in a greater number of scenarios, compared to other methods.

Discussion

Comparing R_t, in different countries, it can be said that this value in the studied interval in Iran, is better than the UK and the USA, while its value is worse than Brazil and India. However, given the value of R_t, it can be said that during this interval, an epidemic situation has occurred in Iran, the UK, and the USA. The interesting point is that the differences among the R_ts in different countries are commensurate with the situation of the COVID-19 epidemic in all these countries. In the period from June to October 2021, the highest weekly value of R_t is also related to the UK. Perhaps the main reason can be attributed to the reduction of restrictions in this country, compared to other countries in this interval. On the other hand, according to the estimated values of R_t in different countries in this period, it can be claimed that vaccination has a significant effect on reducing the estimated value of R_t, thereby controlling the disease⁴⁰ since during this period, in some countries, one dose, and in some others, two doses of the vaccine was injected.

A comparison of the studied methods showed that in general, the SB method underestimates and overestimates the incidence rate, while this is not the case with other methods, the prediction of these methods for the incidence of COVID-19 is acceptable. Therefore, it can be expected that these methods, due to their accurate and acceptable prediction, are more appropriate methods for estimating R_t of susceptible populations. However, in order to give an accurate answer to the question of which method actually offers a more accurate estimate of R_t than others, a simulation study has been used.

Although there are different methods to estimate R_t, no specific and unique method has been identified that is superior to other methods. However, it can be said that five methods of TD, SB, ML, EG, and AR are used to estimate R_t more than other methods.^41-46 Therefore, the aim of the present study was to compare these methods and identify the most accurate method to estimate the R_t of population. According to the main purpose of the present study, which is to compare the studied methods in estimating the R_t of population, a simulation study was designed. This study tried to design different scenarios, different aspects, and existing data, to more accurately compare the studied methods.

In this study, considering MSE values, it was found that the estimation of R_t in the TD and ML methods are more accurate than that in other methods. The main reason for this superiority can be attributed to the GT distribution, because the appropriate and accurate distribution of GT is particularly important in estimating R_t, and this distribution in the TD is more completely selected and used than in other methods. Other reasons for this superiority include the incidence of new cases, which in this method are considered in an epidemic situation, while this is not the case with other methods. Moreover, TD method requires fewer details of data and parameters than other methods, which is an advantage.⁴⁷ On the other hand, as it is known, the logic of ML method to estimate R_t is the MLE method, which is a well-known and acceptable method with minimum bias in statistical analysis.⁴⁸ Here, according to the research results, it can be said that the SB method, unlike the AR, which is a suitable method for non-epidemic status, is a relatively acceptable method for epidemic status. In addition, the reason for the poor performance of the EG method to estimate R_t can be attributed to the low data dispersion, because this method is acceptable when the data are highly dispersed.^34,35

Conclusion

According to the research findings and results, it can be concluded that, in order to estimate R_t, the use of the most accurate method is better than using the most common one. According to the present study, there is a relatively large difference between the R_t estimates in different methods, and this difference highlights the importance of the present study in comparing these methods. Therefore, the general and important result of this study is that TD and ML methods provide a more accurate estimate of R_t of susceptible population than other methods. Therefore, in order to monitor and determine the epidemic situation, as well as control COVID-19 and similar diseases, the use of these two methods to more accurately estimate R_t is suggested. Moreover, TD and ML methods are more accurate than SB, EG and AR methods in predicting the incidence rate of COVID-19.

Conflict of interest

The authors declare no conflict of interest.

Ethical approval

This manuscript with the code IR.KMU.REC.1400.152has been approved by the Ethics Committee ofKerman University of Medical Sciences, Kerman, Iran.

Funding

This study received no specific grant from any funding agency in the public, commercial, or not-for-profit Sectors.

References

1. Infectious diseases and the COVID-19 scenario in Brazil. Braz J Infect Dis 2020; 24(4):277-8. doi: 10.1016/j.bjid.2020.07.002 [Crossref]
2. Infectious disease hospitalizations: United States, 2001 to 2014. Chest 2019; 156(2):255-68. doi: 10.1016/j.chest.2019.04.013 [Crossref]
3. Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009-2014. JAMA 2017; 318(13):1241-9. doi: 10.1001/jama.2017.13836 [Crossref]
4. Characteristics of SARS-CoV-2 and COVID-19. Nat Rev Microbiol 2021; 19(3):141-54. doi: 10.1038/s41579-020-00459-7 [Crossref]
5. The continuing 2019-nCoV epidemic threat of novel coronaviruses to global health - the latest 2019 novel coronavirus outbreak in Wuhan, China. Int J Infect Dis 2020; 91:264-6. doi: 10.1016/j.ijid.2020.01.009 [Crossref]
6. Nowcasting and forecasting the potential domestic and international spread of the 2019-nCoV outbreak originating in Wuhan, China: a modelling study. Lancet 2020; 395(10225):689-97. doi: 10.1016/s0140-6736(20)30260-9 [Crossref]
7. COVID-19: coronavirus vaccine development updates. Front Immunol 2020; 11:602256. doi: 10.3389/fimmu.2020.602256 [Crossref]
8. Coronavirus 2019-nCoV: a brief perspective from the front line. J Infect 2020; 80(4):373-7. doi: 10.1016/j.jinf.2020.02.010 [Crossref]
9. Hepatic complications of COVID-19 and its treatment. J Med Virol 2020; 92(10):1818-24. doi: 10.1002/jmv.26036 [Crossref]
10. Preventing epidemics with age-specific vaccination schedules. Math Biosci 1997; 142(2):63-77. doi: 10.1016/s0025-5564(96)00174-5 [Crossref]
11. Threshold parameters for epidemics in different community settings. Math Biosci 1995; 129(2):189-208. doi: 10.1016/0025-5564(94)00061-4 [Crossref]
12. The concept of R0 in epidemic theory. Stat Neerl 1996; 50(1):89-110. doi: 10.1111/j.1467-9574.1996.tb01482.x [Crossref]
13. Complexity of the basic reproduction number (R0). Emerg Infect Dis 2019; 25(1):1-4. doi: 10.3201/eid2501.171901 [Crossref]
14. Host density thresholds and disease control for fisheries and aquaculture. Aquac Environ Interact 2010; 1(1):21-32. doi: 10.3354/aei0004 [Crossref]
15. A brief history of R0 and a recipe for its calculation. Acta Biotheoretica 2002; 50(3):189-204. doi: 10.1023/a:1016599411804 [Crossref]
16. Quantifying asymptomatic infection and transmission of COVID-19 in New York City using observed cases, serology, and testing capacity. Proc Natl Acad Sci U S A 2021; 118(9):e2019716118. doi: 10.1073/pnas.2019716118 [Crossref]
17. Measuring and preventing COVID-19 using the SIR model and machine learning in smart health care. J Healthc Eng 2020; 2020:8857346. doi: 10.1155/2020/8857346 [Crossref]
18. The estimation of the basic reproduction number for infectious diseases. Stat Methods Med Res 1993; 2(1):23-41. doi: 10.1177/096228029300200103 [Crossref]
19. Different epidemic curves for severe acute respiratory syndrome reveal similar impacts of control measures. Am J Epidemiol 2004; 160(6):509-16. doi: 10.1093/aje/kwh255 [Crossref]
20. Real-time estimates in early detection of SARS. Emerg Infect Dis 2006; 12(1):110-3. doi: 10.3201/eid1201.050593 [Crossref]
21. Real time bayesian estimation of the epidemic potential of emerging infectious diseases. PLoS One 2008; 3(5):e2185. doi: 10.1371/journal.pone.0002185 [Crossref]
22. A preliminary estimation of the reproduction ratio for new influenza A(H1N1) from the outbreak in Mexico, March-April 2009. Euro Surveill 2009; 14(19):19205. doi: 10.2807/ese.14.19.19205-en [Crossref]
23. Estimating the effective reproduction number for pandemic influenza from notification data made publicly available in real time: a multi-country analysis for influenza A/H1N1v 2009. Vaccine 2011; 29(5):896-904. doi: 10.1016/j.vaccine.2010.05.010 [Crossref]
24. The R0 package: a toolbox to estimate reproduction numbers for epidemic outbreaks. BMC Med Inform Decis Mak 2012; 12:147. doi: 10.1186/1472-6947-12-147 [Crossref]
25. Population density and basic reproductive number of COVID-19 across United States counties. PLoS One 2021; 16(4):e0249271. doi: 10.1371/journal.pone.0249271 [Crossref]
26. Serial interval of novel coronavirus (COVID-19) infections. Int J Infect Dis 2020; 93:284-6. doi: 10.1016/j.ijid.2020.02.060 [Crossref]
27. Serial interval distribution of SARS-CoV-2 infection in Brazil. J Travel Med 2021; 28(2):taaa115. doi: 10.1093/jtm/taaa115 [Crossref]
28. Transmission dynamics of the COVID-19 epidemic in India and modeling optimal lockdown exit strategies. Int J Infect Dis 2021; 103:579-89. doi: 10.1016/j.ijid.2020.11.206 [Crossref]
29. Estimation of the serial interval and basic reproduction number of COVID-19 in Qom, Iran, and three other countries: a data-driven analysis in the early phase of the outbreak. Transbound Emerg Dis 2020; 67(6):2860-8. doi: 10.1111/tbed.13656 [Crossref]
30. HOSTED-England’s Household Transmission Evaluation Dataset: preliminary findings from a novel passive surveillance system of COVID-19. Int J Epidemiol 2021; 50(3):743-52. doi: 10.1093/ije/dyab057 [Crossref]
31. State-level tracking of COVID-19 in the United States. Nat Commun 2020; 11(1):6189. doi: 10.1038/s41467-020-19652-6 [Crossref]
32. A data-driven model for COVID-19 pandemic - evolution of the attack rate and prognosis for Brazil. Chaos Solitons Fractals 2021; 152:111359. doi: 10.1016/j.chaos.2021.111359 [Crossref]
33. Risk factors, epidemiological and clinical outcome of close contacts of COVID-19 cases in a tertiary hospital in southern India. J Clin Diagn Res 2021; 15(3):34-7. doi: 10.7860/jcdr/2021/48059.14664 [Crossref]
34. Population-level effects of suppressing fever. Proc Biol Sci 2014; 281(1778):20132570. doi: 10.1098/rspb.2013.2570 [Crossref]
35. How generation intervals shape the relationship between growth rates and reproductive numbers. Proc Biol Sci 2007; 274(1609):599-604. doi: 10.1098/rspb.2006.3754 [Crossref]
36. A likelihood-based method for real-time estimation of the serial interval and reproductive number of an epidemic. Stat Med 2008; 27(16):2999-3016. doi: 10.1002/sim.3136 [Crossref]
37. The reproduction number of seasonal influenza epidemics in Brazil, 1996-2006. Proc Biol Sci 2010; 277(1689):1857-66. doi: 10.1098/rspb.2009.1897 [Crossref]
38. Estimation of time-dependent origin-destination matrices using the path flow proportion method. J East Asia Soc Transp Stud 2013; 10:776-95. doi: 10.11175/easts.10.776 [Crossref]
39. Bayesian sequential estimation. Ann Stat 1977; 5(5):955-68. doi: 10.1214/aos/1176343951 [Crossref]
40. The dynamic effective reproductive number of COVID-19 during the epidemic in Iran. Iran J Public Health 2022; 51(4):886-94. doi: 10.18502/ijph.v51i4.9250 [Crossref]
41. Estimation of reproduction numbers of COVID-19 in typical countries and epidemic trends under different prevention and control scenarios. Front Med 2020; 14(5):613-22. doi: 10.1007/s11684-020-0787-4 [Crossref]
42. Estimation of the basic reproduction number (𝑅0) of the COVID-19 epidemic in Iran. Med J Islam Repub Iran 2020; 34:95. doi: 10.34171/mjiri.34.95 [Crossref]
43. Estimation of the basic reproduction number (R0) for the novel coronavirus disease in Sri Lanka. Virol J 2020; 17(1):144. doi: 10.1186/s12985-020-01411-0 [Crossref]
44. Estimation of the reproduction number and early prediction of the COVID-19 outbreak in India using a statistical computing approach. Epidemiol Health 2020; 42:e2020028. doi: 10.4178/epih.e2020028 [Crossref]
45. Impact of COVID-19 epidemic curtailment strategies in selected Indian states: an analysis by reproduction number and doubling time with incidence modelling. PLoS One 2020; 15(9):e0239026. doi: 10.1371/journal.pone.0239026 [Crossref]
46. Response to COVID-19 in South Korea and implications for lifting stringent interventions. BMC Med 2020; 18(1):321. doi: 10.1186/s12916-020-01791-8 [Crossref]
47. Comparison of methods to estimate basic reproduction number (R0) of influenza, using Canada 2009 and 2017-18 A (H1N1) data. J Res Med Sci 2019; 24:67. doi: 10.4103/jrms.JRMS_888_18 [Crossref]
48. Early transmission characteristics of influenza A(H1N1)v in Australia: Victorian state, 16 May - 3 June 2009. Euro Surveill 2009; 14(42):19363. doi: 10.2807/ese.14.42.19363-en [Crossref]