Recollection: The COVID-19 Vaccines Evaluation Program: Implementation, Management, and Experiences, 2021–2023

The coronavirus disease 2019 (COVID-19) pandemic has presented a significant threat to global health (1). In response, many nations, including China, embarked on the rapid development, testing, manufacturing, and distribution of COVID-19 vaccines using diverse technological approaches. While regulatory approvals were granted based on evidence from short-term randomized clinical trials (RCTs) demonstrating safety and efficacy within controlled demographics, these studies were not equipped to assess several critical aspects of vaccine performance. These aspects include the vaccine effectiveness (VE) in special populations, a wide range of outcomes such as protection against symptomatic infection, severe or fatal illness, the duration of protection against various outcomes, the necessity for booster doses, VE against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variants, and the overall impact of vaccination at the population level.

The initial cohort of COVID-19 vaccines developed in China, including Sinopharm and Sinovac inactivated vaccines, CanSino adenovirus-vectored vaccine, and Zhifei Longcom protein subunit vaccine, underwent safety and immunogenicity testing domestically, whereas efficacy assessments were conducted internationally (2). Following demonstration of their safety and efficacy, these vaccines received approval both within China and globally, ultimately being approved in over one hundred countries (3). During the initial two years of COVID-19 vaccine distribution, international real-world studies offered invaluable policy-relevant insights into the effectiveness of China-manufactured vaccines in diverse populations, insights that were difficult to acquire domestically due to the low incidence of infections and outbreaks in China at that time (4).

In mid-2021, China CDC introduced the COVID-19 Vaccine Evaluation Program (COVEP). This initiative marked the first time China CDC provided competitive grants to independent international research institutions for joint research endeavors. COVEP primarily aimed to support studies in nations extensively using China-manufactured COVID-19 vaccines, regardless of whether other vaccines were also in use. The principal research priorities encompassed targeted evaluation of VE against SARS-CoV-2 variants, VE in special demographics, the longevity of protection, the impact of booster doses, the overall influence of vaccination on the pandemic, and the severity and characteristics of breakthrough infections. Special attention was given to groups underrepresented in the phase 3 efficacy trials of these vaccines, notably the elderly, those with comorbidities, and pregnant women. The volatile progression of the COVID-19 pandemic required that the research adapt to changing conditions at international sites, necessitating both flexible study designs and research questions. The cooperative agreement framework was established to facilitate this adaptability. This article details the objectives, framework, procedures, impacts, and insights gained from COVEP.

The objectives of COVEP were to deliver policy-relevant scientific evidence concerning the real-world effectiveness of COVID-19 vaccines produced in China; to create effective frameworks for facilitating international research during the COVID-19 pandemic; to provide technical assistance to overseas researchers in their study execution; and to serve as a potential prototype for future international research initiatives by the China CDC.

COVEP is integrated within the National Immunization Program, which offers technical guidance for vaccine policy development and played a pivotal role in the execution of China’s COVID-19 vaccination strategy. Figure 1 illustrates the organizational framework and duties of COVEP, encompassing the Principal Investigator (PI), the steering committee, a project officer team, a financial management team, and an academic scientific technical support team. Members of the steering committee included representatives from the World Health Organization (WHO) China office, Gavi, the Bill and Melinda Gates Foundation China office, Fudan University, and China CDC. The committee facilitated the engagement of prospective international PIs with COVEP and offered counsel regarding scientific objectives and methodologies. Project officers and financial managers, employed by China CDC, provided support for funded research initiatives. Expertise for these funded endeavors was independently contributed by an academic team from Fudan University.

Figure 1. 

Structure and responsibilities of COVEP.

Abbreviation: COVID-19=coronavirus disease 2019; COVEP=COVID-19 Vaccines Evaluation Program.

Figure 2 presents the flow diagram of the COVEP processes. The identification of relevant projects was facilitated through a two-stage application procedure, beginning with the submission of a letter of interest (LOI). COVEP released a call for LOIs on the official China CDC website on October 5th, 2021 (5). This call detailed eligibility criteria, research objectives and scope, application guidelines, and review procedures. PIs from public health departments, academic institutions, and international technical agencies capable of conducting studies in countries using China-manufactured COVID-19 vaccines were invited to submit their LOIs by the end of 2021. Submissions required basic contact details, a concise study design outline, projections of project feasibility, a timeline (not to exceed 18 months), and an estimated budget (under one million US dollars).

Figure 2. 

Flowchart of the COVEP program.

Abbreviation: COVEP=the COVID-19 vaccines evaluation program; COVID-19=coronavirus disease 2019; LOI=letter of interest; CAs=cooperation agreements.

Twenty-five LOIs were received from thirteen agencies across eleven regions or countries. These LOIs were reviewed by the COVEP PI, project officers, and the scientific technical support team. Ten projects, originating from eligible institutions, were identified as being within the project scope. PIs of these projects were asked to submit detailed proposals within two months following LOI approval. These proposals included budgeting and detailed protocols describing study goals, design, methodology, timelines, and PI resumes. Throughout the proposal development phase, PIs and their teams were encouraged to engage in discussions with COVEP staff regarding study objectives and methods. The review panel evaluated and scored these full proposals, ultimately awarding funding to ten projects from nine institutions.

Under the advisement of the China CDC’s legal department, the COVEP team developed bilingual cooperation agreements in both Chinese and English. These agreements, which were fine-tuned through negotiations until a mutual consensus was reached between China CDC and the respective awardees, delineated specific rights and responsibilities. Key elements included research objectives and expected outcomes, standards and methodologies, financial protocols, intellectual property rights, contractual liabilities, clauses for unexpected project cessation, and provisions for force majeure that might necessitate amendments to the agreements or research protocols. Participants were offered a choice between two cooperative frameworks: financial only or combined financial and technical. Ten agreements were successfully negotiated and executed by all involved parties, with comprehensive proposals appended as implementation frameworks. These agreements explicitly stated that the project owners were responsible for manuscript drafting and publication decisions independently of China CDC. Additionally, it was mandated that manuscripts originating from COVEP-funded projects must acknowledge COVEP’s financial support.

Initial funding was disbursed to the funded institution within 20 working days following the signing of the cooperative agreement. Subsequent allocations were made after the required interim reports were submitted, with the remaining funds released upon receipt of final reports. Ethical approval was mandatory before the commencement of any research activities. Progress reports facilitated ongoing awareness of research developments and hurdles among project officers. Necessary adjustments to the project plans were implemented following discussions and negotiations between the project officers and PIs, in cooperation with the Fudan University technical support team.

Table 1 displays the list of the ten COVEP-supported projects, including institutions, research topics, basic study designs, and study countries/settings. COVEP funded projects across five of the six WHO regions: two in the African Region, three in the Region of the Americas, two in the European Region, one in the South-East Asia Region, and two in the Western Pacific Region. Supported entities comprised academic and government institutions as well as a health research enterprise. Studies were conducted among diverse populations including children, pregnant women, and individuals with comorbidities such as hypertension, diabetes, and HIV. All projects concluded by June 2023 with the acceptance of their final reports. Due to unexpected and unavoidable shifts in pandemic epidemiology, some project protocols or research questions required adjustments, achieved through mutual agreement between the PIs and COVEP.

COVEP facilitated pandemic responses by supporting independent, international research teams in generating practical, policy-relevant data on the effectiveness of COVID-19 vaccines produced in China, beneficial to nations utilizing these vaccines. The first project funded by COVEP to evaluate the real-world effectiveness of these vaccines was carried out by researchers at the University of Hong Kong, Hong Kong Special Administrative Region (SAR), China (6–7). The prompt dissemination of their findings was crucial for China’s domestic COVID-19 strategy, highlighting the need for very high vaccination coverage among the elderly to optimally protect against severe or fatal COVID-19 at the population level. Notably, the results from Hong Kong SAR, China, which demonstrated an over 90% VE against severe or fatal COVID-19, were later mirrored in national studies during the Omicron transmission phase (8–9). Another study, also funded by COVEP, was conducted in the Federation of Bosnia and Herzegovina (BiH) and assessed the effectiveness of two doses of the inactivated vaccine in individuals aged 60 and older during a period dominated by the Delta variant (10). This research concluded that a primary series of inactivated COVID-19 vaccines provided significant defense against moderate to severe COVID-19 in the elderly, though it also indicated diminishing immunity that suggested the need for a booster. Additionally, a scoping review of VE study methodologies, conducted by researchers in Colombia, revealed numerous studies on inactivated vaccines, yet the diversity in methodologies posed challenges for cross-study comparisons (11).

Final reports from COVEP studies provided additional evidence to the China CDC before formal publication. Key findings included the following: in the Republic of North Macedonia, three doses of inactivated vaccines were shown to protect against COVID-19 hospitalization. In Colombia, researchers demonstrated the effectiveness of three doses of CoronaVac in preventing death among individuals over 50 years and in protecting children aged 3–12 years from COVID-19. The COVEP project in Indonesia highlighted the benefits and safety of inactivated COVID-19 vaccines during pregnancy. A study conducted in Chile indicated diminishing protection 180 days post-vaccination, emphasizing the need for a booster dose. Additionally, a Zimbabwean study confirmed the effectiveness of the inactivated vaccine among people living with HIV.

Results from research funded by COVEP underscored scientifically reliable evidence bolstering the effectiveness of China-produced vaccines in addressing COVID-19 domestically and internationally. Furthermore, these findings affirm that China’s COVID-19 vaccination strategy was progressing appropriately, although there was a need to enhance efforts to achieve maximal vaccination coverage among the elderly population.

COVEP established an effective support system for facilitating independent international research studies during the COVID-19 pandemic. The successful completion of ten projects illustrates that COVEP could serve as a viable model for future international research initiatives. Key insights gained from the COVEP implementation are presented in Table 2, and are organized into categories of structure, relationships, and cooperative agreements.

The steering committee, in collaboration with international partners, guided the COVEP initiative and monitored progress to ensure objectives were achieved. The diverse viewpoints offered by committee members were invaluable for the multifaceted tasks involved in COVEP. They also promoted the funding opportunities worldwide and assisted in selecting highly qualified international investigators. Additionally, partnership with Fudan University contributed technical support to all COVEP-supported projects and enhanced the breadth of assistance available to international projects.

Independence of the investigators and projects was crucial in attracting highly qualified PIs to COVEP. Such autonomy enhanced the credibility of COVEP project findings by reassuring journals and their readers that the funding source did not influence manuscript preparation or publication decisions.

COVEP cooperative agreements outlined the mutual responsibilities of the implementing institutions and China CDC. These agreements were bilingual, transparent, and resulted from free negotiations. They offered comprehensive legal support for funded projects. Given the unpredictability of the pandemic, these projects required the flexibility to request methodological changes if the research environment shifted unexpectedly. To safeguard investigators from unforeseen circumstances beyond their control, the agreements included a force majeure clause.

COVEP is consistent with the primary functions and strategic objectives of the China CDC in advancing the national disease control and prevention system (12). Major areas of alignment comprise advancing collaborative efforts with premier public health institutions in higher education, boosting the China CDC emergency response team’s capability for remote and international support in managing acute infectious diseases, and enhancing capabilities in global public health governance and international collaboration through the development of talent equipped for global public health emergency responses, active participation in international public health assistance, and bolstering international public health cooperation and exchanges.

COVEP-sponsored research provided critical, policy-relevant findings regarding the key performance metrics of vaccines produced in China, which were challenging or impossible to obtain domestically in a timely manner due to the low number of cases in China at the time. COVEP established a viable model for international collaboration, facilitating research projects funded by China CDC and fostering partnerships with world-renowned experts in vaccines, epidemiology, and public health. Through effective collaboration with the financial, management, and legal departments of China CDC, COVEP developed protocols that support the execution of these international research partnerships. This initiative aligns with China CDC’s strategic plans and contributes to the advancement of the national disease control and prevention framework.

No conflicts of interest.

All staff members at COVEP and China CDC for their relentless dedication to the administration and management of this vital program.