November 30, 2020

Behind the Scenes of the PMA COVID-19 Survey

View All News
DRC RE training

When the COVID-19 pandemic rapidly spread throughout the world, devastating families and communities, the Performance Monitoring for Action (PMA) team mobilized quickly to include levels of knowledge of the pandemic and the implications for women’s health and their economic prospects in their family planning surveys. The survey collected data on the knowledge of COVID-19 and impacts of this pandemic in the Democratic Republic of Congo (DRC), Kenya, Burkina Faso and Nigeria and revealed the dire impact the virus has had on the economic status of families. The results of the surveys may be found here.

PMA is a family planning and sexual and reproductive health data collection project funded by the Bill & Melinda Gates Foundation, with direction and support provided by the Bill & Melinda Gates Institute for Population and Reproductive Health at the Johns Hopkins Bloomberg School of Public Health, and Jhpiego, in collaboration with national partners in each project country.

Rapidly adapting the family planning survey platform to include COVID-19 data was a herculean effort. Senior Communications Specialist Laura Wells spoke with several PMA managers to learn the challenges involved in such an effort and what the results have been.

What are you hoping to achieve in collecting COVID-19 data in countries where PMA works?

COVID-19 rapidly spread in countries where PMA operates and had the potential for a devastating impact on the safety, security, and economic status of families. We realized that we were in a position to quickly collect data on COVID-19 that could inform each country’s response to the pandemic- so that was our primary goal. We also have particular interest in the impact of COVID-19 on family planning and contraceptive use.

The COVID-19 pandemic hit the world quickly. What was involved in adapting PMA data collection to the context of the pandemic?

Once we determined that we wanted to collect COVID-19 data, we first had to obtain the consent of the Bill & Melinda Gates Foundation to re-purpose other data collection for this endeavor, and agreement from our in-country principal investigators (PIs) that this would be important and useful for COVID-19 efforts in their countries. Our PIs in-country then liaised with COVID-19 authorities in their countries and considered logistical feasibility. We worked with our PIs to create a new COVID-19 survey instrument, and then coded the instrument in the Open Data Kit software platform. Next, we had to adapt our training approach to the constraints of social distancing mandates within each country. In short, this required creating an entirely new remote training approach, and one that worked for different levels of internet availability/reliability. In contrast to PMA core data collection, COVID-19 interviews were done by phone, so we had to train the interviewers and others in this new survey mode. We also developed a new approach to dissemination, in which we created online dashboards with results for key indicators, along with briefs of detailed results. And finally, we rapidly evolved the way we disseminate our results from in-person meetings and workshops to using virtual platforms.

How were PMA Baltimore and in-country partners able to quickly and effectively develop and implement training for the interviewers on the new module during social distancing restrictions and other challenges presented by COVID-19?

As we began planning for the PMA COVID-19 survey, the project considered options for interviewer training. Country teams deliberated the feasibility of remote training with interviewers participating from home, while also considering current government restrictions and trends in new COVID-19 cases. We recognized the inherent health risks to in-person trainings, but we also acknowledged that conducting minimal or no training carried substantial risk of poor data quality.

Recognizing that interviewers had limited familiarity with online learning, our goal was to develop a training system that mimicked the in-person experience. This included sharing content through video lectures, reinforcement through small group activities, evaluation through electronic quizzes, and active monitoring via one-on-one phone calls between facilitators and interviewers. Interviewers accessed all training materials via their PMA Smartphones, which they also used for data collection. To reduce interviewers’ learning burden, we relied on platforms with which they were already familiar, namely WhatsApp, Google Drive, Open Data Kit and YouTube. We used three curated WhatsApp groups in every country, named “Info,” “My Group,” and “Q&A”, as the central location for information-sharing, small group work, and asking questions, respectively.

We posted training videos to the PMA private YouTube channel, and shared via links to this channel on the WhatsApp “Info” groups. Knowing Internet connectivity would be a challenge, we downloaded offline copies of training materials on Google Drive to the Smartphones before distributing them. To ensure that interviewers had a central reference while watching videos and completing activities, we also distributed a printed version of the training manual. Given the urgent need for the data, we developed ready-to-implement training materials, with options for country-specific adaptation.

What specific challenges did the interviewers face in data collection through phone interviews during COVID?

Phone networks were often unstable, especially on rainy days. Dropped calls were common, interrupting some interviews, and forcing the interviewers to complete a given interview over multiple calls. Some respondents were uncomfortable discussing COVID-19 over the phone because they felt they could become infected by talking about it. Some respondents stated they had no way of verifying the identity of the interviewer, leading to frank refusals to participate in some cases. Additionally, some husbands were not comfortable with their wives responding to a phone survey, resulting in the interviewer making repeated calls until the women were able to convince their husbands to allow them to participate. Other women requested late interview times due to their busy schedules.

How are you sharing the COVID-19 survey results with local governments and other health stakeholders to respond to the pandemic?

We have developed new approaches to disseminate our results. We created online dashboards for each geography, which we posted on the PMA website. Used for dissemination events, these dashboards contained key indicators from the PMA COVID-19 survey (e.g., economic impact, knowledge of COVID-19, behavior change), which could be cross-tabulated with sociodemographic characteristics. We also developed briefs for COVID-19 results to complement the dashboards. Finally, in some cases, we provided specific analyses that were requested by our in-country partners.

Using these materials, our PIs organized virtual dissemination events for a range of audiences in-country, including Ministries of Health, COVID-19 task forces, local and international NGOs, and others.

What kind of response are you seeing from PMA’s government partners?PMA data have been used to inform the response to COVID-19 in several countries.

The initial PMA dissemination with the Burkina Faso government’s COVID-19 task force led to a call for several workshops, with task force members, to focus on devising concrete actions in response to the PMA findings. A particularly illustrative example, as highlighted in a leading local newspaper, is that PMA data showed the importance of community and religious leaders as sources of COVID-19 information, leading to the creation of a civil society workshop to better engage these leaders in COVID-19 community response.

In the Democratic Republic of Congo, a member of the COVID-19 task force stated that PMA data will be used to inform the pandemic response, including the communication plan and other strategies.

Is there anything else you’d like to add about the challenges and results of the COVID-19 survey?

Our survey results show the devastating and widespread impact of COVID-19 restrictions on the economic well-being of households throughout sub-Saharan Africa. This was perhaps our clearest and most consistent result across geographies.

We learned that remote training cannot systematically replace in-person learning, but was effective for our COVID-19 data collection.