Snapshot of Indicators
Survey Design for PMA India Phase 1
Performance Monitoring for Action (PMA), formerly PMA2020, builds on the previous success of PMA2020 surveys in India (Rajasthan) and focused on collecting routine data on key global indicators in family planning and reproductive health, while expanding content area to address questions of contraceptive decision-making and autonomy, in order to better understand the determinants and consequences of unique contraceptive use and patterns of use in India (Rajasthan). These are measured through three-related data collection activities: household and female surveys (HQFQ) producing both cross-sectional and longitudinal data, Service Delivery Point panel surveys (SQ), and an SDP client exit surveys (CQ).
In India (Rajasthan), a cross-sectional and panel household and female surveys (HQFQ) are conducted annually, with follow-up for the panel occurring at Year 2 and 3. The Service Delivery Point Survey (SQ) is conducted biannually with a baseline and a follow-up occurring 6 months after the baseline enrollment each year. The Service Delivery Point Client Exit Interview Survey (CQ) baseline data is collected at Year 1 and follow-up data will be collected annually.
PMA survey uses a multi-stage cluster design, with stratification at the urban and rural level and/or by sub-region. The enumeration area (EA) is the primary sampling unit, obtained from the national statistics agency of the respective geography. Within each urban/rural or sub regional stratum, EAs are selected using probability proportional to size (PPS) method. In each of the EAs, all households and private health facilities are listed and mapped prior to baseline data collection. Listings of public health facilities that serve the selected EAs at all three levels are obtained from the Ministry of Health.
For household and female cross-sectional and panel surveys, resident enumerators (RE) annually conduct a full listing of households within each enumeration area (EA). The annual listing is used to update the baseline weights to generate the cross-sectional estimates. At baseline, 35 households were randomly selected within each EA for interview. RE administered a household questionnaire, including completing a census of household members and guests who slept there the previous night for all selected households who consented to participate. Eligible female aged 15-49, who were usual members, were consented to participate in the female panel survey. Guests who slept at the households were included in the female cross-sectional survey.
PMA uses an open panel design, enrolling new eligible women at annual follow-up (year 2 and year 3). Households selected at baseline and still residing in the EA will be followed-up in subsequent rounds. Adolescents in selected households who were 14 years in the previous round will be enrolled in the panel as 15-year-olds starting in Year 2. Women who were 49 years at an earlier round will not be interviewed in subsequent rounds. Households who moved out of the EA since baseline will be considered lost-to-follow-up. New households residing in residential structures of households interviewed at baseline will be enumerated and enrolled in the panel in subsequent rounds. In addition, when an initially sampled housing structure is vacant or demolished before Year 2 or Year 3 rounds, a new household will be randomly selected form the new household listing to replace the lost one.
PMA India survey target sample size was determined based on the estimate of modern contraceptive prevalence rate (mCPR) among all women, with the 3% margin of error at the state level and 5% margin of error for urban and rural areas.
The Phase 1 survey includes 134 enumeration areas (EAs) selected using a multi-stage stratified cluster design with urban-rural strata. The results are representative at the state level and within urban/rural strata. The final samples included 4,577 households and 5,408 de facto females age 15-49, 575 facilities, and 521 family planning service clients who completed the interviews. Data collection was conducted between August and November 2021.
PMA uses standardized questionnaires to gather data about households, individual females, health facilities and family planning service clients that are comparable across program countries and consistent with existing national surveys. These questionnaires were based on model questionnaires designed by PMA staff at the Bill & Melinda Gates Institute for Population and Reproductive Health at the Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland, USA; the Indian Institute of Health Management Research (IIHMR) University in Jaipur; and the fieldwork materials of the India National Family Health Survey.
Four questionnaires were used to collect data in the PMA India (Rajasthan) Phase 1 survey: the household questionnaire, the female questionnaire, the service delivery point questionnaire, and the client exit interview questionnaire. Prior to launching the survey in each country, local experts review and modify these questionnaires to ensure all questions are appropriate to each setting. All questionnaires were translated into Hindi, using available translations from similar population surveys and the experts in translation.
PMA questionnaires are administered using Open Data Kit (ODK) software and Android smartphones. The questionnaires are in English and could be switched into Hindi on the phone. The interviews were conducted in the local language, or English in a few cases when the respondent was not comfortable with the local language.
Female resident enumerators (REs) in each EA administered the household and female questionnaires in the selected households. The household questionnaire gathers basic information about the household, such as ownership of livestock and durable goods, as well as characteristics of the dwelling unit, including wall, floor and roof materials, water sources, and sanitation facilities. This information is used to construct a wealth index.
Female resident enumerators (REs) in each EA administered the household and female questionnaires in the selected households. The household questionnaire gathers basic information about the household, such as ownership of livestock and durable goods, as well as characteristics of the dwelling unit, including wall, floor and roof materials, water sources, and sanitation facilities. This information is used to construct a wealth quintile index.
The first section of the household questionnaire, the household roster, lists basic demographic information about all usual members of the household and visitors who stayed with the household the night before the interview. This roster is used to identify eligible respondents for the female questionnaire.
The female questionnaire is used to collect information from all women age 15 to 49 who were listed on the household roster at selected households. The female questionnaire gathers specific information on: education; fertility and fertility preferences; family planning access, choice, and use; quality of family planning services; and exposure to family planning messaging in the media, migration, empowerment, and the impact of the Covid-19 pandemic on household and family planning access.
The Service Delivery Point (SDP) questionnaire collects information about the provision and quality of reproductive health services and products, integration of health services, and water and sanitation within the SDP. The client exit interview collects information about family planning services and contraceptive counseling, user experience with the current method, as well as contraceptive use, discontinuation, and future use.
Training
The PMA/Rajasthan fieldwork training started with a Training of Trainers refresher session hosted over Zoom followed by resident enumerator (RE) trainings. Refresher trainings for REs began July 29, 2020. Training included content of the survey as well as protocols for collecting data during the COVID-19 pandemic. The trainings were led by PMA staff from the Indian Institute of Health Management Research (IIHMR) University in Jaipur, with support from the Bill & Melinda Gates Institute for Population and Reproductive Health of the Johns Hopkins Bloomberg School of Public Health. Field supervisors, supported by the central team and PMA team, then became the trainers for the subsequent resident enumerator (RE) training sessions that took place before the start of data collection.
Throughout the training, resident enumerators (REs) and supervisors were evaluated based on their performance on phone-based assessments. The RE training was conducted primarily in Hindi, whereas some small group review sessions were conducted in other local languages.
Supervisors received additional training prior to and after the RE training to further strengthen their supervision skills, including instruction on conducting re-interviews, carrying out random spot checks, and dealing with the local/community leaders and engaging the communities.
Data Collection & Processing
Unlike traditional paper-and-pencil surveys, PMA uses Open Data Kit (ODK) Collect, an open-source software application, to collect data on mobile phones. All the questionnaires were programmed using this software and installed onto all project smartphones. The ODK questionnaire forms are programmed with automatic skip-patterns and built-in response constraints to prevent data entry errors.
The ODK application enabled REs and supervisors to collect and transfer survey data to a central ODK Aggregate cloud server. This instantaneous aggregation of data also allowed for daily monitoring of data collection progress, concurrent data processing, and course corrections while PMA was still active in the field. Throughout data collection, the central staff at central staff at IIHMR in Jaipur, Rajasthan and the Gates Institute at Johns Hopkins in Baltimore, Maryland routinely monitored the incoming data and notified field staff of any potential errors, missing data, or problems found with form submissions on the central server.
The use of mobile phones combined data collection and data entry into one step; therefore, data entry was completed when the last interview form was uploaded at the end of data collection.
Once all data were on the server, data analysts cleaned and de-identified the data, applied survey weights and prepared the final data set for analysis using Stata® version 16 software.
A total of 4,690 households were selected for the survey; 4,633 households were found to be occupied at the time of the fieldwork. Of the occupied households, 4,577 (98.8%) consented to a household-level interview. The response rate at the household level was higher in rural (99.3%) than in urban (97.7%) areas.
In the occupied households that provided an interview, a total of 5,511 eligible women age 15 to 49 years were identified. Overall, 98.1% of the eligible women were available and consented to the interview. The female response rate was slightly higher in the rural (98.1%) relative to the urban (98.0%) enumeration areas (EAs). Only de facto females are included in the analyses; the final completed de facto female sample size was 5,408.
All SOI indicator estimates are weighted. Weights are generated to account for non-response.
Sample errors and confidence intervals for selected indicators are generated using Wilson method. Click here more information about the indicators, including estimate type and base population.
Indian Institute of Health Management Research (IIHMR) University in Jaipur; and the Bill & Melinda Gates Institute for Population and Reproductive Health at The Johns Hopkins Bloomberg School of Public Health. Performance Monitoring for Action (PMA) Survey Phase 1, PMA/India-P1(Rajasthan) Snapshot of Indicators. 2020. India and Baltimore, Maryland, USA.