Change in sexual and reproductive health knowledge among young women using the conversational agent “Nthabi” in Lesotho: a clinical trial

Abstract

Background

Young women worldwide face problems like unwanted pregnancy and sexually transmitted infections. Providing sexual and reproductive health education to young women in low- and middle-income countries is a priority. It is unknown if using digital health interventions to deliver health education is effective in resource-constrained settings.

Methods

We conducted an interventional trial among young women in the low–middle-income country of Lesotho to test the impact of the conversational agent system (“Nthabi”) designed to deliver sexual and reproductive health information on either a smartphone or tablet for up to 6 weeks. We measured participant’s knowledge before and after the discussion of family planning, folic acid use, and healthy eating. The main outcome measure was a change in knowledge among participants in these three topic areas. The number of correct pre- and post-test responses was compared using generalized linear models that directly estimated the proportions and percentages of correct responses.

Results

We recruited 172 young women (mean age 22.5 years, 91% unmarried, 69% completed high school, 23% unemployed, 66% students) to use Nthabi on a mobile phone (11.6%) or loaned tablet device (88.4%). The mean number of interactions with Nthabi was 8.6. Family planning was chosen to be discussed by 82 of the 172 participants (52.2%), and of those, 49 (31.2%) completed the content on this topic, and 26 (16.6) completed the post-test. For the 11 questions about family planning, the total percent of correct responses increased from 85.5% (796 total responses) on the pre-test to 89.9% (320 responses) on the post-test (p = 0.0233). Folic acid use was chosen to be discussed by 74 (47.1%) of 172 participants, and of those, 27 (17.7%) completed the content on this topic, and all 27 (17.7%) completed the post-test. For the 5 questions about folic acid use, the total percent of correct responses was 45.3% (181 total responses) on the pre-test and 71.6% (111 responses) on the post-test (p < 0.0001).

Conclusions

The Nthabi conversational agent system increased knowledge of family planning methods and folic acid use among young women in Lesotho. Digital health interventions like Nthabi have the potential to offer new ways to deliver reproductive health information.

Trial registration

ClinicalTrials.gov registration April 20, 2020; ID: NCT04354168.

Background

Sexual and reproductive health (SRH) remains a global public health challenge among the 1.8 billion adolescents and young people ages 15–24 worldwide [1]. They face problems like unwanted pregnancy, and sexually transmitted infections (STIs) including human immunodeficiency virus (HIV) infections. This population also accounts for 42% of new HIV infections globally, and 4 in 5 young people with HIV live in sub-Saharan Africa [2].

Lesotho has the second-highest HIV prevalence in the world, at 22.7%, and one of the highest HIV incidences among adolescent girls and young women at 0.33% annually [3]. The maternal mortality ratio in Lesotho of 544/100,000 live births is the second highest in Southern African Development Community countries [4]. In Lesotho, this age group commonly reports low SRH knowledge and engages in risky sexual behaviors such as early sexual debut (14 years), unprotected sex, and multiple sexual partners resulting in high rates of STIs, including HIV, unintended pregnancy, and unsafe abortions [5]. Young people in low- and middle-income countries (LMIC) have limited access to SRH information and report challenges accessing SRH services such as providers' judgemental approach and fear of stigmatization [6]. In Lesotho, adolescent girls and young women commonly report low sexual and reproductive health knowledge. In a survey about comprehensive knowledge of HIV, only 30.7% of young women displayed comprehensive knowledge [7].

The rapid diffusion of mobile technologies among young people in Lesotho [8, 9] has the potential to deliver reliable SRH health education and could serve to expand the reach of health professionals in human resource-restricted settings.These technologies could be used to facilitate cost-effective large-scale health promotion efforts [10, 11].

Embodied Conversational Agents (ECAs) are computer-based animated characters designed to simulate face-to-face human interactions. They can automate tasks, improve access to healthcare services, and reduce health professionals’ workload [12, 13]. An ECA named “Gabby,” designed to deliver SRH to reproductive-age African American women in the United States, demonstrated a significant reduction in reproductive health risks in a randomized clinical trial [14, 15]. Subsequently, our research team adapted Gabby for use in Lesotho (named the “Nthabi Preconception Health Promotion Application” or “Nthabi”). The Nthabi character is a Mosotho nurse midwife designed to deliver SRH education to young women in Lesotho [16].

Our prior study reported survey results of the young women who used Nthabi. Results showed they perceived Nthabi to be effective, efficient, and culturally appropriate for delivering sensitive SRH information. Study participants agreed that Nthabi helped them make health decisions, they intend to use it beyond the study period and would encourage others to use it [17].

The aim of this study was to conduct an interventional trial to determine if interaction with Nthabi among the 172 young women enrolled from two rural districts in Lesotho results in a change in participants’ knowledge of SRH health topics assessed before and after using Nthabi.

Methods

Study design, population, and setting

In this interventional trial, we enrolled 172 young women from two rural districts in the low–middle-income country of Lesotho in southern Africa. Participants used the conversational agent system (“Nthabi”) designed to deliver sexual and reproductive health information on either a smartphone or tablet for up to 6 weeks. We measured participants’ knowledge before and after the discussion of family planning, folic acid use, and healthy eating with Nthabi. The main outcome measure was a change in knowledge among participants in these three topic areas.

The study was registered with ClinicalTrials.gov NCT04354168 (https://clinicaltrials.gov/study/NCT04354168?cond=Lesotho&rank=1&tab=results). The study was conducted according to the Consolidated Standards of Reporting Trials (CONSORT) [18]. The CONSORT checklist is available in Additional file 1 and the study protocol is available in Additional file 2.

Recruitment

The research team recruited participants by posting messages on social media (e.g., WhatsApp and Facebook) that described the study and asked potential participants to contact the research team to discuss enrolling in the study. A local non-governmental organization called Help Lesotho (Leribe District, Lesotho) which offers mentorship programs to adolescent girls and young women in the Leribe district, reached out to the research team after seeing the social media posting and offered to disseminate the recruitment announcement to their clients.

Second, the research team approached young women while they were waiting for consultation in the Adolescent Health Corners [e.g., clinics], the HIV, and Mother and Child Health ambulatory clinical departments at the Berea and Leribe government district hospitals. Third, students were approached at the Leribe Vocational School and the Limkokwing University of Technology to identify those interested in participating.

Eligibility

The inclusion criteria were: (1) Basotho women aged 18–28 years who were from the districts of Leribe or Berea and who accessed health services in these two districts, (2) self-reported ability to read and understand spoken English, (3) have access to an Android smartphone, and (4) able to access internet and Wi-Fi at least once at the end of the study. There were no exclusion criteria.

Enrollment

Once eligibility criteria were confirmed, the research team explained the purpose of the study, potential risks and benefits, reimbursement for travel costs, and the right to withdraw from the study at any time. After addressing any questions, participants were asked to sign an informed consent.

The research team then assisted the participants in downloading the Nthabi App on their mobile phones. Participants who were unable to download the App were loaned a Lenovo© (Beijing China) Android 11 OS platform tablet to use for 6 weeks. Participants created a unique username and password and were shown how to log on. Participants then began interactions with Nthabi and were encouraged to use the app at least once daily at their convenience for 6 weeks.

All participants received 50 Maloti (approximately US $5) to cover the cost of using phone data. Participants using tablets received an additional 50 Maloti to cover their travel back to the recruitment sites to return the devices.

Baseline data collection

Socio-demographic information was collected (age, marital status, educational level, employment status, recruitment site, and district).

Nthabi intervention description

We previously described [16] how we adapted an ECA used in the United States (“Gabby”) to create an agent (“Nthabi”) appropriate for Lesotho. Adaptations explored physical and cultural alternatives including the character’s sex, age, occupation, name, physical appearance (hairstyle, clothing), language and speech patterns, and personality believed to resonate with young Basotho women.

The final appearance and persona of Nthabi represent a Mosotho professional nurse midwife wearing a Lesotho nurse’s uniform. Her hairstyle (braids), complexion (medium, similar to the local population), facial expressions (calm and gentle), and mannerisms (a humble professional with a sense of humor) are relatable to young women in Lesotho (Fig. 1).

Fig. 1

Nthabi health education app participant interface

Nthabi’s language is English, as the English literacy rate among young women in Lesotho is above 90% [19], and English is spoken in health settings.

To establish the clinical topics to be included in the system, the Lesotho Ministry of Health key informants recommended five sexual and reproductive health topics for young women (family planning, HIV, tuberculosis (TB), healthy eating, and use of folic acid). The research team then used the Lesotho National Clinical Guidelines to create evidence-based dialogue for use in each of the content areas discussed in Nthabi interactions [16].

Participants chose from the list of health topics offered by Nthabi. Using conversational dialogue, Nthabi describes the importance of the topic and offers suggestions about how to take action on it. Participants engage by selecting a response from a multiple-choice menu updated at each turn of dialogue. For family planning, folic acid, and healthy eating, Nthabi requests that participants take a knowledge pre-test before providing health education, and when they complete the topic, the participants are asked to take a post-test.

Finally, technical adaptations were required to deploy Nthabi on smartphone screens rather than computer screens as done in the Gabby prototype. We constructed the app that is downloadable to users' mobile devices allowing availability when not in a Wi-Fi environment. Nthabi was made available on the Google Play store for download on mobile phones or tablets.

Data collection

Participants were invited to return to the recruitment site to access the internet at the completion of the study to facilitate downloading interaction data to the server, and to return the loaned tablets. When the participant is in a Wi-Fi environment, outcome data (number of logins, topics completed, and pre- and post-test results) is automatically downloaded to a secure server.

Analysis

Data were analyzed using Statistical Analysis System (SAS) software (v9.4) (Cary, NC, USA). The number of correct pre- and post-test responses was compared using generalized linear models that directly estimated the proportions (and percentages) of correct responses. For individual items, the models accounted for the pre-and post-test responses being paired for each participant. Models that compared paired responses across all multiple items also accounted for the clustering of multiple responses within individual participants. P values of < 0.05 were considered statistically significant.

Results

As shown in the CONSORT diagram (Fig. 2), the research team screened 436 young women for eligibility.Young women were recruited through social media (e.g., WhatsApp, Facebook) or direct contact at Limkokwing University of Technology (n = 150), Leribe Vocational School (n = 88), Leribe Health facilities (n = 55), Berea Health Facilities (n = 84) and Help Lesotho (n = 59).

Fig. 2

Consort diagram

Of those screened, 174 were ineligible due to having smartphones without an Android operating system, 64 had phones that were not smartphones, 10 had Huawei Android Smartphones without access to the Google Play store and at this time tablets were not available. Of those found eligible, 90 others were not enrolled due to reaching the limit on available tablets in that recruitment session. Consequently, 172 participants consented and were enrolled. Of those enrolled, only 20 had sufficient memory on their phones to download the Nthabi App. The remaining 152 participants received a tablet device to use.

In the weeks after enrollment, 12 participants opted out of the study because their phones froze when loading. Therefore, 160 young women used Nthabi, with eight using their phones and 152 using tablets. Of the 160, 157 started interactions with Nthabi and selected a topic to view (Fig. 2).

Table 1 shows the characteristics of the 172 participants who were consented and enrolled. The mean age was 22.5 years (standard deviation of 2.63), 157 (91%) were unmarried, 119 (69%) had completed high school, 19 (23%) were unemployed, and 113 (66%) were students. The recruitment sites of participants enrolled were: 34 (20%) from Limkokwing University of Technology, 60 (35%) from Leribe Vocational School, 31 (18%) from the Leribe Health Facilities, 7 (4%) from the Berea Health Facilities, and 40 (23%) from Help Lesotho.

Table 1 Characteristics of young women enrolled on Nthabi in Lesotho (n = 172)

Table 2 shows the number and percentage of participants enrolled who selected a health topic to review, completed a topic and those that then took the post-test. Among the three topics, family planning was most frequently chosen to be viewed by participants with 82 (52.2%) selecting this topic, and of those viewing family planning, 49 (31.2%) completed the topic, and 26 (16.6%) completing the post-test.

Table 2 Participants viewing a topic area, completing a topic area, and taking the post-test (n = 157)

For folic acid, 74 (47.1%) participants viewed the topic, of those, 27 (17.7%) listened to all the content, and all 27 (17.7%) completed the post-test. For healthy eating, 76 (48.4%) selected this topic to view, of those, only 15 (9.6%) completed the topic, and 13 (8.3%) completed the post-test.

Table 3 shows 91 (58.0%) participants completed at least one topic area with 49 completing family planning, 27 folic acid, and 15 healthy eating. Thirty-two participants (20.4%) completed one topic (24 family planning, 7 folic acid, and 1 healthy eating); 21 (13.4%) participants completed two topics (10 family planning, 8 folic acid, and 3 healthy eating); and 38 (24.2%) participants completed three topics (15 family planning, 12 folic acid, and 11 healthy eating).

Table 3 Number of topic areas completed by participants (n = 157)

Table 4 shows the results of the folic acid pre and post-test questions. The percent of correct responses to the 5 questions on the pre-test, increased from 45.3% (181 responses) to 71.6% (111 responses) on the post-test (p < 0.0001). Participants improved on each question. For example, question 4, “Folic acid reduces risks of birth defects” (true or false) showed an increase in the percent of correct responses from 80% (64 participants) on the pre-test to 96.8% (30 participants)on the post-test (p = 0.0029). Question 5 (“How much folic acid should women of childbearing age take daily to prevent birth defects?”) showed an increase in the percent of correct responses from 12.7% (10 participants) on the pre-test to 67.7% (21 participants) on the post-test (p < 0.0001).

Table 4 Folic acid pre-test and post-test responses

Table 5 shows the results of family planning pre and post-test questions. The percent of correct responses to the 11 questions increased from 85% (796 responses) on the pre-test to 89.9% (320 responses) on the post-test (p = 0.0233). For question 1 “Women can have an operation to avoid having any more children” (true/false) showed an increase in correct responses from 87.8% (79 participants) on the pre-test to 97% (32 participants on the post-test (p = 0.0461). For question 11, “As an emergency measure, within 5 days after they have unprotected sexual intercourse, women can take special pills to prevent pregnancy?” showed an increase in correct responses on the pre-test from 83.1% (69 participants) to 96.8% (30 participants on the post-test (p = 0.0106).

Table 5 Family planning pre-test and post-test responses

Table 6 shows the results of pre and post-test questions about the healthy eating content. The percent of correct responses to five questions increased from 69.5% (306 responses) on the pre-test to 72.6% (69 responses) on the post-test (p = 0.421).

Table 6 Healthy eating pre-test and post-test responses

The mean number of interactions (logons) was 8.6 per enrolled participant, with a maximum of 39 interactions.

Figure 3 shows that the number of correct responses (minus the incorrect responses) increases with increasing number of interactions. The curve flattens because when participants exhausted the content for folic acid, family planning, and healthy eating, they were then presented with additional content that did not contain pre- and post-test questions.

Fig. 3

Relationship between the number of interactions with Nthabi and the change in the number of correct responses pre-/post-test

Discussion

We previously adapted a conversational agent used in the United States ("Gabby") to create an agent (“Nthabi”) [16] appropriate for Lesotho. Nthabi’s appearance and persona represent a Mosotho professional nurse midwife who uses conversational dialogue to provide health education on sexual and reproductive health topics for young women. Our previous research showed that young women using the system in Lesotho perceived Nthabi to be effective, efficient, and culturally appropriate for delivering sensitive SRH information [17].

In this study, we recruited 172 young women from the low–middle-income country of Lesotho to use Nthabi on a mobile phone (11.6%) or loaned tablet device (88.4%) for 6 weeks after enrollment. The mean number of interactions with Nthabi was 8.6. Young women enrolled in this study demonstrated a significant increase in knowledge about family planning methods and preconception folic acid use after interacting with the Nthabi.

These data support the idea that systems like Nthabi have the potential to improve the delivery of sexual and reproductive health education in low- and middle-income countries. The use of m-health technologies might be particularly important in countries like Lesotho that face severe constraints on human resources for health that limit the delivery of health education and services. Systems like Nthabi could “leapfrog” over currently limited face-to-face health education and could represent the health care of the future in settings such as Lesotho.

These technologies also could facilitate large-scale population health efforts thereby expanding the reach of health professionals in human resource-restricted settings [10, 11, 20, 21]. The prototype of the Nthabi system used similar conversational agent technology and was shown to increase women reporting healthier preconception health among young African American women in a randomized trial in the USA [15]. Our data show that Nthabi has the potential for use as a population health tool at the national or health system level.

Our findings are in concordance with other studies conducted in low–middle-income countries suggesting that health information tools are a feasible method to increase reproductive and sexual health knowledge [2, 9, 22] and have been shown to be feasible and acceptable for improving health education and knowledge among adolescents and young people [23]. Other studies highlighted the broad potential for digital interventions to enhance health promotion and service delivery towards better sexual health [24]. However, this is the first study showing an increase in knowledge resulting from potentially more engaging and effective conversational agent systems in a low- and middle-income country in southern Africa.

Our results show increased sexual and reproductive health knowledge, but our data do not provide evidence that the intervention improves young women’s health-related behaviors or improves health outcomes. Possessing knowledge is not enough to bring about health behavioral change, but the acquisition of knowledge is part of the pathway to modify individual behaviors [25]. The Kirkpatrick model of health education program evaluation defines four levels of evaluating health education programs [26, 27]. Participants’ reaction to the health education content (level 1) allows them to move on to learning (level 2). Once participants have gained knowledge, they are then able to transfer knowledge to behavioral change (level 3). Improved knowledge leads to measurable change in clinical outcomes (level 4).

Our previous research reported that focus group participants (level 1) perceive that Nthabi provides culturally appropriate and non-judgmental conversational dialogue that overcomes the impact of cultural taboos and social stigma related to the use of sexual and reproductive language. It can also overcome the judgmental attitudes of healthcare professionals [16]. We then surveyed clinical trial participants who used the system for 6 weeks to explore acceptability and usability. They perceived Nthabi to be effective, and efficient and helped them make better health decisions. They intended to use Nthabi beyond the study period, and plan to encourage others to use it [17].

In this manuscript, we report that Nthabi improves knowledge of health topics that were shown to improve clinical outcomes in a systematic review [9]. Our data support the feasibility of conducting studies to determine if improved knowledge of SRH from interactions with Nthabi results in improved clinical outcomes (levels 3 and 4). For example, future studies can test if improved knowledge of family planning could translate to improved usage of contraception methods and ultimately reduction of unintended pregnancies that have major health and social implications [28], and remain unacceptably high among adolescent girls and young women in southern Africa [29].

Despite good access to smartphones among young men and women in Lesotho, there remain significant barriers to using m-Health technologies for health education. In Lesotho, 94% of people aged 18–29 use smartphones, and the number of mobile phone connections in 2021 was equivalent to 102.6% of the population (as some people have more than one phone). Seventy-two percent of web traffic is on mobile phones and 95% of web traffic originates from Android Phones [30].

However, as identified in this study, access to smartphones did not guarantee access to Nthabi. The finding that only 8 of 172 (4.7%) participants were able to use Nthabi on their phones demonstrates that mobile phone use is possible, though, practically, only phones with sufficient available memory could be used.

In this study, early in enrollment, it was clear that many phones were unable to download the App. We purchased 20 tablet devices and loaned them to participants unable to download the App to their phones. All tablets were returned at the end of the study. At $111 per tablet or $14 per participant, this could be a cost-effective way to improve health education compared with the cost of increasing human resources. This strategy could be afforded by several donor-funded projects now implemented in Lesotho [31]. Future research to determine the cost-effectiveness of different technologies to deliver Nthabi is required.

The barrier to using phones is in part due to the design of the system. Nthabi was designed to address our concern that limited internet access in Lesotho would impact participant usage. A decision was made to download the full system to mobile devices so that participants could use the system when not in a Wi-Fi environment. While this design allowed the participants to use the system at their convenience, the inclusion of all the content created significant difficulties for downloading and using Nthabi on most phones due to limited available phone memory. As it becomes increasingly possible for young women to have regular access to public Wi-Fi, it will become more feasible for young women to use Nthabi in the cloud on their phones rather than downloading the full system. Future studies of large-scale health education programs in low-resource settings using cloud-based interventions will be increasingly possible with the increased availability of Wi-Fi access.

Our results show that participants chose to interact initially with family planning followed by folic acid and later with healthy eating, but fewer completed. This could be a result of the participant’s interest in the topics, or a reflection of the quality of the adaptation of the healthy eating content as this topic focused on examples of foods that were mostly unavailable or costly in the local context. These issues were not identified in our qualitative research but will be explored and adapted as needed in future studies.

In addition to the limitations discussed above, our study is further limited in several important ways. First, the attrition between the number of participants completing the pre- and post-tests could bias the results. Before disseminating the application more broadly, we must explore reasons why a large number of participants completed pre-tests but did not complete post-tests. Those who did not complete the post-tests may have understood the content less well. If so, this study’s results may overstate the application’s effectiveness in increasing knowledge.

Second, the sample was recruited from only two of the 10 districts of Lesotho and therefore is not a nationally representative sample of women throughout the country.

The sample included participants from the rural districts of Berea and Leribe, however, when recruited some were attending the technical university in the capital city, and these participants reported residing in and receiving health services in these two rural districts. The participants attending the technical university likely are conversant with the correct use of digital tools and therefore the results do not necessarily reflect women living in rural areas. Further trials are needed to more definitively identify the impact of Nthabi on rural women.

Conclusions

The Nthabi conversational agent system increased knowledge of folic acid use and family planning methods among young women in Lesotho. Digital health interventions like Nthabi have the potential as an additional method to deliver reproductive health information in countries that have limited human resources for health. Further research is needed to study the impact of Nthabi on health behaviors and clinical outcomes before large-scale implementations.