Client: USAID and Digital Frontiers
Dates: September 2022 – April 2023
Country: Mexico, India and Global
As the use of AI in development solutions becomes more prevalent, there is a risk of embedded bias in the design and end-use of tools generating inequitable outcomes across genders.
USAID’s challenge aimed to source innovative approaches to prevent and respond to gender inequity in AI. This project, which is being run under Digital Frontiers, is meant to help decision-makers (such as designers, buyers, sellers, regulators, and users of AI technology) identify and address actual and potential gender biases in artificial AI tools.
Our role
Alongside Athena Infonomics, PIT Policy Lab, and Women in Digital Transformation, we will work with AI tools being used in Guanajuato, Mexico to improve student outcomes as part of a World Bank project.
We will support the Mexican Government to understand bias in these datasets and provide toolkits to address these challenges. Learnings from the process will also be shared in India to better explore the use of these toolkits in different environments.
The state of Guanajuato, Mexico, has a population of over 6 million, of which 51% of inhabitants are women. Its growing economy has seen a successful transition from traditional agriculture-based industries to more complex industries such as information technology and electronic.
Under the current administration (2018-2024), the Ministry of Education in Guanajuato has been working to ensure valid, reliable and up-to-date information about the state of the local education system. To do this, they have been developing and harmonizing databases with information regarding student attainment and overall school performance.
To identify at-risk students in higher education, the Ministry has partnered with the World Bank, within a pioneering initiative called Educational Paths, to create an AI based early alert system. This approach aims to provide at-risk students with support and improve retention and graduation rates.
During the second half of 2022, the Ministry will begin training the algorithm with government data to get preliminary findings on its performance.
Methods and approaches
Itad will support this project by creating awareness about the importance of mainstreaming a gender perspective in Educational Paths’ upcoming phases. This includes specifically accounting for and mitigating potential gender-based bias in both the databases and interventions resulting from the development of the AI early alert system.
We will use a three-pronged strategy to support the Educational Paths team. We will:
- Leverage the AI Fairness 360 toolkit from IBM in order to detect bias
- Develop an Ethical Guide for public officials to complement their work in AI
- Create a Checklist to ensure responsible and equitable deployment of AI systems
This work will be led by an expert group of professionals and consultants with relevant experience in technology development, gender perspective, and public policy.
We will also package the main learnings from the case study and present them in a workshop with stakeholders from the Ministry of Education in the state of Tamilnadu, India to explore the potential for replicability of the tools created and the lessons learnt in the Mexico experience.
Outcomes and impact
By addressing how AI tools can propagate gender bias and suggesting solutions to unfair outcomes that occur as a result of this challenge, we hope to contribute to improved student performance in Guanajuato, Mexico.
We also hope to provide generalized lessons that may be used across development actors to support more equitable use of AI tools across development and humanitarian aid programs.