A game-based approach to learn about fairness in Machine Learning

Game-based approaches offer a range of valuable features for engineering ethics education such as:

  • Virtual spaces to experiment and observe consequences without impact in the real world ;
  • Virtually infinite opportunities to practice ethical skills, in particular ethical reasoning and decision-making, in a low-stakes context ;
  • Experiences with different roles and different points of view ;
  • Engaging ways to grapple with complex knowledge.

A game by students for students

Based on an original idea by two Master students at our institution, Ester Simkova and Alexandre Pinazza, we have developed an interactive narrative game in which players play the role of a data scientist who faces design dilemmas while developing machine learning models. The decisions that players make change the course of the story and lead to different ethical consequences.

– no background in machine learning or in ethics is necessary,
– the game can be used freely for educational purposes,
– it does not collect any data.

Specific features for learning ethics

Our game is specifically designed as an educational tool by implementing a series of well documented ethics education strategies.

The narrative includes two applications of machine learning technology, based on real case studies of algorithmic bias: criminal recidivism prediction (inspired by the COMPAS case) and automated translation (inspired by the Google Translate case).

In each “mini-case study”, players are guided to analyse their decisions before committing to a choice, they get a chance to reflect after observing the consequences of their decisions, and they are led to identify their emotional reactions at specific points in the unfolding story.

Interactive debriefing session

The game creates a shared experience for a group of students, which can then be used to start a discussion about ethics in class. We have developed a short debriefing session that:

  • Makes explicit the ethical dilemmas in the game ;
  • Underlines the responsibility of ML developers as well as ways in which they can act on these ethical issues ;
  • Introduces simplified strategies to identify ethical issues while developing ML systems and has students practice them on an example case (itself a real system developed by researchers).