Definitions of fairness in machine learning explained through examples

Q: How do the definitions of fairness discussed in Verma and Rubin's paper align with the perceptions of fairness among students, particularly those from diverse backgrounds?

The definitions provide a mathematical framework, but students' perceptions of fairness are shaped by personal experiences and cultural contexts. Engaging with students from diverse backgrounds helps researchers ensure that fairness definitions reflect the realities of all individuals, especially underrepresented groups.

Q: How can student input be incorporated into the development and evaluation of fair machine learning models?

Students can participate through design sessions, surveys and focus groups where they share experiences and concerns about algorithms that affect them. Involving students in user testing and feedback collection provides real-world insights into fairness, helping institutions build more equitable and transparent systems.

By David Griffin

Updated Feb 20, 2026

Algorithms can shape who gets shown an advert, shortlisted for a job, or approved for a loan. When those decisions are automated, you need a definition of fairness you can test, not a vague intuition.

Because these systems are mathematical, it is understandable to assume they are fair and unbiased. In practice, models can inherit bias from the data and choices people make, and the impact can fall hardest on women and minorities, a risk explored further in our post on AI and education equity challenges and opportunities.

A recent paper by Verma and Rubin (2018) explores definitions of fairness for classification in machine learning (ML). In this context, classification refers to labelling or categorising data based on predictions learned from a large training dataset. The authors illustrate common definitions of fairness found in artificial intelligence (AI) and ML literature using a single example: the German Credit Dataset (GCD) (Lichman, 2013). This dataset contains records for 1,000 German loan applicants from the 1990s and includes 20 characteristics such as marital status, age, gender, number of dependents, credit history and occupation. Verma and Rubin use the dataset to show how different fairness definitions behave, with an emphasis on gender bias and discrimination. An overview of their work is provided in the following section.

Definitions of fairness

Fairness is not one thing. Each definition below answers a slightly different question. The key is to choose the definition that fits your use case before you start measuring bias. For an education-focused example, see algorithmic fairness in student performance ML models.

‘Group fairness’, ‘Statistical parity’, ‘Benchmarking’ or ‘Equal acceptance rate’ – This definition of fairness is met if individuals with protected and unprotected characteristics are equally likely to receive a positive classification prediction. Protected characteristics are those that should not be used to discriminate between people, such as gender or disability. In the example of the GCD, this would mean that male and female applicants would have an equal likelihood of receiving a good credit score prediction.

‘Conditional statistical parity’ – This definition of fairness implies that subjects of both protected and unprotected characteristics are equally likely to receive a positive classification prediction when other legitimate factors are taken into account. When using the GCD example, legitimate factors would include credit history and employment. Thus, male and female applicants, given parity in credit history and employment credentials, would be equally likely to be predicted a good credit score.

‘Predictive parity’ or ‘Outcome test’ – This definition of fairness is satisfied if, when the model predicts a positive classification, the chance of that prediction being correct is the same for protected and unprotected groups. Using the GCD example, this would mean that both male and female applicants would actually receive a good credit score when predicted a good credit score. That is to say they have equal positive predicted value (PPV).

‘False positive error rate balance’ or ‘Predictive equality’ – This definition is true of a classifier if subjects with protected and unprotected characteristics are equally likely to receive a positive classification when they should receive a negative one. In the GCD example, this would mean that both male and female applicants with bad credit scores would be equally likely to be predicted a good credit score by the classifier.

‘False negative error rate balance’ or ‘Equal opportunity’ – This definition is the reverse of the last; it is true of a classifier if subjects with protected and unprotected characteristics are equally likely to receive a negative classification when they should receive a positive one. Intuitively, in the GCD example, this would mean that both male and female applicants with good credit scores would be equally likely to be predicted a bad one by the classifier.

‘Equalised odds’, ‘Conditional procedure accuracy equality’ or ‘Disparate mistreatment’ – This definition can be deemed true if both of the previous two definitions are true.

‘Conditional use accuracy equality’ – This definition of fairness holds when deserving subjects with protected or unprotected characteristics are equally likely to receive the correct classification, positive or negative. Using the GCD example, this would suggest that both male and female applicants would be equally likely to receive the appropriate credit score classification.

‘Treatment equality’ – This definition is met by a classifier if subjects with protected and unprotected characteristics have an equal ratio of false positive to false negative classifications. In the GCD example, the ratio of incorrect positive and negative predictions would be the same for male and female applicants.

‘Test fairness’, ‘Calibration’, ‘Matching conditional frequencies’ or ‘Well-calibration’ – This definition of fairness is met if subjects with protected and unprotected characteristics are both equally likely to belong to the positive classification. Within the GCD example, this would mean that both male and female applicants who apply for a loan would be equally likely to actually hold a good credit score.

‘Balance for positive class’ – This definition is met if subjects with protected and unprotected characteristics have, on average, the same predicted probability of being classified positively. In the GCD example, this would mean that both male and female loan applicants with a good credit score would have equal probability of being predicted a good credit score.

‘Balance for negative class’ – This definition of fairness is essentially the opposite of the last. It is met if subjects with protected and unprotected characteristics have, on average, the same probability of being classified as negative. Within the context of the GCD example, this would mean that both male and female loan applicants with a bad credit score would have equal probability of being predicted a bad credit score.

‘Causal discrimination’ – A classifier meets this definition of fairness if all subjects with identical attributes are classed as equal. In the GCD example, this would mean that both male and female loan applicants with identical attributes would both either receive a good or bad predicted credit score.

‘Fairness through unawareness’ – This definition of fairness is similar to the last. It is met if a classifier does not use protected characteristics in its classification. In the context of the GCD example, this would mean that whether a subject was male or female would not be considered when applying for a loan.

Conclusions

Verma and Rubin (2018) provide a concise overview of definitions of fairness in ML classification. It is hoped that this summary will provide readers with an introduction to the many and varied designations of fairness.

Their work also makes clear that measuring fairness is a complex task. The authors stress that applicable definitions will vary with the intended use of each system. They also highlight that while verifiable outcomes are available for training data, it is challenging to ascertain whether real data will follow the same distribution. Consequently, inequality may exist in a system even if it was trained to avoid it. A practical takeaway is to state your fairness definition upfront and be explicit about trade-offs when you report results.

FAQ

Q: How do the definitions of fairness discussed in Verma and Rubin's paper align with the perceptions of fairness among students, particularly those from diverse backgrounds?

A: The definitions of fairness outlined by Verma and Rubin provide a mathematical framework for understanding and addressing bias in machine learning algorithms. However, the alignment of these definitions with students' perceptions of fairness, particularly among diverse groups, may vary significantly. Students' perceptions of fairness are shaped by lived experience, cultural contexts and the social dynamics they observe or are subjected to.

Incorporating student voice into conversations about fairness in algorithms can bridge the gap between formal definitions and real-world impact. By engaging students and valuing their perspectives, researchers and developers can choose and interpret fairness metrics in ways that better reflect the realities of underrepresented groups.

Q: What methodologies or approaches can be utilised in text analysis to identify and mitigate biases in datasets like the German Credit Dataset, especially concerning underrepresented groups?

A: Identifying and mitigating biases in datasets such as the German Credit Dataset requires a multifaceted approach that combines technical methods with an understanding of social contexts. Text analysis can play a crucial role in this process. Techniques such as sentiment analysis, thematic analysis and natural language processing (NLP) (see our student feedback analysis glossary for definitions) can help uncover patterns, biases and underlying themes in the data that may not be immediately apparent. For instance, text analysis can reveal discriminatory language or bias in the descriptions of applicants' profiles.

Incorporating student voice can strengthen the analysis by adding diverse perspectives on what counts as bias and how it shows up in practice. Engaging students from underrepresented groups to review and interpret findings can help validate results. It also helps ensure that the biases identified are meaningful and reflect the experiences of those most affected. This collaborative approach can enhance the rigour of the analysis and promote inclusivity in the development of machine learning models.

Q: How can student input be incorporated into the development and evaluation of fair machine learning models, particularly in applications that impact student life directly, such as admissions or financial aid algorithms?

A: Incorporating student input into the development and evaluation of machine learning models is essential for ensuring the fairness of applications that directly impact student life, like admissions or financial aid algorithms. One effective way to involve students is through participatory design sessions, where students from diverse backgrounds can share their experiences, concerns and suggestions about the algorithms that affect them. These sessions can help identify potential biases and areas where the model might not align with students' perceptions of fairness.

Surveys and focus groups can also be used to gather a broader range of student perspectives. Involving students in the evaluation phase through user testing and feedback collection can provide insights into real-world implications and highlight areas for improvement. By engaging students throughout the process, institutions can develop machine learning models that are more equitable, transparent and responsive to the needs and values of all students. This engagement fosters trust in the technological applications that play a significant role in students' academic and personal lives.

References

[Source Paper] Verma, S., Rubin, J. 2018. Fairness Definitions Explained. 2018 ACM/IEEE International Workshop on Software Fairness (FairWare), 2018, pp. 1–7. DOI: 10.1145/3194770.3194776

[1] Lichman, M. 2013. UCI Machine Learning Repository.
Available from the UCI Machine Learning Repository.

Request a walkthrough

Book a free Student Voice Analytics demo

See all-comment coverage, sector benchmarks, and reporting designed for OfS quality and NSS requirements.

All-comment coverage with HE-tuned taxonomy and sentiment.
Versioned outputs with TEF-ready reporting.
Benchmarks and BI-ready exports for boards and Senate.

Prefer email? info@studentvoice.ai

UK-hosted · No public LLM APIs · Same-day turnaround