Why technology cannot solve algorithmic fairness (2/3): Gaps between how computer scientists and welfare economists define fairness

Note: this is Part 2 of the series summarizing the recent paper by Lee, Floridi, and Singh (2020)

As machine learning algorithms are increasingly used to inform critical decisions across high-impact domains, there has been rising concern that their predictions may unfairly discriminate based on legally protected attributes, such as race and gender. Scholars have responded by introducing numerous mathematical definitions of fairness to test the algorithm. As previously discussed, it is increasingly apparent that fairness cannot be summarized into one equation because it depends on the complex real-life context. 

In a recent paper, Lee, Floridi, and Singh (2020) argue that there is a gap between the fairness metrics used in computer science literature and fairness as defined in ethical philosophy and in welfare economics. This blog post will provide a high-level summary of the paper’s findings on the gaps between the welfare economists’ and computer scientists’ definitions of fairness. The paper also addresses the gap between computer scientists’ and ethical philosophers’ definitions of fairness, which was covered in Part 1 of the series.

Challenge of separating “acceptable” inequalities and biases from the “unacceptable” 

The notion of fairness is based on the egalitarian foundation that humans are fundamentally equal and should be treated equally. How equality should be measured and to what extent it is desirable have been a source of debate in both philosophical ethics from a moral standpoint and welfare economics from a market efficiency standpoint. What are the relevant criteria based on which limited resources should be distributed?

Do we assume a level playing field? Some metrics assume all disparity in a given outcome metric is unacceptable, e.g. all men and women should have the same loan approval rates, which is impractical if they have different income distributions.

Does it depend on what is within the individual’s control? Some emphasise a distinction between the features driven by “effort” vs. “circumstances.” However, in reality, it is difficult to separate out what is within one’s genuine control. A credit market does not exist in a vacuum; while potential borrowers can improve their creditworthiness to a certain extent, e.g. by building employable skills and establishing a responsible payment history, it is difficult to isolate these features from discrimination in other markets and the impact of their personal history. In addition, some circumstances are necessary to take into account in a decision-making process. For example, one may not be in full control of one’s income, but it is a crucial indicator of credit risk. Race and gender may be causally relevant in differential medical diagnosis (e.g. sickle cell anaemia, ovarian cancer).

Does it depend on whether the outcome disparity is explainable and justifiable? Scholars have also proposed that the source of inequality should determine which fairness metric is appropriate for each use case, i.e. whether the outcome disparity is “benign” or due to structural discrimination. However, in reality, there is rarely such a separation. There are many structural and statistical factors that lead the lenders to both over-estimate and under-estimate the risk of black borrowers.

Fleurbaey (2008) cautions that “responsibility-sensitive egalitarianism” in welfare economics could be used to hastily justify inequalities and unfairly chastise the “undeserving poor.” He points out that the idea that people should bear the consequences of their choices is not as simple as it seems; it only makes sense when individuals are put in equal conditions of choice. This is not true in most systems. When one has fewer opportunities than another, one cannot be held fully responsible insofar as one’s choice is more constrained.

Lack of consideration of welfare

By focusing narrowly on the redistribution of the target outcome, the metrics overlook the key considerations of the impact on the stakeholders’ welfare and autonomy. Because of the challenge in quantifying the relevant biases and disentangling them from the outcome of interest, correcting for a bias without a clear understanding of its source carries the risk of not only increasing the inaccuracies of the predictions but also causing more harm than benefit to the under-privileged group.

For example, one of a lender’s objectives is to build the most accurate algorithm possible to predict the outcome. A more accurate credit risk algorithm would lower the aggregate portfolio risk for the lender, enabling more loans to more people who would otherwise not have qualified, giving more people access to credit that is crucial to upward socioeconomic mobility. It would also consider the affordability of a loan to each individual. The objective of the affordability consideration is to minimise the borrower’s financial difficulty, given the adverse effects of unaffordable debt on both the market level (causing instability and a “bubble”) and the borrower level.

While scholars have presented “bias mitigation” techniques as potential solutions to unfair discrimination, in fact, adding fairness constraints and overcorrecting for inequalities and biases may end up harming the groups they intended to protect in the long-term (Liu, Dean, Rolf, Simchowitz, and Hardt, 2018). In the presence of a feedback loop, we need to consider not only providing a resource (a loan) to an applicant in a disadvantaged group, but also what happens as a result of that resource. If the borrower defaults, his/her credit score will decline, precluding the borrower from future loans. Putting in place a fairness constraint to give more loans to underqualified applicants without considering the underlying inequalities results in a worse outcome. It is important to view fairness in the resource distribution - not in isolation - but rather, as a function of long-term objectives in promoting the customer’s financial well-being. From a welfare economic standpoint, a notion of fairness includes a consideration of well-being: from both utilitarian and libertarian perspectives, a fair reward principle maximises the sum total of individual well-being levels while legitimising redistribution that enhances the total outcome of individuals.

This is not necessarily contradictory to the egalitarian perspectives discussed in ethical philosophy. In accordance with the Difference Principle, Rawlsian EOP Max-Min social welfare function should also maximise the welfare of those who are worst-off. A model that results in financial harm of already disadvantaged populations fails to meet the Rawlsian EOP criteria, even if the False Negative Rates are equalised as per the mathematical definition. Without consideration of the long-term impact on welfare, the fairness metrics fail to capture the full extent of ethical dilemma embedded in a model selection process.

Lack of consideration of autonomy 

Ex post inequalities and systematic disadvantages affecting individual autonomy and freedom are also important to consider alongside equalisation of opportunities. It is argued that luck egalitarians have no principled objection to a society in which, on a background of equal opportunities, some end up in poverty or as the slaves of others. From the perspective of freedom, this view is not acceptable. Intervention is necessary when basic autonomy is at stake, and this should be a constraint on the distribution of wellbeing in any circumstances. Egalitarians should be concerned - not only with equality of opportunities - but also with the egalitarian content of the opportunities themselves, with freedom as the leading principle in defining responsibility in social justice. By focusing on equality of opportunities, one may dismiss the differences in preferences as driven by choice and thus irrelevant. However, Fleurbaey argues that the ex post inequalities due to differences in preferences are also a target for intervention on the grounds of improving the range of choices to suit everyone’s preferences. If more women prefer lower-paid positions than men, what is problematic is not only the societal and environmental conditioning that questions whether this is a genuine preference, but also the unfair advantages that becomes attached to these jobs - a differential value of the “menu” of options for women than for men because of their preferences. Considerations of fairness and the associated policy response must operate at the level of the menu, rather than distribution of the outcome, e.g. the jobs. This menu goes beyond the formal options that are available to the individuals in principle and addresses what alternatives are accessible that may be chosen in practice.

Fleurbaey also discusses a concept that is not addressed in algorithmic fairness literature: forgiveness. He argues that the ideal of freedom and autonomy contains the idea of “fresh starts”: in absence of cost to others, it is desirable to give people more freedom and a greater array of choices in the future. This is in conflict with the “unforgiving conception of equality of opportunities” that ties individuals to the consequences of one’s choices. In many countries, lenders are restricted in their access to information about borrowers’ past defaults; for example, many delinquencies are removed from U.S. credit reports after seven years. Forcing a lender to ignore information about past behaviour may reduce the accuracy of its default prediction model, and it may be “unfair” by some definitions by putting those who have made more responsible financial decisions on equal level as those who have not; however, it is also widely accepted practice to ensure that one decision does not have a disproportionate impact of limiting one’s access to credit for good. A more complete coverage of fairness and justice, therefore, should go beyond redistribution of outcome features and consider the impact on individual welfare, autonomy, and freedom.

Fleurbaey critiques the existing notions of responsibility-driven egalitarianism in welfare economics to promote an “autonomy-driven” alternative due to the ethical values that are not captured in the notion of “responsibility.” He argues that for responsibility to be upheld, individuals must have and exercise freedom, and three conditions must be met: 1) a minimum level of autonomy is attained, 2) with a minimum level of variety and quality of options offered, 3) with a minimum decision-making competence. A comprehensive egalitarian theory of justice is not just about equalising opportunities but also about providing adequate opportunities and making them accessible. In algorithmic fairness, this is relevant to two bodies of literature: 1) on explanation, transparency, and accountability and 2) on vulnerability from a contractarian perspective. See the full paper for discussion of these two topics.

Conclusion

The fairness definitions may provide a simple methodology for model developers to incorporate metrics relevant to equalisation of outcomes between groups and individuals, they fail to address the important debates on what fairness means in ethical philosophy and welfare economics. The narrow definition of unfair bias in each of these metrics only provides a partial snapshot of what inequalities and biases are affecting the model and does not consider the long-term and big-picture ethical goals beyond this equalisation.