Machine Learning Bias and Fairness

Machine learning is becoming integral to how the modern world functions, with more and more sectors harnessing the power of algorithms to automate tasks and make decisions. As machine learning models become ingrained within decision-making processes for a range of organisations, the topic of bias in machine learning is an important consideration. The aim for any organisation that deploys machine learning models should be to ensure decisions made by algorithms are fair and free from bias. 

Identifying and resolving machine learning bias is important so that the model outputs can be trusted and seen as fair. It’s linked with considerations around model explainability, the process of a human understanding how a machine learning model made its decision. Machine learning models learn from the data itself, so the trends and patterns it maps and learns aren’t developed directly by a human. If left unmonitored and unchecked, bias in machine learning can occur for a range of different reasons.  

A common cause is that the sample of training data doesn’t accurately represent real world conditions faced by the model once deployed. The model may be overfit to this training data which isn’t representative. Even if the training data is of high quality, it may contain historic bias from wider societal influence which can impact the model.  Once deployed, a biased model may favour groups or become less accurate with specific data subsets. This may lead to decisions which unfairly penalise a specific group of people, which can have serious ramifications in a real world setting. 

This guide explores the topic of machine learning bias, including what it is, how it is detected, and risks of bias in machine learning. 

What is machine learning bias?

Machine learning bias is when a model may favour a specific group or subset of data, and is often caused by non-representative training datasets. A biased model will underperform with a specific subset of data, negatively impacting its accuracy. In a real-life scenario this could mean a model’s output is skewed towards a specific ethnicity, age group, or gender because of unrepresentative training data. The resulting machine learning outputs may be unfair or discriminatory as a result. Fairness in this context is the assumption that a model won’t favour a specific group. 

Bias in machine learning is often caused by non-representative training datasets. If the training data is incomplete or over representative of a certain data grouping, the resulting model may be biased against other, unrepresented groupings. This can happen if the sample of training data doesn’t accurately reflect the real-life deployed environment. 

A key example is machine learning in the healthcare sector, which can be used to screen patient data against known diseases or illness. When performed accurately, models can help speed up interventions by health professionals. However, bias can occur. If the training data used to develop the model mainly includes patient data from a lower age bracket, the model may not be accurate if tasked to identify potential illness in an older patient.  

The historic data itself may also be biased. For example a model trained to screen job applicants might favour male applicants because the majority of historic employees were male. In both cases machine learning bias will impact the model’s accuracy, and in the worst cases might even make decisions that are discriminatory and unfair. As machine learning models replace more and more manual tasks and decision-making processes, decisions need to be scrutinised to ensure no bias. Monitoring for machine learning bias should be an integral part of any organisation’s model governance processes as a result.  

Machine learning models are being deployed to complete a huge array of tasks in a variety of fields. Models are now automating more complex tasks and are being leveraged to make decisions and recommendations. Bias in this decision making process means a model may favour a specific group based on a learned bias. This can have serious ramifications when deployed to make risky decisions with real-life consequences. For example, a biased model might discriminate against a certain group when used to automatically agree loan applications. This is a particularly important consideration in regulated industries where any decisions may be audited or scrutinised.  

What causes bias in machine learning?

Machine learning bias has a range of causes, but usually results from bias within the training data itself. The root cause of the biases in training data can be varied. The most obvious example is training data being an unrepresentative subset of conditions found in a deployed environment. This could be training data which contains a disproportionate amount of one subgroup, or underrepresentation of another. This is referred to as sampling bias, and can be caused by non-randomised sampling of training data. 

Bias can occur in the data itself because of how it was collected, how it was processed or labelled, or the historic origins of the data. The data might even reflect historic biases in the wider society from which the data was collected. For example, representation bias can stem from undiverse training data. Historical bias in the wider society can often impact the accuracy of machine learning models too. This can also be described as human or social bias. It can be difficult to source large arrays of data that aren’t at risk of some form of social bias. 

Human bias can also be present in the data processing phase of the machine learning lifecycle. Supervised machine learning depends on labelled datasets, often processed and prepared by a data scientist or specialist. Bias in this labelling process can cause machine learning bias, whether that’s from the array of data that’s cleaned, the way in which data points are labelled, or the selection of features. 

The main causes of bias in machine learning include:  

• Training data which doesn’t represent real world conditions. 
• Human or societal biases in historic data used to train models. 
• Bias in the process of preparing or labelling data for supervised machine learning. 

The risks of bias in machine learning

Models are decision making tools driven by data, so the assumption is that models are making fair and balanced decisions. However, a degree of bias in machine learning models is common, which can skew outputs. Machine learning is being deployed across more and more sectors, replacing traditional software and processes. Models are increasingly used to automate complex tasks, so biased models can have real world impacts. 

Organisations and individuals expect transparency and fairness when decisions are made, and machine learning is not different. There is sometimes even more scrutiny of machine learning decisions as the process is out of the hands of humans. Bias in machine learning can often have discriminatory or damaging impact on specific groups, so it’s integral that organisations are proactive in understanding the risks. 

The risk of bias in machine learning is a particular consideration for regulated environments. For example machine learning in banking may be used to screen mortgage applicants, automating the initial acceptance or rejection. If the model has bias against a specific subset of applicants, this can have serious ramifications for both the individual and the organisation. In any deployment environment that may involve scrutiny of decisions, any observed bias may cause serious issues. The model may be ineffective and in the worst cases be shown to be actively discriminating. This can lead to the model being pulled from deployment all together, so it’s important bias is actively monitored and planned for. 

Machine learning bias is an important element in gaining trust in model decisions. Perceived bias in model decision-making can impact trust within the organisation and with external service users. Models won’t be utilised to their full potential within an organisation if they are not trusted, especially when informing high risk decisions. Accounting for bias should be an angle that is taken into account when considering a model’s explainability.  

Unchecked machine learning bias can have a serious impact on the accuracy and validity of model decisions. In some cases it can lead to discriminatory decisions which may impact individuals or groups. Different types of machine learning model have many different applications, and all are at risk of a degree of machine learning bias. 

Examples of machine learning bias include: 

• Facial recognition algorithms may be less accurate for certain ethnicities because of the lack of diversity in training data. 
• Natural language processing may be more accurate with a specific dialect or accent, and may not be able to process an accent underrepresented in training data. 
• Racial and gender-related bias can be picked up by the model from human or historical bias in data. 

Detecting and solving bias in machine learning

Machine learning bias can be solved through monitoring and retraining models when bias is detected. Model bias is generally a symptom of bias within the training data, or at least the bias can be traced back to the training phase of the machine learning lifecycle. Processes should be in place at every stage of the model lifecycle to detect bias or model drift. This includes processes for machine learning monitoring after deployment too. 

The model and datasets should be regularly evaluated for signs of bias. This could include an analysis of a training dataset, focusing on the distribution and representation of groups within it. Datasets which aren’t fully representative can be amended and or refined. The model performance should also be evaluated with bias in mind. Testing model performance on separate subsets of the data might reveal the model to be biased or overfitted in light of a specific group. Techniques in cross validation of machine learning models can be leveraged to measure performance on specific subsets of data. The process sees partitioning of data into separate training and testing datasets.  

Bias in machine learning can be resolved by: 

• Setting up a process to actively monitor for biased outputs and anomalous decisions. 
• Encouraging a continuous cycle of detection and optimisation, where detected bias can be resolved. 
• Retrain the model with expanded, representative training data whenever required. 
• Account for bias by reweighting features and tweaking hyperparameters when needed.  

Machine learning deployment for every organisation

Seldon moves machine learning from POC to production to scale, reducing time-to-value so models can get to work up to 85% quicker. In this rapidly changing environment, Seldon can give you the edge you need to supercharge your performance.

With Seldon Deploy, your business can efficiently manage and monitor machine learning, minimise risk, and understand how machine learning models impact decisions and business processes. Meaning you know your team has done its due diligence in creating a more equitable system while boosting performance.

Deploy machine learning in your organisations effectively and efficiently. Talk to our team about machine learning solutions today.