Any media article referring to Artificial Intelligence (AI) will inevitably mention ‘human bias’ somewhere. Bias is the big taboo in this fast-emerging field. Indeed, many of our counterparts out in industry have stated that ‘fear of rogue algorithms’ is one of the primary concerns for holding off on AI projects. This is a legitimate concern. If you are planning to release an algorithm into the public domain, you will want to check whether your algorithm hates minorities or holds gender values from the 1930s.
Is AI truly biased?
Considering the problems and views some trained algorithms hold, there is a prevailing problem in how the ‘AI bias’ problem is being articulated. It is not an AI problem at all. It is a human problem, which is transmitted to machines. A possible re-articulation of the problem is to treat AI bias as a manifestation of our own inconsistency. Training an algorithm is like holding up a statistical mirror to our own flawed processes and logic.
In this light, AI is a fantastic highlighter of bias. It can be trained to quantify how bias impacts on our businesses and our societies. It allows us to evaluate our own perspectives, ideologies and behaviours from a distance and from a neutral perspective.
There is an argument to be made that organisations who deploy AI are significantly more likely to discover their biases, thereby allowing them to address the underlying reasons why bias may exist. For instance, one of our first projects involved automating literacy assessments for a client. We analysed hundreds of literacy samples in order to allow the client to automate this process.
It quickly emerged that the English teachers had pre-defined opinions about what someone of a certain age should be able to do. In one specific example, when a 16-year-old wrote a 100-word literacy sample, it was considered plentiful, whereas when a 34-year-old student wrote the same amount, the sample was considered short. We discovered this when we fed the age of students into the predictive model, and prediction accuracies improved substantially.
Factoring age into a learning assessment may seem quite normal, and appropriate. After all, students are divided into age groups at school. The scary thing for the client was that this literacy assessment was designed to be universal, and age was not supposed to factor into the overall score. It was just a natural bias that every educator had hardcoded into their teaching styles.
This discovery prompted some fantastic internal discussions about the ethics of differentiating on age. Ultimately, the 16-year-old and the 34-year-old were both being supported into similar college courses. It was agreed that the educators should remark the training samples without knowing the age of the learner, and to use this new data to train the literacy assessment tool and to evaluate the merits of differentiating support based on age.
Why do organisations rollout biased AI?
The mistake many organisations make is to treat predictive algorithms as finished, off-the-shelf products, without necessarily thinking about how to manage these algorithms over the longer term. A famous example of this was the PredPol algorithm, which is used by police forces to predict where crime is likely to happen within an area. Despite being a powerful tool in predicting where crimes would happen, PredPol experienced serious teething issues associated with what data the algorithm could or should use to make predictions.
The algorithm would absorb the racial biases of reports produced by police officers, leading to a greater police presence in areas of cities that contained minority groups. This issue was compounded by the fact that the output from the algorithm was essentially a self-fulfilling prophecy because arrests would go up wherever it decided to send police officers (police can only arrest people where they are sent). This unhealthy feedback loop reinforced the idea that these areas needed more policing and the cycle continued. Needless to say, this led to some really bad PR for the police departments involved.
A step by step approach
Here are some top tips for implementing AI without suffering from the headache of bias:
1. Begin with internal processes
We have found that the most profound discoveries in AI come from analysing internal business processes. Aside from sheltering your clients from the quirks of your early AI models, AI can be an effective compliance, auditing and quality assurance tool.
2. Learn from unsuccessful training attempts
Sometimes, the seemingly simplest processes fail to achieve any kind of meaningful accuracy, however, rather than treating this as a disaster, it can reveal two key insights:
- Blind spots in your data. There may be weaknesses in the information you track and value.
- Inconsistent processes. There may be something fundamentally disorganised about how current decisions are made. Analysing the underlying logic of how AI attempted to solve a problem can prove just as valuable to a business as the model itself.
3. Test AI through a pilot / onboarding phase
To adjust the famous phrase, no dataset survives first contact with reality. AI projects involve a great deal of self-discovery. This can be treated as a huge selling point of AI, and the lessons learnt can have a lasting impact on your business. Ensure that you factor this learning time into any future AI project.
4. Ensure there is auditability
To quote Mr Weasley from Harry Potter, “never trust anything that can think for itself if you can’t see where it keeps its brain.”
- If you can’t work out how a decision was made, you can’t learn from your mistakes. Therefore, ensure that you begin any AI project with solid answers about how you can interrogate the output of a model.
- Get to know your biases. This Wikipedia article will keep any data scientist up at night, however, developing an understanding of the kinds of bias that your team is susceptible to can be a valuable exercise.
5. Understand the difference between opinion and bias
We humans use profiling and pattern recognition all the time, and most of the time, it is a powerful tool for forming reasoned opinions. Unfortunately, there is a fine line between useful gut instinct and bias. Having mechanisms auditing how decisions are made can lead to profound changes in the way teams operate.
6. Be wary of treating any algorithm as a finished product
Great AI is something you interact with on an ongoing basis. Humans should play a critical role in auditing the decisions that AI makes, critiquing the outcomes that are generated. Every predictive algorithm comes with strengths and weaknesses. It is important to have the tools in place to audit decision making on a continuous basis.
At Fluence, we believe in transparent, auditable AI. If you’d like to explore the potential of AI in your organisation, we offer free consultations. Arrange one today by contacting us on firstname.lastname@example.org or call 0121 638 0760.