Understanding Data Bias When Using AI or ML Models

Richard Godfrey July 22, 2024

Collected at: https://datafloq.com/read/understanding-data-bias-when-using-ai-or-ml-models/

Artificial Intelligence (AI) and Machine Learning (ML) are more than just trending topics, they’ve been influencing our daily interactions for many years now. AI is already deeply embedded in our digital lives and these technologies are not about creating a futuristic world but enhancing our current one. When wielded correctly AI makes businesses more efficient, drives better decision making and creates more personalised customer experiences.

At the core of any AI system is data. This data trains AI, helping to make more informed decisions. However, as the saying goes, “garbage in, garbage out”, which is a good reminder of the implications of biased data in general, and why it is important to recognise this from an AI and ML perspective.

Don’t get me wrong, using AI tools to process large amounts of data can uncover insights not immediately apparent, guiding decisions and identifying workflow inefficiencies or repetitive tasks, recommending automation where it is beneficial, resulting in better decisions and more streamlined operations. 

But the consequences of data bias can have significant ramifications for any business that relies on data to inform decision making. These range from the ethical issues associated with perpetuating systemic inequalities to the cost and commercial risks of distorted business insights that could mislead decision-making.

Ethics

The most commonly discussed aspect of data bias pertains to its ethical and social implications. For instance, an AI hiring tool trained on historical data might perpetuate historical biases, favouring candidates from a specific gender, race, or socio-economic background. Similarly, credit scoring algorithms that rely on biased datasets could unjustly favour or penalise certain demographic groups, leading to unfair practices and potential legal repercussions.

Impact on business decisions and profitability

From a business perspective, biased data can lead to misguided strategies and financial losses. Consider a retail company that uses AI to analyse customer purchasing patterns. If their dataset primarily includes transactions from urban, high-income areas, the AI model might inaccurately predict the preferences of customers in rural or lower-income regions. This misalignment can lead to poor inventory decisions, ineffective marketing strategies, and ultimately, lost sales and revenue.

Another example is targeted advertising. If an AI model is trained on skewed user interaction data, it might conclude that certain products are unpopular, leading to reduced advertising efforts for those products. However, the lack of interaction could be due to the product being under-promoted initially, not a lack of interest. This cycle can cause potentially profitable products to be overlooked.

Accidental bias

Bias in datasets can often be accidental, stemming from seemingly innocuous decisions or oversights. For instance, a company developing a voice recognition system collects voice samples from its predominantly young, urban-based employees. While unintentional, this sampling method introduces a bias towards a specific age group and possibly a certain accent or speech pattern. When deployed, the system might struggle to accurately recognise voices from older demographics or different regions, limiting its effectiveness and market appeal.

Consider a business that collects customer feedback exclusively through its online platform. This method inadvertently biases the dataset towards a tech-savvy demographic, potentially one younger and more digitally inclined. Based on this feedback, the business might make decisions that cater predominantly to this group’s preferences.

This could prove to be acceptable if that is also the demographic that the business should be focusing on, but it could be the case that the demographics from which the data originated do not align with the overall demographic of the customer base. This skew in data can lead to misinformed product development, marketing strategies, and customer service improvements, ultimately impacting the business’s bottom line and restricting market reach.

Ultimately what matters is that organisations understand how their methods for collecting and using data can introduce bias, and that they know who their usage of that data will impact and act accordingly.

AI projects require robust and relevant data

Adequate time spent on data preparation ensures the efficiency and accuracy of AI models. By implementing robust measures to detect, mitigate, and prevent bias, businesses can enhance the reliability and fairness of their data-driven initiatives. In doing so, they not only fulfil their ethical responsibilities but they also unlock new opportunities for innovation, growth, and social impact in an increasingly data-driven world.