AI Snake Oil

AI snake oil is AI that does not and cannot work as advertised.

AI snake oil is AI that does not and cannot work as advertised. Defining AI's Scope. Artificial Intelligence (AI) is a broad term encompassing diverse technologies, from generative models like ChatGPT to predictive algorithms used in finance. It's crucial to distinguish between these different forms of AI, as their capabilities, applications, and potential for failure vary significantly. The Rise of Generative AI. Generative AI, exemplified by chatbots and image generators, has captured public attention with its ability to create realistic content. However, it's essential to recognize that this technology is still immature, unreliable, and prone to misuse, often accompanied by hype and misinformation. The Perils of Predictive AI. Predictive AI, used to forecast future outcomes and guide decision-making in areas like policing, hiring, and healthcare, is often oversold and ineffective. AI snake oil refers to AI that doesn't and can't function as advertised, posing a societal problem that requires critical evaluation and discernment.

Even if AI can make accurate predictions based on past data, we can’t know how good the resulting decisions will be before AI is deployed on a new dataset or in a new setting. Automated Decision-Making. Predictive AI is increasingly used to automate consequential decisions about individuals, often without their knowledge or consent. These systems, employed in areas like healthcare, hiring, and criminal justice, can have profound impacts on people's lives and opportunities. Recurring Shortcomings. Despite claims of accuracy and fairness, predictive AI systems are plagued by recurring shortcomings, including: Making good predictions that lead to bad decisions Incentivizing gaming and strategic manipulation Over-reliance on AI without adequate human oversight Using data from one population to make predictions about another Exacerbating existing inequalities Embracing Unpredictability. The pervasiveness of predictive logic stems from a deep discomfort with randomness. However, accepting the inherent uncertainty in many outcomes can lead to better decisions and institutions, fostering a world genuinely open to the unpredictability of the future.

The same fundamental roadblocks seemed to come up over and over, but since researchers in different disciplines rarely talk to each other, many scientific fields had independently rediscovered these limits. Limits to Prediction. Accurately predicting people's social behavior is not a solvable technology problem, and determining people's life chances on the basis of inherently faulty predictions will always be morally problematic. The challenges are ultimately not about AI, but rather the nature of social processes. The Fragile Families Challenge. The Fragile Families Challenge, a large-scale study that tried to predict children’s outcomes using AI and lots of data, found that the best models were only slightly better than a coin flip, highlighting the difficulty of predicting life outcomes. The Meme Lottery.…