Simplicity is Key

Designer: Geoff Sence

Artificial Intelligence (AI), which includes machine learning and deep learning, is a technique that uses a wide range of computation and statistical power to extract information from large datasets. Depending on the problem, this can then be used for prediction.

The above paragraph may sound impressive and may charm or mislead people into thinking that anyone using these models is somewhat of a genius. With the advancements in AI, anyone can use this model to produce results. You do not need a particular degree or a mere understanding of the technique behind the model to use them.

I put particular emphasis on use and did not say anything about understanding. I can use a car, but that doesn't make me a mechanic. I can use a hair dryer, but that won't qualify me as a hairdresser. I can put a bandage on or help someone cure a skin burn, but most certainly, it won't promote me to a doctor of medicine.

Learning how to use complex techniques is useful, but it also requires some knowledge of when to use them. Would you use a knife to erase a pencil mark? Would you use a chain saw to cut paper? Would you inject anaesthetic into a patient who only requires a checkup?

Understanding these technologies will distinguish users from experts in the coming years. Moreover, you can already tell the difference.

In general, users tend to promote the latest technology for every problem. Experts tend to promote a solution.

Every expert in their domain will tell you that before going to the most complex and unpredictable solution, they try to solve a problem with what is known, more familiar, transparent, and explainable. It can only be achieved when

we focus on solving the problem rather than using a particular solution because it is trendy.

I have been working in data for more than seven years now, and before that, I was doing a doctorate in Astrophysics. Every person I tell this to thinks I used some complex mechanism, and in a way, I did apply complex statistical methods beyond the graduate curriculum. However, they remain explainable. Still, the buildup to my doctorate started with a simple and direct solution.

This philosophy remains true today. Despite the numerous solutions and advanced, and yes, incredible technology, some of those are good for research but have not yet found a good, ethical, and sustainable lifecycle.

Since 2022, we have seen an unprecedented rise in the use of AI, GPUs, and others. Many startups have linked to LLM API models and built businesses to tackle particular problems. While some use cases demonstrated success for AI, and I applaud them for their solution, the same technology faced challenges when adapted to other use cases or even the same use cases but different businesses.

How can this "magical" (I despise using this word for any technology, but I am doing so to match the current perception of AI) technology fail to serve us all? Fail to do our jobs while we enjoy our time by the pool? The answer is simple.

The technology doesn't try to understand the problem to solve it.

Humans have always sought to use the resources around them to solve a particular problem. Similarly, technology was built with the sole purpose of helping humans achieve a specific lifestyle. But let us not forget that someone had to create the technology in the first place.

Robots were designed based on workers. The process of workers in a factory was documented and studied, broken down to the tiniest detail, in order to design and build a machine that replicated the abilities of a human. Expanding that to AI, the technology is aimed at studying human behaviour and intellect to simulate human intelligence.

The problem lies in the process of thinking. AI doesn't think, despite claims saying otherwise. If it did, we would not hear about data theft or automated opt-ins for using data to train AI. AI is like someone who speaks one language, sitting in a room of people conversing in another. AI can't understand anything, but that doesn't mean it can't use logic to infer an outcome. If someone shows the AI that when you see "ABC", type "DEF", the AI may not need to know why, but it won't keep the AI from successfully completing the task, thus giving the appearance of understanding what it is doing.

Regardless of how many tools there are, neither aims to understand the problem. Whenever a user writes a prompt, AI aims to execute it. Any conversation every human pretends to be having with the AI tool is similar but not the same as talking to a person who, instead of listening, wants to answer. Have you ever been in a situation like that? It isn't very pleasant.

Now, while you may get some valuable insights, and I do acknowledge that, certain solutions are too complicated for the problem at hand. Asking an AI to "build me a Churn Prediction model" will not return advice on "data cleaning, exploratory data analysis, data transformation," and so on. It will return a code snippet of a model because that is what you asked.

In my experience, there were occasions when exploratory data analysis on a dataset aimed at a machine learning project led to invaluable results. It was immediately clear that we did not need a model after all. Instead, simple rules sufficed, and the business was more confident about putting in a budget and taking action accordingly.

Now, I am not saying that AI has no value—far from it. I repeat what I shared on LinkedIn: For me, AI in coding would be the most valuable tool as long as it does not rely on the hard work of experts who spend hours thinking outside the box in pursuit of a solution and are not rewarded.

However, using AI can mean going for the most complex solution, whereas a few internal conversations with experts may lead to a simple, easily maintainable, and, let's face it, neat solution.

I conclude this article by adding another advantage of exploring simple solutions: Regulations. Simple solutions will pass with flying colours, whereas pursuing complicated "black box" solutions will mean delays, audits, and costs.