AI replaces 'Commodity work'

I have been posting a lot to address the narrative around 'AI is replacing Data Science' and whether you said it, or you're a data scientist or worse, masquerading as one; this is for you.

Sharing my knowledge and experience with data and AI thrills me, as I know firsthand how it can change the way businesses plan strategies that lead to success and revenue. However, individuals using AI are now building scripts using prompts that mimic data science workflows. What used to take a couple of lines, now takes an essay. The feeling of being pushed out as a Data Scientist is felt by many. Individuals use AI, and all of a sudden, everyone is a data expert. Worse still, these individuals brag about finding 'a robust solution' for ten minutes (true story!) only to find out it was a simple solution that beginners learn in Python 101. Some claim to have built a great model only to fail with new data, or they outright tell you they can't explain what is going on, "but look, the Accuracy is remarkable". Well, as long as the painting is fresh, we can ignore the broken foundations.

If you're in this situation, it is imperative to differentiate between AI and the individuals using AI. AI only enhances the individuals' skills, or lack there-of.

Needless to say, this leads to burnout because non-experts tend to be loud while experts usually believe that their work will speak for them and for the majority, that is true. However, mental burnout leads to overthinking, overworking, and, personally, I enrolled in more courses than there are hours in a day.

Yet no matter how many courses I do, the learning I gain is limited unless it describes governance, and there is a lot still to learn. I took courses on AI workflows and agents, and the only thing I learned was how to 'describe an action I want the AI to do', or how to organize folder structure depending on the scope of that folder so the AI knows where to look for. That's software engineering and unfortunately, data scientists were pushed to learn this when the industry mistook data science for engineering. I dare say it was more about choosing the right words, almost a language course rather than anything complex. This may excite individuals because anyone can ask someone else to do something, but that doesn't mean you accomplished it. That is what is happening today - vibe prompting is promoted as skills and expertise.

Of course, no one wants to masquerade as someone having unpopular yet common skills. Obviously, the roles experiencing this push-out are the ones deemed to have rare skills. Such as, mathematical understanding that everyone has the opportunity to learn, but as statistics show, very few comprehend it, and even fewer apply it. Yet this shortage of skill is now narrated as irrelevant to AI; the whole Mathematics curriculum is now in jeopardy! Or so they want to convince you.

So let me tell you what I enjoy as an Astrophysicist working in Data.

I enjoy solving problems that seem impossible to solve. The thrill of using mathematics and statistics to solve complex problems in such a way that they are explainable and stable. I enjoyed every single project where the task was "spend some time to see if we can do something about it". No guidance except blue sky thinking. Whenever someone tells me "it can't be done" or "no one will care," that's a challenge I happily take on.

Mostly, I enjoy equations!

In the universe of equations, there is a world of solutions. Individuals prompt LLMs, and for the life of me I don't understand why, they give the LLM access to production data without understanding fundamentals of data and basic mathematics and theorems such as 'Central Limit Theorem'. Does everyone need to know the details? If a person is masquerading as a data expert, then yes. You guessed right that this article is a rant. I am tired of hearing the same thing, "AI is replacing Data Science", or wannabes bullying data teams on social media or otherwise. The role of Data Science was often mistaken for Engineering for a long time due to a lack of understanding of Science. But the fact remains that Data Science is the application of scientific techniques to data, and AI is helping the role come into itself. As soon as someone says 'black box' in reference to Machine Learning, that tells you they're not experts.

As these individuals masquerading as data scientists do, I prompted Claude the following:

I have a doctorate in astrophysics and been working in data science since 2017. Is AI capable of taking my job? Don’t be considerate about feelings or emotions. Use logic

The output was about determining whether the role I settled into in data science was commodity work. Upon further prompting, it generated that the median data scientist, the one who does commodity data science and runs models on structured data, is 'vulnerable within a decade'. The competitive advantage is to 'continue operating at the intersection of deep scientific reasoning and data work'. It also generated

The threat isn't replacement today — it's gradual irrelevance if you don't stay positioned above the automation line.

Vibe-prompting modelling is the automation line. When you vibe prompt, the LLM searches its data, which is sourced from books or publicly available materials. Any publicly available model is a default, and anything default can be automated at scale.

To my fellow scientists, I urge you to take a holiday and avoid distractions from the noise. Vibe-Prompting Data Science will be self-defeating as data experts continue to develop niche approaches to solve real-world problems.

As a take away, Vibe-prompters complain about data quality and the time it takes to try and clean it. I bet you some are even prompting LLMs to clean their data. Experts take it on as a challenge, work with it and drive influential insights with ethical confidence.

Rant Over!