On the ground: A data professional's warning exposes the hidden dangers of LLMs

OpenAI has just released GPT-5, its biggest and best model to date. Microsoft, OpenAI, Google, and all of the other AI builders out there keep saying AI is going to make us more productive, but it’s not that simple or cleancut; kids at school use it to cheat on assignments, and people at work use it in ways they shouldn’t.

In a recent blog post that came up on Hacker News, data professional Jordan Goodman penned a post explaining that he sees colleagues “blatantly skipping over the fundamentals” due to LLMs in his role as a data analyst.

Worryingly, Goodman notes that some of his colleagues are trusting SQL queries outputted by the AI without being able to explain or debug the query. This over-reliance on AI could lead to incorrect output or code that’s harder to manage or debug because the person who wrote it didn’t develop it and get a good understanding of it.

Goodman said he worries that organizations are being oversold on the idea of AI that it will "magically make up for a lack of proficiency.” While LLMs can certainly get you to an answer faster, it’s still important for professionals to be able to understand what’s going on, especially if trying to pass outputs off as their own work.

Aside from the very real risk of vibe-coded messes being generated, there is a worry that automation could lead to a loss of human skill and oversight, which could lead to serious, undiscoverable errors in professional work. Goodman jokes that a new industry could emerge which might clean up vibe-coded code, but it’s not too far-fetched, after all, businesses like banks hire people who know old programming languages like Fortran to keep applications maintained.

Goodman’s central point is that professionals need to understand what the AI is outputting to be able to explain or debug the work. He argues that professionals have a responsibility to ensure the quality and accuracy of their work, regardless of the tools they use, including AI.

For anyone out there thinking of getting into a programming career and wondering whether there will still be jobs for coders given the rise of AI, the experiences Goodman has had point strongly to the fact that it"s still a valuable field to pursue. While AI is definitely a helpful assistant, it doesn’t seem good enough to hand over the reins to entirely.

Image via Depositphotos.com

Report a problem with article
Next Article

Latest leak spills potential details about the upcoming Google Pixel Watch 4

Previous Article

Here's how hackers can trick Windows Hello into thinking it's you and break into your PC