Stories from AI at the workplace
2d 9h ago by lemmy.world/u/madeindex in MadeInDex
A very very senior game developer recently told me how he is changing specialization because AI is much better than him at what he does already.
I asked him if it was just the speed and mass of code, to which he replied that Claude 5 Fable/Mythos is just so advanced, it comes up with solutions he would never have thought of, not even requiring good prompts anymore.
He further said in some areas like Unity it's not perfect yet, but improving drastically.
Honestly this is pretty different to what I read in many articles regarding AI code.
They mentioned errors, bad performance & such.
However if it is already used in production and he as a professional evaluating it, is saying it's better than him...
That same day I also talked to somebody that got hired to train an LLM. His interview was with that companies chatbot, that offered him over 120$ per hour, a roughly 500% increase from his previous job 🤣
I dunno, maybe they aren't that senior then? Just because it came up with a different solution doesn't mean it's the right or appropriate one. I've seen LLMs output very clever code for the wrong situations entirely.
People have also been really impressed by simply functional code, but the architecture it was written for was completely incorrect. It barely took the existing code and structure into account. It was also my job to review AI code written by models just weeks ago. It had been prompted and reviewed by senior devs and I couldn't stop finding extremely questionable code e.g hundreds of database requests in a function that was called in a loop. They had beefy dev machines which were OK with that but the production environment had the power of 200€ mini PC and limited RAM.
With code like that solutions like "let's scale vertically and horizontally" actually become costly. The cloud bill has been stradly increasing and jumped 30% in one quarter. I'm afraid of what will happen if they let the AI loose on the cloud and give it full access via Terraform. Their bill will probably be integer multiples not just floats.
Yes I have read that multiple times and also found the code by normal chatbots e.g. chatgpt to be inefficient and long, however when I brought the performance issue up, he said this is a query problem and the code he is getting from the latest Claude versions is efficient and cloud ready.
Pretty senior, team lead, in his 50s. Also testing/using AI for a while, so not just somebody hyped.
He did say that it isn't great at Unity, as there seems to have been insufficient training.
So... two things can be true. The AI can be writing terribly dangerous code full of subtle bugs, at the same time as being (as far as your friend notices) a better coder than your friend. (Sorry.)
AI remixes all the code publicly available to humanity. Doing this remix introduces subtle (and sometimes obvious) errors.
The quality of this code (before the subtle errors from remixing) tends to be right smack around the middle level quality of all (public) code, in the world.
Considering that public code often goes through several rounds of expert review before being published - there's no shame that code is better than what your friend writes (on a random Monday, before peer review). The AI code was stolen from great coders working on great teams. I also don't usually individually write better code than great public teams produce. (And we tend to be our own worst critics.)
And considering that the AI's sole purpose is to create an attractive looking remix, there's also not much shame that your friend didn't spot the errors it makes.
Spotting the errors made by AI is difficult.
It's also now probably the most important part of the job of a programmer.
(The most important job of a programmer, previously, was knowing which examples not to copy and paste from Stack Overflow.)
Edit: Anyway, your friend is probably underestimating themself. AI is bringing new completely unqualified people into the developer talent pool, daily. It's a shame to hear about someone who actually thoughtfully judges the quality of their own code giving it up.
Interesting take, much of this I have not considered before and can certainly see your logic. The team code vs individual code might certainly offer the explanation. He is now moving into software architecture he said. As AI is not really creative, it might be much easier to compete with it - in that field.
The nature of LLM being trained on all available data and being based on statistical inference means that if it were actually a perfect system, it would be exactly of average (50th percentile) skill level. That would mean that for half the people, LLMs would be better at the job.
Of course it's not a perfect system so it's not even close to 50th percentile, however it's probably still going to be better than a solid bunch of devs. I had multiple people at previous jobs before LLMs that were a net negative on productivity.
Doesn't this assume, that it is giving all training data the same importance? I'm pretty sure they rate the quality of the samples and give it different value. An example for this theory is the data by human professionals, hired to train AI.
Yes, I also read several articles stating that many developers felt they were being more productive using AI, while data showed, they reduced in productivity. Wonder how much Claude really changed since then, I was told: that it improved dramatically with the past 2 updates.
I think generative coding has a massive wealth of methods and code samples to pull from, but it still requires minding to stay in task, remain consistent, and not introduce security issues.
The devs I have talked to say their job is more minding the bots and interpreting their code into the project and less writing the code themselves
Yeah, this guy is also going into software architecture, likely for that reason. However he did say with the recent model the query doesn't even have to be great anymore to get high quality results.