I would throw the same challege back to you with a Poweshell script written by a programmer...no one would ever introduce a script into production without fully tesing in a sandbox.
...no one that values their career at least.
What AI accompishes is stremlining the development layer before it gets to test. How long would it take a programmer to write those thousand lines of code?
AI can do it in seconds.
So, in my position, none of this is a hypothetical scenario, to be clear. I am just sharing what I know from being part of various AI focus groups, working with talented engineers and ultimately speaking from the perspective of a technical stakeholder.
As stated, these are best viewed as productivity tools. I agree with you there. What I am trying to convey is that the nuances of the real world are a lot more complex than that.
You shouldn't just generate 1,000 lines of code, test it in a lower environment and walk away. That's not what a responsible person who wants to keep their job would do anyways. You need to know what's in the code, review it, analyze potential security exploits, ensure the execution routine has correct permissions in all environments, inspect the code for re-usability, assess the code's scalability, ensure the code is readable (by a human), assess the libraries used, ensure there are not wayward lines of code that simply shouldn't be there. I could list all kinds of things that are required to validate machine-written code.
Of all those things, many could even be automated to some degree. Now, how do you know which things should be automated? What do you do when these automated tests fail? Do you trust the automation? What's needed to model the automation? How do you go about triaging inevitable defects? How do you recover if things go south in production? So many questions.
Lastly, let's talk about the equilibrium of features and competition in lieu of AI. One might assume that productivity gains leads to decreased headcount. In the short term, that's possibly even true for many companies, but also, what productivity gains lead to is a higher rate of features, more feature parity, higher competition, pressure for innovation. There's less time spent on monotonous tasks and more time spent on innovative tasks and value drivers. Company B, meanwhile, has to do twice as much to keep up with Company A. Company C doesn't decrease headcount but multiplies output with performance gains. Now companies A and B need to catch C.
In a similar lens, this may not be all that dissimilar to the industrial revolution. Jobs did not decrease during the industrial revolution. They changed, for sure, but ultimately employment increased in lieu of automation. Ultimately, though, anyone who says the know for sure is lying, but this is where I stand.