If you imagine something like how the crew of the enterprise talk to the holodeck and create games via prompt engineering (e.g. in one episode Geordi and Data tell the computer "create an adventure game in the style of the novels of Sherlock Holmes") - what would those drawbacks be? And why would that not be noteworthy?
Obviously it would be in a VR environment at best. And rather limited in flexibility at first. And balancing and game design would not become considerably easier. And yes, you'd still need grit and the luxury to at least work part time on it without getting paid. IMHO good games are already drowning in human sweatshop made slop that is controlled by publishers who have the capital.
I've become disillusioned with such approaches because coders tend to loose themselves in the aesthetics of ideas and architectures. And that can lead to high verbosity for something that could be much simpler, which then makes code harder to read and browse. Basically I doubt that complex software problem can actually be planned and designed without programming them first. That could well be just my own limits.
So today I prefer a bit of a "return to monke" C-style procedural approach and don't even like OOP anymore.
I do wonder if there is a more formal framework for natural language (AI prompt) programming. Maybe LLMs could be used as an actual research tool to explore a high number of different architectures of well defined programs, and then compare them. Like have the same app programmed in hundreds of different languages, designs and idioms, and then compare them to get the answer to your question.