ChatGPT’s Storytelling Chops Are No Match for Dungeons & Dragons

ChatGPT’s Storytelling Chops Are No Match for Dungeons & Dragons
Written by Techbot

Our overeager party—an elvish druid; a dwarven wizard; a halfling rogue; and a human paladin—has arrived at a dusty, cluttered library. Hearing of our quest for the fabled Orb of Zarekath, the head librarian—Thimblewick, a gnome—recounts how it was once “a powerful artifact” that has long since disappeared in the nearby ruined city.

But the rogue is less curious about Orb-lore and more interested in snooping and stealing from the nearby shelves. Sneaking into the shadows, he’s caught by a librarian.

“Oh, sorry,” the rogue says with a disarming smile. “I was just looking for a quiet place to read. I must have gotten turned around.”

Then comes a break in the action—a disruption in the very fabric of space-time—as we, the players of this game of Dungeons & Dragons, find ourselves correcting the almighty dungeon master, or DM.

Players: Thanks for all this description, but in the future, it would be great if the players could describe the actions of the characters.

Dungeon Master: Sure, I understand. As an AI language model, I am programmed to respond to your requests and help guide the story based on the information you provide. So, feel free to describe your characters’ actions and make decisions for them as you see fit. I will do my best to provide responses and keep the story engaging and interesting.

In D&D, the DM is the chief storyteller and referee who builds the world and interprets the many rolls, roles, and rules of the game. But what the characters say or do is the job of the players. When your DM is an AI and it steps on your dice-rollin’, character-building toes, it feels good to talk back to the machine: “I’m sorry, ChatGPT, I’m afraid you can’t do that.

Sadly, having to dictate a metafictional correction dispels the illusion. We are not adventurers, but plain boring humans, playing a tabletop game in our mundane, non-magical universe. Perhaps we ought to be more patient. Our particular DM has never played D&D, and it’s not even human. We’ve asked ChatGPT to play what we’re calling “ChatD&D.”

Some context: We are two Gen-X gamers who’ve been playing D&D since the Jimmy Carter Administration. ChatD&D was an attempt to see if computer-driven, role-playing experiences had improved since the days of classic text-based games like Zork and Colossal Cave Adventure, which merely guided players through pre-generated scripts.

Considering that AI is already being tested in creative realms—generating perfunctory tourist itineraries and podcasts, writing college essays and novellas, and rendering images in a range of styles—christening a ChatDM felt like the next logical step. It also seemed to offer a limitless gaming experience. What was unclear, though, was whether a computer could match a human’s capacity for the harder work of gaming: being innovative, adapting to unexpected situations, and actually interacting with players. The Writers Guild of America strike was called partly in reaction to fears about AI replacing human creativity in the realm of storytelling, but could AI also be coming for dungeon masters’ jobs? ChatD&D was a chance to find out.

The experiment began quite simply: We popped open a few cans of Mountain Dew, put some Hawkwind and Blue Oyster Cult on the turntable, grabbed our bags of polyhedral dice, and asked ChatGPT, essentially, “Shall we play a game?” Initially, ChatDM was reluctant to play: “I’m not capable of serving as a Dungeon Master for a game of Dungeons and Dragons.” Much like convincing your older sister to run a game back in the day, the bot required some coaxing at first. Then, knowing you can trick ChatGPT into writing porn or teaching you to build a bomb by asking sideways, we shifted tactics: “We would like you to first describe a fantasy world where our adventure will take place.” Bingo.

ChatDM immediately showed talent for verbosity. In moments, it churned out broad plot lines, adventure settings, and an entire mystical land for us, complete with place-names, lore, and backstory.

Unfortunately, Fantasy World 1.0 was no fresh take on the genre; rather, it came off as generic. “In this world, magic is real” and there are “powerful wizards and sorcerers”; “humans, elves, dwarves, and halflings”; “dragons, trolls, and giants”; “sprawling forests, rugged mountains, vast deserts”; and also “sprawling cities, bustling towns …” Yawn. Drawing from narrow ideas of gender, race, geography, and culture, ChatDM’s taste for fantasy was often a bland amalgam of fantasy scenarios harvested from decades of D&D lore and Tolkienesque tropes sown across the internet, chewed, processed and re-extruded for our consumption.

Unlike the insular world of the Omegaverse, finding a space between innovating and remaining true to the form can be tricky in D&D. We prodded for a more specific location, and ChatDM cheerily complied: “Sure! How about I describe a ruined city that could serve as a setting for your adventure?” Better. (One way to know ChatGPT isn’t human: It takes feedback so easily.)

When urged to name the city, ChatDM offered: “The name of the city could be something like ‘Zarekath,’ which sounds suitably ancient and evokes a sense of mystery and intrigue.” ChatDM understood that a place-name’s faux-exotic mouthfeel was all part of the game. Additional details emerged from the blurry generic shapes. A pattern of play began to form: ChatDM would quickly sketch too much of a story; we’d ask for intervening steps or additional information; and it would happily oblige, backing up and filling in details. Only through interaction—with us, the players—was it able to focus on moving the story forward in smaller, iterative chunks punctuated by decision points or actions.

When the rogue tried to sneak into the library, ChatDM “knew” the rules well enough to determine that a roll of “10” on a 20-sided die is a failure. But getting the bot to understand the back-and-forth necessary to role-play combat took a lot of work. It easily lost track of details, locations, and geography, especially in more complicated scenarios, like in the thick of battle:

ChatDM: That concludes the first round of combat. What will your characters do in response?

Players: Wait, we were supposed to go after Challenger Number 3. And you never told us whether the burly half-orc Challenger Number 3 actually hit.

ChatDM: My apologies for the confusion. I must have misspoken earlier. As for Challenger Number 3, the burly half-orc swings his axe at Pelthor and misses, leaving an opening for the group to act.

It also tended to summarize how the melee played out, omitting any exciting attacks, spells, multiple dice rolls, or saving throws. Of course, that faster-paced, minimalist action, more like a novel or movie than a step-by-step battle simulation of play, might appeal to those intimidated by the game’s granular minutiae.

Ultimately, ChatDM’s performance, and its struggle to maintain a consistent story, should assuage any fears human DMs may have about AI taking their place at the D&D table. Similarly, game shops shouldn’t plan on welcoming their new chatbot overlords anytime soon. Put plainly: AI sucks at this. But experimenting with ChatGPT in this way also provided new insight into how games like D&D work on a fundamental level, not as a system of rules for simulation, but rather as a shared platform for interaction.

Like all collaborative storytelling and participatory media, the ChatD&D experience reminded us that a good D&D adventure isn’t like being told a story by a novelist or storyteller. The narrative unfolds communally around a table, with plenty of backtracking, retconning, and joking—and avoiding the small-minded hobgoblins of consistency and rules-lawyering. That has always been the core gaming experience. When gaps, inconsistencies, or contradictions occur, the group—human to human—devises creative solutions.

People can settle for ChatGPT’s intrinsic lack of originality, or they can use the technology to challenge them to subvert the cliches. Ironically, given the high level of interaction required to guide the DM (and the DM’s willingness to be guided), ChatD&D may in fact represent a more free-form version of D&D, truer to the game’s improv-oriented roots, where no one, not even the DM, has a clue as to where the adventure will go.

Original Article:

About the author