This article is part of a series on the leadership behind the companies shaping the independent publishing industry. To find other articles in the series, click here.
Phil Marshall generated an audiobook of his novel with a single click. That was just the start of the months-long process.
With more than one hundred speaking characters in his debut Hard Sci-Fi novel, Taming the Perilous Skies, the Spoken co-founder had plenty of opportunities to hear the difference between a generated voice saying the right words and a character delivering them convincingly. He spent six to eight months working through the production, wrestling individual passages into the emotional shape he wanted.
“It was just an unbearable amount of work,” he recalls.
Marshall’s experience highlights a question that’s been at the center of AI audiobook production as the technology has developed over the past few years: Can automation take on the work required to create a performance worth publishing? His answer—and Spoken’s approach—considers more than better voice generation. It also takes into account the need to direct scenes, retain an independent listener, and establish how the humans behind licensed voices participate in the business.
Six Characters, Six Accents, One Production Problem
Before Marshall’s hundred-character novel came a much smaller cast. His foray into the world of audiobook creation began with a short story he wrote in less than twenty-four hours at a Taos Toolbox writing workshop in New Mexico. Its six characters, each with a different accent, gathered at a chalet in France before calamity arrived.
Marshall had brought the opening of his novel to the two-week workshop, an annual event led by Nancy Kress and Walter Jon Williams. He remembers the eighteen-person group that gathered for the workshop that year as a source of lasting friendships. The second week also supplied him with the short story that demonstrated a problem he wanted to solve: He could imagine his story’s ensemble performance, but making it was another matter.
“I’m an audio-only reader. And so I wanted to be able to bring that to life. And I'm an AI person, and some of the folks there were very motivating on where the tech was coming, so I started to experiment with that tech and get to know it and I was really floored by it,” he says. “But when I tried to make this six-character story really come to life through AI narration, it was nightmarish.” However, he believed the manuscript contained the information the software would need to understand who was speaking, what distinguished each voice, and how the characters interacted. Extracting that information could take much of the work out of his hands—and make vivid narration with multiple voices accessible to authors for whom the production cost otherwise put it beyond reach.
The impulse was familiar, even if publishing was new territory. Before working among authors and publishers, Marshall had gone to medical school in Indianapolis and moved to Portland, Oregon, for a surgery residency. He later became head of product at WebMD and founded Conversa, a conversational AI company in healthcare. After selling Conversa in 2021, he shifted his focus to finishing his debut science fiction thriller.
In healthcare, Marshall had been drawn to the internet’s potential to give patients more information and control. In publishing, he encountered a constraint through his own creative work and began thinking about a product that might remove it. Both reflected his habit of looking for a system that can solve a common difficulty.
His novel would show him, in the publishing world, how much of that system still needed building.
An AI Voice Still Needs a Scene
The initial Spoken production of Taming the Perilous Skies gave Marshall a base recording. It did not give him the delivery he wanted throughout the book. He described going back through every chapter to adjust individual passages.
“I had to go back through every chapter and wrestle every one of those passages to the ground and get that arc of inflection just right for everyone,” he says. “It was a tremendous amount of work. I mean, it was just an unbearable amount of work.”
A voice can read a series of lines, but its performance may still fail to fit the moment, he acknowledges. The author must decide whether a response arrives too quickly, whether the tone belongs to that character, or whether the scene builds as intended. Marshall’s months of correction show how much directing can remain after the words have been rendered.
“I’m an audio-only reader. And so I wanted to be able to bring that to life.”
—Phil Marshall
Earlier this summer, Spoken released Magic Mode, part of a newer version of its system. Marshall described its development as an effort to coordinate several layers of performance, including accent, pacing, tone, and general delivery. Marshall says the inspiration came, in part, directly from the months of work he spent on his own production, emphasizing the need for “narrative cohesion”: a collection of plausible voices has to become an exchange that makes sense to the listener. Instead of requiring him to shape every passage separately, the newer system was intended to carry the emotional arc across scenes.
At the time of this interview, Marshall planned to regenerate his novel after a further adjustment to the software. His confidence in that forthcoming version was emphatic, though the new production was still ahead of him. He had spent months correcting the earlier version; now he expected the system to generate the voices correctly in a single click. “It’s actually kind of miraculous,” he says.
An Independent Ear
The prospect of better automation prompted a question from Spoken’s own producers. Spoken’s Certified Producer program includes sound engineers, voice actors, and authors. Writers using the service could ask those producers to help complete a project. Their role gave authors access to another listener and someone willing to work through the recording. But if the platform’s Magic Mode could handle the arc of narration across scenes, why would authors still need them?
“I have met very few authors who really like proofing their audio,” he recalls telling them in his answer.
Marshall pointed out that authors would still value an independent ear even if the new system made mistakes less frequently. His own exacting attention to inflection is proof: Producing the file can be done automatically, but deciding that a performance is finished still needs a human’s involvement.
“I wait till I hear it just exactly as I had always heard it in my head.”
—Phil Marshall
He also described a conversation with a producer whose charges reflected the labor required by the older manual mode. Marshall considered those charges fair for that work. With the newer system, he expected the hours involved to fall and thought prices should fall with them. The conversation exposes an important part of the company’s business model—and the hidden complexity behind generating audiobooks. Although authors may save money and time on the initial audio creation, the true savings from automation depend partly on what happens after generation, including the time and expertise required to review the result.
For an author with a backlist, the appeal of delegating that stage is understandable: another person can carry responsibility for listening through a substantial project. Yet whether it’s the author’s responsibility or that of a producer, someone must judge whether the audio conveys what the author intended.
Payment for Licensed Voices
Authors and producers still play a role in Spoken’s audiobook production process, but the platform ensures the work of the voice actors and narrators is respected, too.Spoken uses permission-based models and data, meaning the voice actors included in the platform’s library—roughly 150, at the time of the interview—licensed their voices to the platform. Outside of the library of licensed voices, authors can also use more generic voices produced by the platform for their audiobooks. However, authors who want to use one of the licensed voices for their casting can do so, and the voice actors “get paid for every use,” Marshall says.
Repeatedly, Marshall emphasizes the place that human creativity plays in the process of generating an audiobook with the platform, even as the technology has grown more advanced.He also draws a distinction between the manuscript an author brings and Spoken’s role in turning it into audio: “We take their art, and now we want to bring it to life,” he says.
His stated preference is for human storytelling, particularly its capacity for unusual and original ideas. “The thing about AI is that it can only work from what it's trained on. It can only tell you what's likely or what's possible based on what has happened before. So when it comes to truly unique and novel and innovative, maybe groundbreaking things, that's story, and that's human,” he says. “I want to help those stories that come from humans come to life.” That preference should not be mistaken for a restriction on the platform: he also says Spoken can narrate an AI-generated story. His argument concerns what he values in the work and how he wants to help present it.
For all his enthusiasm about Spoken’s generative software, Marshall’s own audiobook plans included a human-narrated duet edition of his novel alongside the Spoken version. Marshall said the female narrator for his duet edition had previously narrated a story that profoundly influenced his thinking about literature and science: Ted Chiang’s Story of Your Life. As someone who encountered books through listening, he associated that work with the voice that had first delivered it to him.
“I was so fortunate. Oh, I just had to pinch myself,” he says.
For Marshall, the magic of that moment isn’t limited to human narration; he feels the same joy when an interesting or relatable character is brought to life in an audiobook no matter how the audio file is created. “I love hearing it in audio—generating those unique voices for every character. I wait till I hear it just exactly as I had always heard it in my head,” he says. He hopes Spoken can help authors find the same delight in their own books.
A Product Founder’s Unfinished Measure of Success
Marshall describes himself as a product-oriented chief executive. He said that learning to understand his own working style—and to work with people whose strengths complemented his—had become an important part of running companies. Although Spoken’s team originally tried an in-office arrangement—a majority of the company lives in Portland, Oregon—the company has shifted to working remotely since then.
He wants colleagues to understand what drives one another, including what makes them worried or hopeful. For someone intensely focused on the product he wants to build, that awareness helps explain decisions and keep the team working toward the same experience for authors. He recognizes that other CEOs may lead through finance or marketing; he looks for people with strengths he lacks.
“I think as long as you're doing things for the right reasons and everybody understands personalities, and we're all working with each other to maximize everybody's potential, I think that's the recipe to success,” he says.
Asked to look ahead, he laughed at first. “It's funny. Maybe a year ago, somebody said, ‘What do you think it'll be like in five years?’ I said, ‘Let me tell you what I can do today that I couldn't do six months ago. And you want me to tell you about in five years?’ We just delivered the first vivid multi-cast, full-audiobook experience, where everything's tied together, and it's cohesive, and it works, and it's emotively accurate—and I wasn't able to do any of that six months ago,” he says.
Ultimately, he says he imagines future storytelling becoming shorter, more richly voiced, and increasingly visual, with audio eventually opening a path to visual versions of a story. At the center of it still, he says, is the human element. “I just keep coming back to the fact that authors bring their words for us, and so to me, groundbreaking, innovative, interesting, unique, compelling story is inherently human.”
“I think as long as you're doing things for the right reasons and … we're all working with each other to maximize everybody's potential, I think that's the recipe to success.”
—Phil Marshall
The more immediate question is demanding enough. Six characters in a chalet first exposed the difficulty of building an ensemble performance. More than one hundred speaking characters then revealed how much work could hide behind a single click. Marshall’s response has been to build toward a fuller production process, with scene-level direction, licensed performers, and people who can help finish the audio. His own revisions add the author and editor to that picture. However quickly a recording can be generated, the standard he keeps returning to is recognizable to any storyteller: hearing the work come through as intended.
“Just use the tech to bring that story to life,” he says.
