Are Robot Writers Really About to Take Over the World?

Science fiction makes it seem like artificial intelligence is going to take control of our lives, but in all likelihood, those apocalyptic fears are blown way out of proportion. The machines won’t take over, Skynet won’t win, and HAL will open the damn pod bay doors. The dystopia might be a lot more subtle—especially for writers. AI won’t take our freedom. It’ll just take our jobs.

Most professions have been progressively taken over by technology in some capacity over the past century, but creatives have been generally immune to the machine takeover. That may change quickly. In the past few years, a number of companies have developed natural language generation (NLG) platforms capable of mimicking the quality of publishable written work you’d expect from a human.

Back in March, The New York Times published an op-ed with the cheeky title “If an Algorithm Wrote This, How Would You Even Know?” to show just how far NLGs have come. For example, Automated Insights, one such service, which has a product named Wordsmith, produced more than 1 billion stories last year for clients including the Associated Press and Yahoo—mostly work like short corporate earnings recaps or fantasy football analysis. The Times piece also includes an interactive quiz that asks readers to choose whether a human or computer wrote eight different passages. Even after spending hours researching NLGs for this article, I still got two wrong.

But does that mean all journalists should find a new line of work? It’s not that simple.

For the most part, these language algorithms operate by taking dense data sets and using certain statistics and parameters to tell a story. Think box scores for sports, or financial reports for business. By nature, the algorithms work using formulas that are extremely complex but somewhat limiting. For now, an algorithm could take hundreds of financial data points and give historical context and identify relevant trends and correlations, but the insights will still lack nuance. While a human can easily bring in context that falls outside of the data set, NLGs can’t look past the numbers to add opinions or research, or to interview someone and incorporate quotations into the text.

In other words, NLGs are extremely powerful tools that still need human oversight.

“There’s data out there, and that data actually means something,” said Kristian Hammond, chief scientist and co-founder of Narrative Science, a technology company that created an NLG named Quill. “If you run the appropriate analysis of it, you can glean facts from the data, which can then participate in a later narrative.”

Narrative Science, which was formed in 2010, has become a major player in the NLG space thanks to Quill. The company has raised more than $30 million in funding in the last few years, and its client roster includes Forbes, Credit Suisse, and Deloitte. In the financial space, there’s been a lot of interest in Quill and its competitors. Forbes, for example, publishes short pieces of investing analysis that are heavy with data. And a company like Credit Suisse, while not publishing Quill’s work on a blog, is still using the algorithm to produce investing reports that help help analysts save time during the workday.

Before Quill became the T-1000 version of an NLG, Hammond and Larry Birnbaum, Narrative Science’s chief scientific advisor, helped create a program called StatsMonkey, which could automatically formulate baseball game recaps, while they were professors at Northwestern University. Once they saw that StatsMonkey worked, they began to think of ways the technology could apply to other industries.

“This has nothing to do with baseball. This has nothing to do with sports. This has nothing to do with media,” Hammond said. “We are now in a world where we have massive, massive data sets available to us. And very few people can actually understand what they mean. So we started to think of ourselves and the technology as a conduit for information.”

Narrative Science’s success comes at a time when data journalism is booming. Publishers like Vox, FiveThirtyEight, and The Upshot (a New York Times blog) have set a high standard for producing statistical analysis that separates itself from traditional reporting. The upside here is crucial: Those who break news may get a few minutes of fame before their stories are recycled by other outlets, but those who produce original research and analysis have material that typically leads to greater recognition and is harder to rip off.

Brands like Zillow, Jawbone, and even PornHub are progressively jumping on the bandwagon as well, using unique internal data to tell interesting, viral stories without the need for hard-hitting journalism that could open up conflicts of interest.

It’s worth pointing out that the reason sites like FiveThirtyEight and Vox have gotten so much respect is that their writers have managed to complement esoteric data analysis with a human voice and perspective. The content is smart but also entertaining and accessible. Brands that are successful with data storytelling follow a similar formula. But in the NLG world, the hope is that the technology will get closer and closer to getting the job done.

“We iterate with the clients with regard to what Quill is producing. From the client perspective, it almost feels like they hired an analyst/writer, and they’re just getting feedback, and the feedback gets fed into an automated system,” Hammond explained. “It has that feel that you’re working with a new writer. But the reality is that new writer just happens to be a computer.”

But while NLGs may seem appealing on the enterprise level for the way they could potentially cut content costs and save time, the situation is a bit more complicated on the writer side. The lazy argument in support of automated content is that game recaps and basic financial reports are so formulaic that they don’t need to be written by humans. But cutting off the lower rungs of the journalism ladder would make it even harder for new talent to break into editorial careers that are already extremely competitive.

And even if algorithms can generate prose that is just as good as—if not better than—human work, that doesn’t mean all readers will back these robot writers. If you saw the byline for this article came from a computer, would you want to read it? And if you decided to read it, would you attribute its quality (or lack thereof) to the publisher or a computer?

Those philosophical questions don’t have easy answers at the moment. “Our technology in general is what we call truth-preserving. The way it’s structured, once things are in place, it can’t lie to you. And that’s actually a reassuring moment,” Hammond countered. “There might be a moment of surprise, but then it turns out, ‘Oh, I actually understand this story.'”

In 2014, researchers from Karlstad University, in Sweden, ran a small study that attempted to assess whether there was any difference in quality between content created by humans and content created by algorithms. The study only relied on 46 undergraduate students—so take this with a little salt—but the results suggested that the NLGs scored higher on trustworthiness and accuracy. To the credit of the human journalists, they scored better in terms of writing skills, but for formulaic recaps, that may not matter. And, tying back to The New York Times quiz, the study found that more than one-third of respondents thought a journalist had written text that had actually been produced by software.

“It’s not writing the next Harry Potter, but is explaining how your portfolio is doing,” Hammond said of Quill. “And it is explaining what the water quality of your beach is like. Or it’s telling you about a little league game. And you don’t have to worry about it getting it wrong or having bias.”

In today’s media world, bias isn’t necessarily a bad thing. Just about every major publisher has adopted some sort of blogging persona that values opinion more than hard reporting in an effort to make the news more compelling. Interestingly, the need to suffocate bias could be most useful in a newer space like content marketing, where brands might have more issues with transparency and honesty than newspapers and magazines. Particularly for companies with small teams that crunch a lot of data in industries like finance and insurance, algorithms could help cut costs and improve accuracy without eliminating jobs.

Regardless of how much NGLs take over in the future, it’s important for publishers to remember why it’s wise for humans to stay involved in the creation process.

“Answers alone don’t make us smarter,” Hammond said. “Answers and communication—that’s actually what we think the future of collaboration with intelligence systems is going to look like.”