AI chatbots need more books to learn from. These libraries are opening their stacks
By MATT O’BRIEN
CAMBRIDGE, Mass. (AP) — Everything ever said on the internet was just the start of teaching artificial intelligence about humanity. Tech companies are now tapping into an older repository of knowledge: the library stacks.
Nearly one million books published as early as the 15th century — and in 254 languages — are part of a Harvard University collection being released to AI researchers Thursday. Also coming soon are troves of old newspapers and government documents held by Boston’s public library.
Cracking open the vaults to centuries-old tomes could be a data bonanza for tech companies battling lawsuits from living novelists, visual artists and others whose creative works have been scooped up without their consent to train AI chatbots.
“It is a prudent decision to start with public domain data because that’s less controversial right now than content that’s still under copyright,” said Burton Davis, a deputy general counsel at Microsoft.
Davis said libraries also hold “significant amounts of interesting cultural, historical and language data” that’s missing from the past few decades of online commentary that AI chatbots have mostly learned from. Fears of running out of data have also led AI developers to turn to “synthetic” data, made by the chatbots themselves and of a lower quality.
