The Harry Potter book series is being used to make AI forget certain information, including copyright infringement.
JK Rowling introduced the world to her magical universe more than two decades ago, but now her books are reviving their relevance in another literature – science. More and more researchers are using Harry Potter texts in experiments with artificial intelligence technology and referencing them in a whole series of scientific studies and articles.
One recent example is the article “Who is Harry Potter?”, which describes technology that helps large language models “forget” certain information, including problematic and copyrighted content (the use of such training data has already led to a number of lawsuits against II developers). The authors of the article, Microsoft researchers Mark Russynovych and Ronen Eldan, tried to remove information about the existence of the Harry Potter books (including characters and plots) from the open code of the Meta Llama 2-7B language model.
They developed a special technique that involves three steps to eliminate data from the model: first, the model is trained on target data (in this case, the Harry Potter books) to identify associated tokens and compare predictions to the base model; replaced unique expressions from Harry Potter with common counterparts and produced alternative predictions close to the model without such training; And eventually fine-tuned the base model to these alternative predictions, effectively erasing the original text from memory when the context suggests it.
Attention to Rowling’s books is primarily due to the great popularity of her characters:
“Anyone can come up with clues for a model to test if she knows the books. Even those who haven’t read the books know the elements of the plot and characters,” says Russynovych, Chief Technical Officer of Microsoft Azure.
Rowling’s work has also been used to test the intelligence of artificial intelligence systems, such as those that spawned the ChatGPT chatbot. Terrence Sejnowski, director of the Computational Neurobiology Laboratory at the Salk Institute for Biological Studies, argued in the paper that chatbots simply reflect the intelligence and biases of their users, like the Mirror of Enelage in the first Harry Potter book showing human desires.
Currently on arXiv, an open access research repository, recent articles list titles such as Machine Learning for Hogwarts Potion Development, Large Language Models Meet Harry Potter, Spell Detection in Fantasy Literature, and more.
Source: Bloomberg
