Last April, a New York startup called Runway AI unveiled technology that allows people to create videos, such as a cow at a birthday party or a dog talking on a smartphone, simply by typing a sentence into a box on a computer screen.
The four-second videos were blurry, choppy, distorted, and annoying. But it was a clear sign that AI technologies will generate increasingly compelling videos in the coming months and years.
Just ten months later, San Francisco startup OpenAI unveiled a similar system that creates videos that look like they were taken from a Hollywood movie. The demonstration included short videos – captured in minutes – of woolly mammoths trotting across a snowy meadow, a monster staring at a melting candle, and a scene on a Tokyo street apparently captured by a camera swooping across the city.
OpenAI, the company behind chatbot ChatGPT and still image generator DALL-E, is among many companies racing to improve this type of instant video generator, including startups like Runway and tech giants like Google and Meta, owner of Facebook and Instagram. . This technology can speed up the work of experienced filmmakers, while completely replacing less experienced digital artists.
It can also become a quick and inexpensive way to create misinformation online, making it more difficult to know what's real online.
“I'm absolutely terrified that this kind of thing will impact a narrowly contested election,” said Oren Etzioni, a University of Washington professor who specializes in artificial intelligence. He is also the founder of True Media, a non-profit organization that works to identify online misinformation in political campaigns.
OpenAI calls its new system Sora, after the Japanese word for sky. The team behind the technology, including researchers Tim Brooks and Bill Peebles, chose the name because it “evokes the idea of limitless creative potential.”
In an interview, they also said that the company has not yet released Sora to the public because it is still working to understand the risks of the system. Instead, OpenAI is sharing the technology with a small group of academics and other outside researchers who will “red team,” a term that refers to looking for ways it could be abused.
“The intent here is to give a glimpse of what's on the horizon, so people can see the capabilities of this technology, and we can get feedback,” Dr. Brooks said.
OpenAI already places watermarks on system-produced videos that identify them as AI-generated, but the company admits they can be removed. It may also be difficult to detect. (The New York Times added “Generated by AI” watermarks to videos containing this story.)
The system is an example of generative AI, which can generate text, images and sounds on the spot. Like other generative AI technologies, the OpenAI system learns by analyzing digital data — in this case, videos and captions describing what those videos contain.
OpenAI declined to say how many videos the system learned or where it came from, except to say that the training included publicly available videos and videos licensed from copyright holders. The company says little about the data used to train its technology, likely because it wants to maintain an edge over competitors — and it has been sued several times for using copyrighted material.
(The New York Times sued OpenAI and its partner, Microsoft, in December, alleging copyright infringement on news content related to AI systems.)
Sora creates videos in response to short descriptions, such as “a gorgeous paper-made world of coral reefs, full of colorful fish and sea creatures.” Although videos can be impressive, they are not always perfect and may include strange and illogical images. For example, the system recently generated a video of someone eating a cookie — but the cookie never got smaller.
DALL-E, Midjourney, and other still image generators have improved so rapidly over the past few years that they now produce images that are almost indistinguishable from photographs. This has made it more difficult to identify misinformation online, and many digital artists complain that it has made it difficult for them to find work.
“We all laughed in 2022 when Midjourney first came out and said, 'Oh, that's cute,'” said Reid Southin, the film's Michigan-based concept artist. “Now people are losing their jobs to Midjourney.”
