A major revolutionary breakthrough in AI video generation!OpenAI In Sora, launching today, just a few textual ideas can be transformed into vivid videos.
Whether you want classic black and white to animated styles, everything is possible.
Come and discover what Sora is and what makes it so powerful!
What is Sora?
Sora is a new video generation model released by OpenAI on 2/15. It is a text-to-video (text-to-video) model.
You can use simple text commands (Prompt) and pictures to generate videos of various styles, such as black and white, animation, and real-life HD 1080p videos.Currently it can be up to one minute.
What's so great about Sora?
Sora is built through the research and data of past DALL·E and GPT models, so compared to AI video generation on the market, Sora can more accurately generate text commands in videos.
When you give a text command, Sora also understands how these characters and objects exist in the real world.
For example, if you ask it to draw a person dancing, Sora will know how to draw the correct posture of the person dancing.
Sora can also change several scenes in a movie, but keep the characters and style in these scenes consistent.
Just like when we watch a movie, the story will change from one scene to another, and the story will be coherent.
The characters are still those characters, retaining character and visual style accurately.
Sounds awesome, right? In the article and so on, I will also list three powerful features of Sora to help you understand it easier.
What are the shortcomings of Sora?
Sora is still under development, so it may not be able to accurately simulate some more complex scenarios or some causal relationships.
For example, a person bites a biscuit, but there are no bite marks on the biscuit. This is a cause-and-effect identification error.
In addition, there may be errors in identifying things on the timeline. For example, if something happens over time, he may not be able to successfully produce it.
For example, in the video below, Sora fails to model the chair as a rigid object, so you can see that his physical interactions are a bit weird.
Three key points let you simply understand the features of Sora
After studying the introductions of domestic and foreign media, I compiled three key points to help you simply understand the characteristics of Sora. Let us continue reading!
Key Point 1: Be able to understand the way the physical world exists and operates, and predict its actions
Is there anything special about this? It’s easier to understand if I use an example:
As can be seen from NVIDIA research scientist Jim Fan's Tweet, his only command was "a realistic close-up video of two pirate ships fighting each other while sailing in a cup of coffee."
You can see that the way Sora produces videos is based on understanding the way objects exist and operate:
- Objects are sailing and avoiding each other's path
- Hydrodynamics of coffee and foam around the boat
If you think OpenAI Sora is a creative toy like DALLE, ... think again. Sora is a data-driven physics engine. It is a simulation of many worlds, real or fantastical. The simulator learns intricate rendering, "intuitive" physics, long-horizon reasoning, and semantic grounding, all… pic.twitter.com/pRuiXhUqYR
— Jim Fan (@DrJimFan) February 15, 2024
Point 2: You can expand the video forward or backward in time
Sora is also able to expand the film forward or backward in time, what does that mean?
Can refer toofficial video, Sora can generate additional content for the original video to enhance the narrative or visual storyline, providing more sources of creative ideas in the future!
Point 3: You can turn DALL·E’s pictures into animations
In addition to using text commands to generate videos, Sora can also use images and videos as commands.
For example, the picture on the left below is a monster illustration in a graphic design style. After being converted by Sora, it can be turned into animation, as shown on the right.
Looks awesome! I break out in a cold sweat just looking at these examples XD
How is it different from other AI video generation? What is the difference between Runway and Pika?
Continuing with the above process of generating animations from pictures, there are now many AI generation tools that can do it, such as Runway and Pika. I have come across these two tools quite a bit before, and I would like to share with you the differences between them:
Runway
Runway was launched in 2018. With its second-generation update, Runway introduced director mode and custom camera controls, providing more granular control over camera movement, zoom levels and motion combinations.
It supports high-resolution video export up to 4K, which can support up to 18 seconds.
Pika
Mainly operated through Discord, the movie effects produced by his videos are famous, and many people use it to make "fake movie trailers". Compared with Runway, Pika is more focused on capturing delicate human movements and emotions, but it lacks sophisticated camera and motion control and can only produce 3 seconds at a time.
How is Sora different?
Based on the above, you can see that you can create realistic and imaginative scenes with simple text commands and image commands.
His abilities far exceed even those of his competitors.
Sora can not only generate videos up to 60 seconds long, it even has complex camera angles and can depict human movements and emotions in detail. It makes us think, it is really a major revolution in the rules of the industry.
Is Sora open? Who can use it now?
Sora development is not yet complete, and is now only open for use by a few people, such as visual designers, artists, or filmmakers, to try it out and get helpful feedback from them.
It is not yet fully open to the general public, so you can wait a little longer!
Will videos made using Sora cause copyright infringement?
You may be worried: Will the content of the video produced by Sora be infringing?
Researcher Peeble said that Sora's current training materials are all from their authorized content and public content.
However, when it is opened in the future, users can take other animators' videos and give instructions to ask Sora to make videos. This type of content may not be subject to stricter regulations until it is launched in the future.
How will Sora improve in the future?
according to OpenAI officialThey said they are now working with red teamers to test whether there are misinformation, hateful content, bias, etc., to understand the flaws of the Sora model.
What is a Red teamer?
Red team members are simply cybersecurity professionals who help companies improve their IT security framework by attacking and compromising these same frameworks. For more details, please refer to:What is a red teamer? All you need to know
In addition, OpenAI is also working internally to build a detection classifier to help determine whether the video was generated by Sora. It will also use a text detector to check whether the user's text input commands comply with the regulations. Text input commands such as violence, sexual content, hateful images, celebrity portraits or other people's intellectual property rights will be rejected.
In the future, if OpenAI plans to import Sora into their other models, they will also plan to interpret the data C2PA Metadata Add it to the video to help everyone more easily verify sources and related information.
------------
After reading the above information, I am still digesting it XD
At the end of the article, I will post a few videos generated by people on Twitter, which make people feel really excited and nervous!
I had expected that this world of AI-generated videos would come, but I didn’t expect it to be so soon haha!
Let us continue to look forward to the time when Sora is opened in the future~
Introducing Sora, our text-to-video model.
— OpenAI (@OpenAI) February 15, 2024
Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions. https://t.co/7j2JN27M3W
Prompt: “Beautiful, snowy… pic.twitter.com/ruTEWn87vf
Just enhancing some of OpenAI's Sora generations with some AI music from Splash 😉 pic.twitter.com/spqlmHxkVt
— AmliArt (@amli_art) February 15, 2024
If you're not paying attention to OpenAI's Sora, you're making a big mistake (https://t.co/9znskh7FzX).
— griff (@griffinbeels) February 15, 2024
Whether you like it or not, when fully released, it WILL revolutionize filmmaking and content creation.
Let me tell you why these examples are groundbreaking: pic.twitter.com/GZPzfVXuK7
https://t.co/uCuhUPv51N pic.twitter.com/nej4TIwgaP
— Sam Altman (@sama) February 15, 2024
https://t.co/rmk9zI0oqO pic.twitter.com/WanFKOzdIw
— Sam Altman (@sama) February 15, 2024
You may be interested in these articles...
Is the full launch coming true? Prophetic AI launches AI dreaming instrument for lucid dreaming
