Google Introducing a new AI-powered video generator: Lumiere. This makes it possible to create realistic, diverse and, above all, smooth videos.
In a video, Google shows what's possible with Lumiere. In addition to the usual Text to video function Lumiere should also animate images and Edit existing videos ability. For example, a person in the video can turn into a flower figurine or a real car can become a Lego car by entering text. The clothing of the person in the video can also be swapped. He can too style The video can be changed by entering text – for example from photorealistic to animated.
Global video instead of keyframes
According to Google, all this is possible due to the new approach to the artificial intelligence model. Produces many video generators Key frames Then fill in the gaps. You can imagine this somewhat like an AI image generator. For example, if you type: “Syrup is poured over the pie,” the first image will be created with the jug on top of the pie, the second image with the syrup hitting the pie, and the third image with the syrup, the pie covered and nothing coming out of the pot. Then, based on these generated images, it was done Medium pictures I was born. There is also another one third model, Which upscales the small frames generated into high definition video. This works, but often causes jerky motions or ugly artifacts and parts of the image.
Lumiere also uses a model to increase resolution, but before that video is used Created entirely. This is made possible by a spatiotemporal structure that generates the entire video with a single run of the model.
There is no miracle AI
Currently, Lumiere can produce 80-frame clips at 16 fps. This results in a 5 seconds Long video in resolution 1,024 x 1,024 pixels. The AI model has other limitations. So it's not designed to create videos with multiple camera angles. Transitions between scenes are also not possible. These are problems that need to be addressed in future research.
Lumiere also builds on an existing one Pixel based text to image generator on me. Therefore, a model has yet to be used to scale low-construction frames to higher resolution. The ideal case is that this will also be integrated into just one model in the future.
You can't try Lumiere for yourself at this time. It is also unknown whether Google will make Lumiere available as open source or plans to eventually integrate it into its own AI products, such as a chatbot. cold or twin.
Shortly after the announcement, Google had to face severe criticism for its artificial intelligence tool Gemini. The video that was supposed to show off the AI's capabilities was clearly doctored.
➤ Read more: Google faked Gemini AI video
Lifelong foodaholic. Professional twitter expert. Organizer. Award-winning internet geek. Coffee advocate.