新湖畔网 （随信APP) | 叫板Sora！Adobe推出了能够让P视频变得超神的AI工具

最近AI视频领域异常火爆，Adobe也刚刚加入了这场竞争。

在今年的Adobe Max大会上，Adobe正式推出了自己的AI视频模型——Firefly Video Model，进军生成式人工智能领域。

Adobe的首席产品营销经理Meagan Keane表示，Firefly Video Model的推出旨在简化加速视频创作，并增加视频的故事讲述能力。概括来讲，这个模型包含三个有趣且实用的功能：

• 生成拓展
• 文生视频
• 图生视频

其中，生成拓展已经集成至Adobe所开发的专业视频编辑软件Premiere Pro中；文生视频和图生视频则刚刚在Firefly Web app中推出beta版本。

顾名思义，如果你拍摄的视频片段太短或缺失某些部分，生成拓展可以帮你在视频片段的开头或结尾进行"生成式扩展"。

也就是说，假如所需的镜头不可用，或者某个镜头的剪辑时机过早或过晚，剪辑师只需拖动该视频片段的开头或结尾，生成拓展就能自动填充生成式内容，以保持视频的连贯、平滑。

这意味着如果视频创作者在拍摄过程中得到了一个"废镜头"，他可能不需要重新拍摄，而是可以利用生成拓展对原片段进行扩展和修复。

一定程度上说，这有助于提高视频创作效率，当然是在生成拓展所延伸的片段质量过硬的情况下。

遗憾的是，该功能目前的最大分辨率限制为1080p，且片段最长只能扩展两秒钟。

因此生成拓展只适合对视频片段进行微小调整，帮助用户修复某些细节，但不能代替创作者进行大篇幅的创作。

生成拓展还可以帮助用户在拍摄过程中进行调整，纠正在拍摄过程中视线偏移或其他意外产生的移动。

除了视频之外，生成拓展还可以用于音频的平滑编辑。它可以将视频中的环境背景音效扩展长达十秒钟时间，但无法对对话或音乐进行拓展。

如果你想在视频创作过程中省点力气，可以使用文生视频功能直接生成。用户只需输入想要生成的视频的文本描述，它就能模拟各种风格生成相应的视频片段。

以下是一些使用文生视频功能生成的视频片段：

...

值得一提的是，用文生视频和图生视频功能生成的视频片段最长仅为五秒钟，质量最高为720p和每秒24帧。视频生成时间约为90秒，Adobe正在开发"涡轮模式"来减短这一时间。

视频生成质量的不足表明，用户还不能用Firefly Video Model生成完整的电影，只能作为创作辅助工具。

Adobe强调，Firefly Video Model只会在许可内容和公共领域内容上对该模型进行训练，而不会在客户提供的内容上进行。

此外，使用Firefly Video Model创建或编辑的视频可以内嵌内容凭证，以帮助声明AI使用情况和所有权权益，保证商业安全。

感兴趣的读者可以在以下链接加入Adobe Firefly Video Model的体验候补名单：
https://www.adobe.com/products/firefly/features/ai-video-generator.html

Recently, the AI video field has been buzzing, and Adobe has just joined the fray.

At this year's Adobe Max conference, Adobe officially launched its own AI video model - Firefly Video Model, entering the generative artificial intelligence field.

Adobe's Chief Product Marketing Manager Meagan Keane said that the launch of Firefly Video Model aims to simplify and accelerate video creation, while enhancing the storytelling capabilities of videos. In summary, this model includes three interesting and practical features:

• Generative Extend
• Text-to-Video
• Image-to-Video

Generative Extend has already been integrated into Adobe's professional video editing software Premiere Pro; Text-to-Video and Image-to-Video have just been released in beta version on the Firefly Web app.

As the name suggests, if your video clip is too short or missing certain parts, Generative Extend can help you "generatively extend" the beginning or end of the video clip.

This means that if a required shot is unavailable, or if a shot's editing timing is too early or too late, editors only need to drag the beginning or end of the video clip, and Generative Extend can automatically fill in generative content to maintain video continuity and smoothness.

This means that if a video creator gets a "bad shot" during filming, they may not need to go back to the original location to reshoot, but instead use Generative Extend to directly extend and repair the original clip.

To a certain extent, this does help improve video creation efficiency - of course, only if the quality of the extended segment by Generative Extend is solid.

Unfortunately, the current maximum resolution limit for this feature is 1080p, and the clip can only be extended by two seconds at most.

Therefore, Generative Extend is only suitable for making minor adjustments to video clips, meaning it can help users fix certain details, but cannot replace the creator for extensive creation.

Generative Extend can also help users adjust during the filming process, correcting eye shifts or other unexpected movements that occur while filming.

Aside from videos, Generative Extend can also be used for smooth audio editing. It can extend environmental background sound effects in videos for up to ten seconds, but cannot extend dialogues or music.

If you want to save some effort in the video creation process, you can directly generate videos using the Text-to-Video feature. Similar to its "predecessors," Runway and OpenAI's Sora, users only need to input their desired video's text description, and it can simulate various styles such as "real movie," "3D animation," and "stop motion animation" to generate corresponding video clips.

Here are some video clips generated using the Text-to-Video feature, take a look:

In addition to realistic, cinematic visual images, Text-to-Video also supports generating some "abstract" visuals. For example, it can be used to generate videos with elements such as flames, water, lens flares, and smoke, which can be overlaid onto existing videos to enhance visual depth and interest in existing content.

It is worth mentioning that video segments generated by Text-to-Video can also be further refined with a series of "camera controls," which can simulate real camera angles, movements, and shooting distances.

The Image-to-Video function goes even further, allowing users to add "reference images" during the video creation process to generate videos that align more closely with their imagination.

Video creators can start from an image or photo and use the Image-to-Video feature to directly create "B-roll" (a term in film and video production referring to auxiliary video material apart from the main footage - A-roll).

Users can also upload a single frame of a video and have Image-to-Video automatically fill in missing frames, creating complete close-up shots.

"As everyone knows, video cannot be faked, so it must be real."

However, the "cyber reshoot" feature of Image-to-Video turns this statement into past tense directly. It's like a "God's hand" that allows users to directly manipulate and modify video content (such as character movements) in the video. Take a look at the examples below:

Currently, the maximum length of video segments generated by Text-to-Video and Image-to-Video features is only five seconds, with a maximum quality of 720p and 24 frames per second. Video generation takes about 90 seconds, and Adobe is developing a "Turbo Mode" to shorten this time.

The shortcomings in video generation quality indicate that users still cannot create a complete movie using the Firefly Video Model, as it can only be used as a creative tool.

Adobe emphasizes that Firefly Video Model will only be trained on licensed content (such as Adobe Stock) and public domain content, and will not be trained on user-provided content.

In addition, videos created or edited using Firefly Video Model can embed content credentials, aiming to provide attribution proof for creators and content sources, helping to declare AI usage and ownership rights to ensure "business security."

Interested readers can join the Adobe Firefly Video Model experience waitlist at the following link:
https://www.adobe.com/products/firefly/features/ai-video-generator.html