Alibaba has introduced the Wan2.6 series, the latest iteration of its visual generation models. This new release offers advanced features for creators, enabling them to appear in AI-generated videos using their own likeness and voice.
These upgrades include flexible multi-shot storytelling, designed to enhance professional-grade content production with capabilities for extended narratives and multi-person dialogue.
Introduction of Reference-to-Video Model
The Wan2.6 series launches a novel reference-to-video generation model, Wan2.6-R2V, alongside updates to four existing models. Users can now upload a character reference video that captures both appearance and voice, allowing vivid new scenes to be generated using text prompts.
This feature supports video creation with various subjects, such as people, animals, or objects, while retaining the original reference's unique visual and auditory characteristics.
Power of Multimodal Reference Capabilities
Wan2.6-R2V revolutionizes short-form drama show by enabling users to integrate themselves
As China's first reference-to-video model with multimodal reference generation, Wan2.6-R2V revolutionizes short-form drama production by enabling users to integrate themselves or other subjects into AI-generated videos seamlessly. This approach enhances storytelling and optimizes the production process.
In addition to R2V, the series includes advancements in its text-to-video model (Wan2.6-T2V), image-to-video model (Wan2.6-I2V), and two image generation models (Wan2.6-image and Wan2.6-T2I).
Innovative Multi-Shot Storytelling
The new models incorporate advanced multi-shot storytelling techniques, allowing for more engaging narratives with consistent visual quality. Improved synchronization between audio and video components results in more lifelike scenes coupled with enhanced sound effects.
With support for up to 15-second video outputs, these upgrades provide creators ample space to craft their stories, leveraging precise instruction-following and high visual fidelity to achieve cinematic-quality content.
Enhanced Image Generation Capabilities
In image generation, the Wan2.6 series enables the creation of interleaved text-image outputs
In image generation, the Wan2.6 series enables the creation of interleaved text-image outputs, bolstered by advanced logical reasoning to promote clear visual storytelling.
The series excels in producing realistic portraits with precise artistic style control. It also advances in understanding complex Chinese and English text prompts, allowing for the creation of expressive visual content that highlights nuance and artistic expression.
Access Through Alibaba Cloud's Platform
These models can be accessed and deployed via Model Studio, part of Alibaba Cloud's AI development platform, and are also available on Wan’s official website. Integration into the Qwen App, Alibaba’s prime AI solution, further extends the series' reach.
Since its initial unveiling earlier this year, the Wan series has consistently evolved, underscoring Alibaba's commitment to innovation in AI-driven multimedia technology.
Alibaba now unveiled the latest evolution of its visual generation models, the Wan2.6 series.
It enables creators to appear in AI-generated videos as themselves and in their own voices with flexible multi-shot storytelling – new features designed to unlock creative possibilities for professional-grade content production with enhanced multi-person dialog and extended duration for richer narratives.
new reference-to-video generation model
The Wan2.6 series features a new reference-to-video generation model as well as comprehensive upgrades to its four existing models. Wan2.6-R2V enables users to upload a character reference video with both appearance and voice, utilising text prompts to generate vivid new scenes starring that same character.
Users can create videos featuring a person, animal or object, or even multiple subjects together, while preserving the distinctive look and sound of the original reference.
Multimodal reference generation capabilities
Powered by multimodal reference generation capabilities, Wan2.6-R2V is China’s first reference-to-video generation model that makes it possible for users to insert themselves or other subjects into AI-generated scenes with consistent visuals and audio. This changes the way short-form drama creators tell stories and streamlines their production process.
The Wan2.6 series also includes enhancements to its text-to-video model (Wan2.6-T2V), its image-to-video model (Wan2.6-I2V), and to its two image generation models (Wan2.6-image and Wan2.6-T2I).
Intelligent multi-shot storytelling capabilities
The new models introduce intelligent multi-shot storytelling capabilities that allow for richer, more expressive narratives with visual consistency throughout. Its improved capabilities in audio-visual synchronization and audio-to-video generation deliver more realistic scenes with richer sound effects.
Supporting video outputs of up to 15 seconds, the models give creators more room to develop their stories. Combined with enhanced instruction-following precision and improved visual quality, they enable creators to produce cinematic-style content with professional-grade results.
Advanced logical reasoning capabilities
For image generation, the Wan2.6 series enables users to create interleaved text-image output with advanced logical reasoning capabilities, to further support coherent visual storytelling. It also demonstrates outstanding capabilities in precise artistic style control, generating realistic portraits with remarkable fidelity and image editing.
Advanced understanding of lengthy Chinese and English text prompts enables creators to produce high-quality, expressive visual content that captures nuance and artistic intent.
Alibaba Cloud's AI development platform
Users can access and deploy the models through Model Studio - Alibaba Cloud's AI development platform - and Wan’s official website. The models will also be integrated into Qwen App, Alibaba's flagship AI application.
First unveiled earlier this year, the Wan series has undergone continuous upgrades, reflecting Alibaba’s leadership and innovation in AI-driven multimedia technologies.
