Skip to content
Alibaba Cloud

Wan v2.6 Reference-to-Video

Wan v2.6 Reference-to-Video is Alibaba Cloud's quality-first reference-to-video model, extracting identity from short clips and rendering new scenes with high-fidelity appearance, voice, and motion preservation at up to 1080p. Your use is subject to Alibaba Cloud's Terms & Privacy Policies.

reference-to-video
Use with AI GatewayView docs
import { experimental_generateVideo as generateVideo } from 'ai';


const result = await generateVideo({
  model: 'alibaba/wan-v2.6-r2v',
  prompt: 'A serene mountain lake at sunrise.'
});
Read docs
OverviewAPIAboutProvidersSimilarFAQ

Getting started

Generate videos with Wan v2.6 Reference-to-Video using the experimental_generateVideo function from AI SDK 6 or later. AI Gateway handles routing and polls until the video is ready.

Install the AI SDK (pnpm add ai dotenv), create an API key from the API Keys page, and set it as AI_GATEWAY_API_KEY in your environment. Full setup is covered in the video generation quickstart.

index.ts
import { experimental_generateVideo as generateVideo } from 'ai';
import fs from 'node:fs';
import 'dotenv/config';


async function main() {
  const result = await generateVideo({
    model: 'alibaba/wan-v2.6-r2v',
    prompt: 'character1 and character2 have a friendly conversation in a cozy cafe',
    inputReferences: [
      'https://example.com/cat.png',
      'https://example.com/dog.png',
    ],
  });


  // Save the generated video
  fs.writeFileSync('output.mp4', result.videos[0].uint8Array);


  console.log('Video saved to output.mp4');
}


main().catch(console.error);

Top-level parameters

Pass references through inputReferences and control the output with the top-level resolution and duration parameters. Wan uses resolution, not aspectRatio.

reference-to-video-top-level.ts
import { experimental_generateVideo as generateVideo } from 'ai';
import fs from 'node:fs';
import 'dotenv/config';


async function main() {
  const result = await generateVideo({
    model: 'alibaba/wan-v2.6-r2v',
    prompt: 'character1 and character2 have a friendly conversation in a cozy cafe',
    inputReferences: [
      'https://example.com/cat.png',
      'https://example.com/dog.png',
    ],
    resolution: '1920x1080',
    duration: 4,
  });


  // Save the generated video
  fs.writeFileSync('output.mp4', result.videos[0].uint8Array);


  console.log('Video saved to output.mp4');
}


main().catch(console.error);
ParameterTypeRequiredDescription
promptstringNoText description of the video to generate. Max 1500 characters.
durationnumberNoVideo length in seconds. 2-10 seconds.
resolutionstringNoResolution ('1280x720', '1920x1080').
aspectRatiostringNoAspect ratio ('16:9', '9:16', '1:1', '4:3', '3:4').
generateAudiobooleanNoGenerate synchronized audio with the video.
inputReferencesArray<string>NoReference images and videos, mapped in order onto character1, character2, and so on. URLs only — a non-URL reference is skipped with a warning, so upload local files to Vercel Blob first. See the Input limits table for supported counts and formats.

Input limits

InputFormatsSourcesMax countMax sizeLimits
Text————Up to 1500 characters
Imagejpeg, jpg, png, bmp, webpurl520 MB≥240px · ≤8000px
Videomp4, movurl3100 MB1-30s
Up to 5 reference inputs total across images and videos.

Provider options

Load every Wan reference-to-video option under providerOptions.alibaba. References themselves go in the top-level inputReferences, where the first entry maps to character1, the second to character2, and so on.

reference-to-video-provider-options.ts
import { experimental_generateVideo as generateVideo } from 'ai';
import fs from 'node:fs';
import 'dotenv/config';


async function main() {
  const result = await generateVideo({
    model: 'alibaba/wan-v2.6-r2v',
    prompt: 'character1 and character2 have a friendly conversation in a cozy cafe',
    inputReferences: [
      'https://example.com/cat.png',
      'https://example.com/dog.png',
    ],
    resolution: '1920x1080',
    duration: 4,
    generateAudio: true,
    providerOptions: {
      alibaba: {
        negativePrompt: 'blurry, low quality',
        shotType: 'single',
        watermark: false,
        pollIntervalMs: 5000,
        pollTimeoutMs: 600000,
      },
    },
  });


  // Save the generated video
  fs.writeFileSync('output.mp4', result.videos[0].uint8Array);


  console.log('Video saved to output.mp4');
}


main().catch(console.error);

Pass Wan-specific options under providerOptions.alibaba in your generateVideo call. References come from the top-level inputReferences; referenceUrls is the legacy fallback.

ParameterTypeRequiredDescription
referenceUrlsstring[]NoArray of URLs to reference images or videos. The first URL maps to character1, the second to character2, and so on. Legacy alternative to the top-level inputReferences, used only when inputReferences is omitted. See the Input limits table for supported counts and formats.
negativePromptstringNoWhat to avoid in the video. Max 500 characters.
shotType'single' | 'multi'No'single' for a continuous shot. 'multi' for multiple camera angles.
audiobooleanNoGenerate audio with the video. Provider-side alias for the top-level generateAudio, which wins when both are set. v2.6 only — v2.7 always generates audio and ignores it with a warning.
watermarkbooleanNoAdd watermark to the video. Defaults to false.
pollIntervalMsnumberNoHow often to check task status. Defaults to 5000.
pollTimeoutMsnumberNoMaximum wait time. Defaults to 600000 (10 minutes).

Reference-to-video vs image-to-video

Reference-to-video uses the top-level inputReferences to show the model what your characters look like, then generates a brand-new scene from your prompt. The reference media never becomes the video content; reference each one in the prompt with character1, character2, and so on (first entry maps to character1).

Image-to-video instead animates the actual image you pass in frameImages or prompt.image. The image you provide becomes the video content, and you add motion to that exact scene.

Reference media with Vercel Blob

References must be URLs. If you have local files, upload them to Vercel Blob first, then pass the returned URLs in inputReferences.

reference-to-video-with-blob.ts
import { experimental_generateVideo as generateVideo } from 'ai';
import fs from 'node:fs';
import 'dotenv/config';
import { put } from '@vercel/blob';


async function main() {
  const catImage = fs.readFileSync('./cat.png');
  const { url: catUrl } = await put('cat.png', catImage, { access: 'public' });


  const dogImage = fs.readFileSync('./dog.png');
  const { url: dogUrl } = await put('dog.png', dogImage, { access: 'public' });


  const result = await generateVideo({
    model: 'alibaba/wan-v2.6-r2v',
    prompt: 'character1 and character2 play together in a sunny garden',
    inputReferences: [catUrl, dogUrl],
    resolution: '1280x720',
    duration: 4,
    providerOptions: {
      alibaba: {
        shotType: 'single',
      },
    },
  });


  // Save the generated video
  fs.writeFileSync('output.mp4', result.videos[0].uint8Array);


  console.log('Video saved to output.mp4');
}


main().catch(console.error);