Skip to content
Alibaba Cloud

Wan v2.7 Reference-to-Video

Wan v2.7 Reference-to-Video is the Wan 2.7 reference-to-video model from Alibaba Cloud, generating new scenes from reference images and videos with combined subject and voice referencing at 720p or 1080p. Your use is subject to Alibaba Cloud's Terms & Privacy Policies.

reference-to-video
Use with AI GatewayView docs
import { experimental_generateVideo as generateVideo } from 'ai';


const result = await generateVideo({
  model: 'alibaba/wan-v2.7-r2v',
  prompt: 'A serene mountain lake at sunrise.'
});
Read docs
OverviewAPIAboutProvidersSimilarFAQ

Getting started

Generate videos with Wan v2.7 Reference-to-Video using the experimental_generateVideo function from AI SDK 6 or later. AI Gateway handles routing and polls until the video is ready.

Install the AI SDK (pnpm add ai dotenv), create an API key from the API Keys page, and set it as AI_GATEWAY_API_KEY in your environment. Full setup is covered in the video generation quickstart.

index.ts
import { experimental_generateVideo as generateVideo } from 'ai';
import fs from 'node:fs';
import 'dotenv/config';


async function main() {
  const result = await generateVideo({
    model: 'alibaba/wan-v2.7-r2v',
    prompt: 'Image 1 and Image 2 have a friendly conversation in a cozy cafe',
    inputReferences: [
      'https://example.com/cat.png',
      'https://example.com/dog.png',
    ],
  });


  // Save the generated video
  fs.writeFileSync('output.mp4', result.videos[0].uint8Array);


  console.log('Video saved to output.mp4');
}


main().catch(console.error);

Top-level parameters

Pass references through inputReferences and size the output with resolution and duration. v2.7 also accepts an aspect ratio, through providerOptions.alibaba.ratio.

wan-v2.7-reference-to-video-top-level.ts
import { experimental_generateVideo as generateVideo } from 'ai';
import fs from 'node:fs';
import 'dotenv/config';


async function main() {
  const result = await generateVideo({
    model: 'alibaba/wan-v2.7-r2v',
    prompt: 'Image 1 and Image 2 have a friendly conversation in a cozy cafe',
    inputReferences: [
      'https://example.com/cat.png',
      'https://example.com/dog.png',
    ],
    resolution: '1920x1080',
    duration: 4,
  });


  // Save the generated video
  fs.writeFileSync('output.mp4', result.videos[0].uint8Array);


  console.log('Video saved to output.mp4');
}


main().catch(console.error);
ParameterTypeRequiredDescription
promptstringNoText description of the video to generate. Max 5000 characters.
durationnumberNoVideo length in seconds. 2-10 seconds.
resolutionstringNoResolution ('1280x720', '1920x1080').
aspectRatiostringNoAspect ratio ('16:9', '9:16', '1:1', '4:3', '3:4').
generateAudiobooleanNoGenerate synchronized audio with the video.
inputReferencesArray<string | { data: string; mediaType: string }>NoReference images and videos, mapped onto the request automatically. Pass videos as { data, mediaType: "video/mp4" }. See the Input limits table for counts and formats.
frameImagesArray<{ image: string; frameType: 'first_frame' }>NoOpening frame of the clip, as a single first_frame entry. Replaces prompt.image and wins when both are set. URLs only. Wan does not interpolate to an ending image, so a last_frame entry is ignored with a warning.

Input limits

InputFormatsSourcesMax countMax sizeLimits
Text————Up to 5000 characters
Imagejpeg, jpg, png, bmp, webpurl, base64520 MB≥240px · ≤8000px · aspect 1:8–8:1
Videomp4, movurl3100 MB1-30s · ≥240px · ≤4096px
Audiowav, mp3url—15 MB1-10s
Up to 5 reference inputs total across images and videos.

Provider options

Set the ratio and, when the automatic mapping is not what you want, list the media explicitly. media overrides inputReferences entirely.

wan-v2.7-reference-to-video-provider-options.ts
import { experimental_generateVideo as generateVideo } from 'ai';
import fs from 'node:fs';
import 'dotenv/config';


async function main() {
  const result = await generateVideo({
    model: 'alibaba/wan-v2.7-r2v',
    prompt: 'Image 1 walks past Video 1 as the camera pans',
    resolution: '1920x1080',
    duration: 4,
    providerOptions: {
      alibaba: {
        media: [
          { type: 'reference_image', url: 'https://example.com/cat.png' },
          { type: 'reference_video', url: 'https://example.com/street.mp4' },
        ],
        ratio: '16:9',
        negativePrompt: 'blurry, low quality',
      },
    },
  });


  // Save the generated video
  fs.writeFileSync('output.mp4', result.videos[0].uint8Array);


  console.log('Video saved to output.mp4');
}


main().catch(console.error);

Pass Wan-specific options under providerOptions.alibaba in your generateVideo call. References come from the top-level inputReferences unless media overrides them.

ParameterTypeRequiredDescription
mediaArray<{ type: 'reference_image' | 'reference_video' | 'first_frame'; url: string; referenceVoice?: string }>NoExplicit media list, overriding the mapping from the top-level inputReferences and frameImages. Images and videos are numbered separately in array order, so a prompt refers to them as Image 1, Video 1, and so on. referenceVoice attaches a voice reference to one item.
ratio'16:9' | '9:16' | '1:1' | '4:3' | '3:4'NoAspect ratio of the generated video. v2.7 text-to-video and reference-to-video only.
negativePromptstringNoWhat to avoid in the video. Max 500 characters.
promptExtendbooleanNoEnhance prompt for better quality. Defaults to true.
watermarkbooleanNoAdd watermark to the video. Defaults to false.
pollIntervalMsnumberNoHow often to check task status. Defaults to 5000.
pollTimeoutMsnumberNoMaximum wait time. Defaults to 600000 (10 minutes).

Reference-to-video vs image-to-video

Reference-to-video uses the top-level inputReferences to show the model what your characters look like, then generates a brand-new scene from your prompt. The reference media never becomes the video content; reference each one in the prompt with character1, character2, and so on (first entry maps to character1).

Image-to-video instead animates the actual image you pass in frameImages or prompt.image. The image you provide becomes the video content, and you add motion to that exact scene.