16 Dec, 2023 @ m-onz web page

Hallucinating generative ai visuals

This is an experiment using generative ai image-to-video ML to hallucinate extended video sequences from a single source image.

You will need an API key from replicate.com exported to an environment variable:

Tested on debian linux

This requires ffmpeg

This costs money to run *see replicate.com pricing

Add a new image file and configure the script to use your new image.. or replace "avatar.png" with a file of the same name.

sudo apt-get install ffmpeg
git clone https://github.com/m-onz/hallucinate
npm i
export REPLICATE_API_TOKEN=r8_BRU**********************************
node hallucinate.js

Wait a long time!

You should then eventually get an "output.mp4" with a complete sequence.

I added audio using musicGen on replicate.com and ffmpeg to add the audio *see my last blog article for an example of doing that.

This script does not do any clean up of generated files so you need to manually delete them.

An example output
m-onz: hallucinate
Get the code
github.com/m-onz/hallucinate