Creating Motion Graphics and Videos with Generative AI

Present-generation generative AI (genAI) tools are able to take as input still images and text prompts to generate motion graphics (animations) and videos. (The content generated is illusory motion on a 2d plane.) This slideshow explores how such animation models may function (in the Deep Dream Generator). It proposes some practical use cases. This work is based on several years of sparse and sporadic idiosyncratic first-hand experimentation. This slideshow works a little as a visual album as well as a slide deck. The account used for this exploration is https://deepdreamgenerator.com/u/sjjalinn.

1. Creating Motion Graphics
and Videos
with Generative AI

2. Overview
•Present-generation generative AI (genAI) tools
are able totake as input still images and text
prompts to generate motion graphics
(animations) and videos. (The content
generated is illusory motionona2dplane.)
This slideshow explores how such animation
models may function (in the Deep Dream
Generator). It proposes some practical use
cases. This work is based on several years of
sparse and sporadic idiosyncratic first-hand
experimentation. This slideshow works a
little as a visual album as well as a slide
deck. The account used for this exploration
is
https://deepdreamgenerator.com/u/sjjalinn.
2

3. Overview (cont.)
•The Deep Dream Generator (DDG) is a generative AI tool that
enables the making of still images, music, video, animations, and
other digital contents.
•There are a number of models that exist for the various creations
in DDG.
•For the making of animations and videos, there are six models
(ProVideo, SeeDance, MiniMax, Veo 3, Morph, and Video Editor).
•The works referenced here refer to those made with ProVideo.
3

4. Light Background
4

5. 5

6. Prompting ProVideo
•Various paths may be taken to prompt the ProVideo model.
•One may begin with a still image and just click the “Generate”
button.
•One can begin with a still image, which is treated as predominant,
and add text (which does have an effect on the animation output).
•One can go with pure text without an image in ProVideo.
•The rendering can take from seconds to several minutes.
6

7. Other Animation and Video Models on DDG
Platform
•Other models have other variations in terms of possible inputs.
The models output between 5 – 8 seconds for the free versions
currently. Additional points may be purchased for more elaborate
outputs.
7

8. 8

9. 9

10. End-to-End Generative AI Only?
•It is possible to use the DDG to create images, animations, sound,
and video fully from text. There can be an end-to-end AI, from a
text-based expressed concept. Such works do have a machine-
created digital arts patina.
•Such outputs may look wholly of a piece.
•Human inputs, such as still visuals, are a way disrupting the
machine. The visuals serve as a kind of forcing function to make
the genAI work a little harder and integrate more into the end
product. It gives humans more control.
10

11. End-to-End Generative AI Only? (cont.)
•In the still image sequence seeded with a digitally scanned analog
visual, the human user may apply a percentage of effect of that
visual on the final output visual. The visual may also be evolved on
the platform until the desired output is achieved.
11

12. Loose Experimentation
12

13. Loose Hands-On Experimentation
•DDG started offering animation outputs in March 2024.
•The author started experimenting with DDG some two years ago.
•She deleted numerous created images over time to try to narrow
the focus on what was visually unusual / original for her directly.
•Also, only 30 videos may be on the account at any time, so many
videos have been created and deleted. (This is why some of the
animations are shown with the play arrow overlay, and others do
not.)
•The animations were experimented with using all images seeded
by analog visuals and evolved on DDG.
13

14. Loose Hands-On Experimentation (cont.)
•Multiple iterations of animations were run to gauge the creative
range of the genAI. How much convergence could be expected?
How much divergence?
•The various types of visuals included figurative and abstract
works. They also included sometimes one principal character and
sometimes others. Some were scenes along, without any in-
scene character (animal, machine, human, other).
•ProVideo can veer off in new directions, new scenes. It can riff
from a starting still visual.
14

15. The Loose Research Approach
•The early research were mostly still images used as prompts and
without any text prompts.
•The idea was to let the software view the visual prompt and to
allow it to infer the image and to apply the motion, however it
tended.
15

16. Initial Research Observations
16

17. Initial Research Observations
•The genAI does not infer motion in a way that people might.
•The genAI leans towards narrative and storytelling. It will take a
scene and character…and spin it out to something more.
•ProVideo may depict scenes that are possible in an AI’s
Imaginarium but perhaps not in the real world.
•The animations / videos generated can be compelling in various
ways.
•One is a sense of spectacle.
•Another is a sense of flight-of-fancy.
•Another may be a sense of poignance.
17

18. Initial Research Observations (cont.)
•The genAI model may sometimes confuse the foreground and
background. It may pick up part of the background and treat it as
foreground in an animation / a video.
•The genAI model does not play with complexity of layers per se.
18

19. Initial Research Observations (cont.)
•The genAI may be put off by unusual topics. It may revert to
familiar visual tropes or motions. It may not commit to illustrating
the seeding visual with sophistication. (It may not have known
what a balut was.)
•It may be put off by complexity. (If there are many visual elements,
it may not handle it well.)
•It does and does not have a filter. One scene that was created
was fairly graphic (and violent) for the platform (but not for those
who watch nature shows).
19

20. A Visual Walkthrough of Animations
that Worked
20

21. A Visual Walk-through
•What follows is a visual walk-through of some of the
stills used to create the animations. There are notes
about how the genAI models performed…and perhaps
ways for the human to improve the output performance.
•Some of the referenced animations are available on the
platform. These will ultimately be removed as new mini-
experiments are tried with other seeding visuals.
•The video player on the platform may be adjusted to
speed-up or slow the animations / video, for the user’s
pleasure.
21

22. Minor Motions
The generative AI (ProVideo) is fine with
essential stasis if the seeding image is
perhaps calm. It does not go to drama
right away, which is a net positive.
If a human user may want something more
dramatic, that may be indicated in the text
prompt that accompanies the seeding
visual.
22

23. A Narrative Arc
A frog goes for an insect lunch (which
looks suspiciously like a splat of opaque
gouache watercolor or India ink)…but does
not quite capture it . The frog has lost its
prey.
There is a narrative arc.
The dramatic background augments the
story.
23

24. Selling the Motion
A kite is flying in the sky. Its banners, its
relationship to the sun, the clouds at the
lower part of the 2d plane, and even a bird
flying in the background…sell the motion.
This visual has both foreground and
background, as depicted.
24

25. Going to Play
This elegant cat plays with a leaf that
blows by.
This is a common theme, of animals
engaging with their surroundings, be it
outdoors or indoors.
This animation model seems to
sometimes keep the emotions light.
25

26. Interacting Entities
The ProVideo AI model shows two
characters dancing with elegance, and the
characters do not deform as they swirl and
turn.
26

27. Playing with Ephemera
This green-eyed cat plays with the swirls
with its front paws, in a way that melds
reality and fantasy.
Seeding images offer a sense of context
and mood and personality. (These were
prompted using analog artwork in various
still image genAI models. The artifice goes
deeper than the animation layer.)
27

28. Motion and Sound
This neon-ish train chugs along at speed in
the animation.
The genAI animation model made a visual
so compelling that one could almost also
imagine the sound.
28

29. Chasing Reward
The visual shows bees chasing sweet fruit
and blossoms under water.
The improbability of the given scene did
not intimidate the ProVideo genAI model.
It merely used a general movement of the
bees and the background
simultaneously…and the bubbles were
another element of motion.
29

30. Underlying
Musculature
The seeding visual showed two horses that
looked like they were 3d based on shading
and contour.
The animation showed rippling
musculature in the two horses. It showed
the two horses interacting naturally.
30

31. Change of Light as
Motion
The generative AI showed different neon
light changes…as a way to evoke motion.
It also used jitter and then a light
explosion, for a sense of drama.
31

32. Instantiating
Possibilities
This visual shows dinosaurs hatching from
a dinosaur nest.
A generative AI may depict an artful
expression of a possible future and a
possible past. (Dinosaurs did hatch from
eggs, but how they looked and their
sequence and their egg placements could
be wholly unlike this depiction, of course.)
32

33. Contrasting Motion
The generative AI started with this visual of
sailboats moving right, and then a larger
imagined sailboat entered the scene from
the right and sailed left.
The foreground and background contrast
worked memorably.
33

34. Wholly Imaginary AI
Life
Remember “artificial life” created in digital
ecosystems?
This visual came about from some wild
experimentation. The author chose to see
what the generative AI would make of
these. It made the various objects float.
The author put a label on this as
“Dissemination,” to invoke something of
the fecundity of life…and its many forms.
[One trick to being creative in the genAI
space is to not label something until after
it has been created. That way, things may
align much better.]
34

35. Physical Motion
This last of the bioluminescent hunting
cats was depicted by ProVideo as chasing
a butterfly. But its first movement shows
an unusual twist of its back end that does
not look likely for a cat. The author does
not own a cat and is only guessing…but the
cat’s slinky turn does not quite look right.
35

36. Going into the
Imaginary
The author found that when she used
“seaweed bulbs” as a prompt along with
an evocative painting, she could get some
very cool effects.
The underlying image shows one such still
image.
ProVideo chose to make the bulbs expand
and rise and burst.
Pure imaginary physics, scene, and
motion…but so fun!
Part of the joy is going with what the eyes
see…in contrast with what one knows of
the world in the real.
36

37. Wonder and Concern
The visual shows the ocean seeping into
the old town part of a modern city. There
are fish swimming in air. There is algae
growing up from the sidewalk. A stylish
woman walks down the sidewalk and turns
to look at the fish approaching.
Her strolls will never be quite the same.
37

38. In Pursuit…of Human
Attention
This blue-eyed black cat is on the hunt. It
hunches down as part of its stalking…and
jumps away at a motion.
Many of these short animations are
mesmerizing. Even several times in…
38

39. Turning on the Charm
This little mouse lives in a cheesy fortune
cookie. In the animation, it emerges from
its home to explore. This mouse is living its
best life.
39

40. Interplay of
Abstractions
This visual depicts winter. The motion
brings one part of this visual to the fore.
40

41. Destruction
This visual of dusty trees ends with a giant
eruption of dirt from the ground that
swallows the trees.
Human attention is drawn to destruction.
41

42. A Surprise is Waiting
This visual was seeded with a depiction of
Northern lights,,with a wooden cabin at the
base of the image.
Suddenly, as the in-video camera pans
down, the doorway and window of the
cabin light up. In the doorway is a deer.
The genAI has come up with a plausible
surprise.
42

43. Falling Parachutes
This visual animation shows several
parachutes dropping in apparent close
proximity and perhaps falling into each
other.
43

44. Tumult
These ships are being tossed in ocean
waves in this animation.
One of the secrets to creating a somewhat
sparse image is to use the white space in
the seeding analog image or doodle.
44

45. Sea Foam Islands
The generative AI offered a paced evolution
of these sea foam islands.
The human visual system can be
manipulated to create a sense of reality
from unreality. The author would argue
that people have a duty to know and to
discern. She would also argue that all
have a duty to enjoy the game of generative
AI, too.
45

46. Flow
The generative AI used the still image to
create a sense of water flow disturbing the
undersea scene.
46

47. Surprises
This video shows an experimental
submarine about to deploy.
The original image was used to seed the
submarine, which was already a surprise.
Then the motion of the experimental
submarine sliding forward was another
happy surprise.
47

48. Twirling
ProVideo took the visual and made the
umbrella twirl.
The original image was created from a
social network graph (from an account on
the former Twitter) created in NodeXL in an
artful way.
It is a challenge to evolve data into visuals
into art…
48

49. Echolocation
The visual depicts echolocation. The
animations may evoke multimodality,
including sound. Sound outside human
hearing ranges… in some cases.
49

50. Celebrating
Brainstorming
In work, people have experiences with
prodigious brainstorming sessions.
This visual shows scrapped paper
churning as more and more ideas are
created and dissipated.
The idea is fun…of brainstorming using
paper (which is so old school).
50

51. Getting Something
Crunchy from the Bar
The best genAI models give humans
something to work with.
Here, two birds are sipping from a berry
drink. The one on the left runs off to get
“something crunchy” from the bar.
This one is a case of applying an idea after
watching the bird run off a few times. That
bird will be back, and it will be back with
something delicious to go with the berry
drink.
51

52. Approaching a
Decision Juncture
The motion of this scene is a forward one,
going towards a split in the road.
Sometimes, having no characters in a
scene can enable the viewer to immerse
more thoroughly.
In other cases, characters may be
conduits used to enter and understand a
scene.
52

53. Contemporary Life
Contemporary living takes on many forms.
The genAI showed the woman patting her
work oxen.
53

54. Encroaching Heat
The generative AI took this visual…and
depicted a severe brightening of the sun to
indicate the pre-singe phase.
54

55. Transitioning to a New
Scene
This visual shows a microbot.
The animation shows it growing a spine
and swishing its tail…into a new undersea
scene.
ProVideo riffed in an innovative way.
55

56. Hiding
The generative AI took the still visual and
depicted the small mammal sitting quietly
among moving grass.
This survival strategy is evoked even as the
visual itself is quite unusual. Look at the
texture of the grasses.
56

57. To Freedom
A man in a cage is approached by a flying
bird…and suddenly, he can step outside of
the cage between the bars.
That motion is an act of digital
physics…but that sense of emancipation is
inspiring.
57

58. Industrial Size Action
This visual is depicted effectively, also
evoking a sense of internal sound given the
fidelity of the motion.
58

59. Particulate Effects
A white-out effect was illustrated as a still.
In the animation, it provided a credible
sense of snowiness…albeit in an artful
sense.
True testing of particulate motion was not
done here per se.
59

60. Drama
The visual shows pants on a laundry line.
The genAI animator created a sense of
drama by showing these blowing in a high
wind. It’s a good thing the wooden clips
held tight.
60

61. Some “Failed” Still Image Seeds
61

62. A Disintegrating Cat
ProVideo did not recognize the cat, and in
the animation, the cat fully disintegrated.
62

63. Encryption -
Decryption
The idea was to have the lettering and
numbering change and shift. No such
luck.
In one iteration, the genAI punted and just
panned the camera.
In another, a male hand came into frame
and was trying to move things around.
The genAI does not seem to have the
precision of letters and numbers yet.
(GenAI for still images are now getting
somewhat better at text and spelling.
Somewhat.)
63

64. Reverse Direction?
When this visual was created, the idea was
the rain from the sky was filling the goblet.
This illustrated “drinking the sky.”
The generative AI saw it the other way and
had the drink evaporating into the sky.
64

65. Partial Cat
The visual was run twice in ProVideo.
In the first rendition, the cat was illustrated
with a tail but no back legs. The AI did not
infer a whole cat because it was not in the
seeding image. In this scene, the cat
swung itself around but lacked two hind
legs but did have an impressive tail.
In the second found, the genAI drew the
whole cat, which moved with flourish.
65

66. Motion for Motion’s
Sake
The visual of dancers on netting resulted in
motion for motion’s sake. Movement
captures human visual attention, but it
should also contribute to some sense-
making.
66

67. An Interpretive Lens
What’s an AI to do with strange egg sacs
floating in physical space?
The genAI chose to show some up and
down movement of the respective visual
elements.
67

68. Deformations
The seeding still visual shows two aircraft
flying against a gridded sky…but the two
aircraft change physical form as they fly.
Their deformations made it seem like the
crafts were reshaping themselves as they
sped around.
The grid was interesting and added to the
sense of motion…as if there was a point of
reference.
68

69. Direction of Rocking
A caterpillar is enjoying its creature
comforts. It is sitting in a hammock
plumply.
Instead of rocking from side-to-side, it
rocks forwards and backwards. Still
cool…but not intuitive.
69

70. Rising Potatoes
The original visual was of potatoes growing
eyes…sprouting.
In the animation, these potatoes did not
grow per se but started rising into the air,
as if they were filled with helium.
70

71. Low Hanging Fruit
The still visual is a stylized expression of
the concept of “low hanging fruit,” or
things that are trivial to do.
The genAI animator used stretching of the
items and an overall vertical panning.
71

72. Tonality
The visual depicts a “relentless” dog.
The visual (using Illustrator and
Photoshop) shows a fierce animal.
In the animation, this dog just ends up
playing ball with the person behind the
camera. Adult machine to puppy.

72

73. Lost Spooky
The original image is a little strange, with
the light source behind the trees and the
shadows cast toward the viewer.
The animation emphasizes the path and
speeds the viewer forward. The tone of
spookiness is lost.
73

74. Into the Drink
The original shows a roller coaster in
water…with two cars on the tracks.
None of the cars go into the water.
The genAI punts and just twirls the image
horizontally. It took a pass.
74

75. Explosive Snacks
The visual shows people in a factory
creating snacks.
The various types of machinery make no
sense per se…because they do not line up
in a productive sense.
The generative AI has one of the machines
in the back explode.
75

76. Just Think
The visual shows a carousel of intertwined
hands.
ProVideo just had a pair of hands unfold
from this, and that was that.
It felt like some potential was left of the
table.
76

77. Children at Play
The seeding image shows little children at
play.
The generative AI has them skate around
woodenly, without moving their feet per se.
(The genAI models have improved in the
time that the author has been using the AI
animators…)
77

78. Foreground -
Background
This visual shows some UAVs / drones
flying in the Arctic.
In the animation version, only the UAVs in
the foreground move.
78

79. Flouncy
This glamor cat, created in part using
glitter on paper for the seeding image,
flounces and sits.
Just think if it did more.
79

80. Taking the Background
The visual shows hyenas in the still image.
In the background is something of a chain
link fence.
The hyenas move off right, but they take
the background fencing with them.
Of course, the original image was not so
crisply defined.
80

81. Graphic Violence
This visual shows a majestic praying
mantis in the attack pose.
The praying mantis does in an insect for its
dinner. It then turns to look directly at the
camera as a form of challenge.
The animation was so violent that the
author did not leave the published video
up for long.
81

82. Running
Backwards…into the
Fire
The visual shows two wild horses escaping
flames or on fire.
In the animation, the horses run
backwards into the fire.
82

83. End of Twitter
Back in the day, it seemed like Twitter was
on its last legs.
DDG was used to create a cake with the
blue twitter bird as filling.
The generative AI showed the cake
rotating.
83

84. Rotating
The turducken was also shown rotating,
but in this one, the male mallard duck’s
head turns toward the camera. It
maintains eye contact.
The prompt for a turducken was just to see
if the former still image AI had a point of
reference, which it clearly did not. (The
author has also never had a turducken and
so has no point of comparison…but she
knows it’s not this.)
84

85. A Race Home
The author took one of her student’s
descriptions from her teaching days. The
student described how they would set the
lobsters loose down the grocery store
aisles to see which one would win the
race. They would do this in the early
morning hours of 24/7 grocery stores back
in the day.
In the animation, a lobster runs down a
grocery store aisle and leaps up back into
its glass tank. The glass tank and such are
wholly in the imagination of the genAI.
The genAI created a story arc. A happy
ending.
85

86. A Bridge of Frenemies
The original image shows a man and a
woman walking across a bridge with a
strange-headed dog.
In the first iteration over a year ago, the two
walk forward awkwardly.
In the latest iteration, the woman turns
around and smiles glamorously at the
camera.
Neither one works well with the seeding
image. The sense of the unknown is not
captured. The mood is not captured. The
stylish couple fail to maintain stride into an
unknown future in a new land.
86

87. Background Stasis
This sidewalk scene is a complex one.
The generative AI had those in the
foreground move (with believable
gaits)…but those in the background do not.
They are just window-dressing.
87

88. Chunky Clouds
These marshmallow clouds look fanciful.
In the motion, some chunks move in one
direction. Another chunk moves in
another.
The various clouds are not seen as
separate elements per se.
There is not a sense of wind pushing the
elements.
88

89. Run Run Run!
This glowy rabbit in the still image has a
part of it that reads as glass, as ice. It
glows preternaturally and does not seem
able to camouflage itself into the snowy
surroundings even though it is white.
This hunted rabbit hops very slowly to a
divot in the ground to hide.
It has no sense of urgency. It seems to
think its naivete is a protection.
89

90. The visual shows a colorful egg in an
incubator.
What’s in it?
In the animation, there is a flare of feathers
that emerge in a way that doesn’t break the
shell. The feathers stick out in a full circle
marking the circumference of the egg.
90

91. Running Children
The original image shows a moody scene
of heavy rain and children running away
from the viewer.
In the animation, there is a sharp change in
mood from the original image. New child
characters run into scene. They are happy
children at play. The sun has risen.
Okay then.
91

92. An Abundance of
Hope
The visual shows a stylized version of
balut.
In the animation, a small living bird
emerges from the balut.
92

93. 93

94. Testing for Human Receptivity
•Part of the learning of genAIs in the present iterations is the
receptivity of the created contents by people. Do they download
the content? Do they share the content? Does the share evoke
conversations?
•The outputs exist within ranges, based on the permissiveness or
impermissiveness of the tool’s creators. The serendipity is highly
controlled.
•Even if a technology is willing to “go there,” people may not be.
•People can be highly limiting in where they are willing to let their
imaginations go.
94

95. Some Practical Use Cases
95

96. Some Practical Use Cases
•Amusement
•Entertainment
•Self-expression (co-created between humans and machines)
•Storytelling
•Teaching and learning
•Art
•Motion storyboarding, design thinking
•Others
96

97. Post Processing, Too
97

98. Post-Processing
•The animations can be transcoded into various formats of stills
and motion animations and videos.
•Snippets may be cobbled for longer works (although that would be
tedious). It would be simpler to pay for longer animation
renditions, if one can be semi-confident that the investment
would be worth the effort and the cost.
98

99. Conclusion and Contact
99

100. Conclusion and Contact
•Dr. Shalin Hai-Jew
•[email protected]
•The researcher co-created all the visuals and animations used in
this slideshow using the Deep Dream Generator and its rich
embedded content-creation models.
100