Get your dancing shoes on! AI and virtual reality team up to bring the party to you

Staying
home
to
dance
the
night
away
may
soon
be
the
next
big
thing.
EU-funded
researchers
are
using
AI
to
create
an
online
dancefloor
for
the
whole
world
to
share.

By
Tom
Cassauwers

The
festive
season
can
be
a
lonely
time
for
many,
and
too
much
time
spent
online
can
add
to
the
feeling
of
isolation.

“We
have
this
fantastic
tool
with
the
internet,
which
is
really
changing
the
way
the
people
interact,”
said
Pieter
van
der
Linden,
co-founder
of
French
digital
solutions
company
Vivitnet.
But
the
way
we
use
the
internet
can
inhibit
real
human
connection.

“Today’s
interaction
online
is
passive.
It
is
about
the
brain.
It’s
only
about
thinking
and
much
less
about
feeling
and
emotions,”
he
said.
“For
many
people,
the
internet
acts
to
isolate
them
instead
of
bringing
them
closer
together.”

This
is
something
that
van
der
Linden
and
a
team
of
researchers
from
Germany,
Finland,
the
UK
and
France
have
set
out
to
change.
Their
aim
is
to
overcome
the
limitations
of
online
interaction
by
extending
it
to
include
movement.
And
they
have
chosen
dancing
as
their
main
use
case.

Dancing
the
blues
away

Van
der
Linden
is
part
of
an
EU-backed
research
project
called
CAROUSEL
that
has
brought
together
AI
and
immersive
interactive
technologies
to
enable
people
to
connect
and
dance
with
a
partner
–
real
or
virtual
–
online.

While
loneliness
has
been
shown
to
be
detrimental
to
both
physical
and
mental
health,
the
opposite
is
true
of
dancing.
Dancing
offers
a
wide
range
of
health
benefits
that
span
physical,
mental
and
emotional
well-being.

“Dance
is
a
profoundly
human
activity,”
said
van
der
Linden.
“Dance
is
used
for
celebrating,
for
rituals,
and
also
for
having
fun.”

For
the
researchers,
the
most
interesting
aspect
of
their
work
is
the
deep
connection
created
by
dancing,
where
you
have
to
concentrate
on
the
other
person
and
follow
their
movements.

“Dancing
is
something
very
natural.
It’s
not
something
you
have
to
learn,
even
babies
and
toddlers
dance
spontaneously,”
said
van
der
Linden.

One
of
the
driving
motivations
of
the
research
team
was
to
find
a
way
to
help
combat
loneliness
in
society,
particularly
in
the
wake
of
the
COVID-19
pandemic.

In
an EU-wide
survey
in
2022,
13%
of
respondents
reported
feeling
lonely
most
or
all
of
the
time
over
the
four
weeks
before
the
survey,
while
35%
reported
being
lonely
at
least
some
of
the
time.
Following
the
pandemic,
loneliness
is
increasingly
recognised
as
an
issue
of
public
importance,
along
with
mental
health
problems.

Keeping
in
sync 

The
research
is
being
led
by
the
German
Research
Center
for
Artificial
Intelligence
in
Kaiserslautern,
Germany,
which
has
partnered
up
with
Grassroots
Arts
and
Research
UG,
an
SME
based
in
Cologne,
Germany,
as
well
as
with
Aalto
University
in
Finland,
Edinburgh
Napier
University
in
the
UK
and
Vivitnet.

The
research
team
has
come
up
with
a
combination
of
technologies
that
allows
users
to
sense
each
other’s
presence
and
movement,
even
if
they
are
not
physically
in
the
same
space.

Cameras
first
capture
the
dancers’
movements,
while
AI
then
transforms
their
bodies
into
avatars
that
accurately
reproduce
their
movements
on
screen.
One
of
the
technical
challenges
was
getting
rid
of
any
time
lag
in
the
communication
which
would
make
it
difficult
for
dancers
to
stay
in
sync.

An
effort
was
made
to
use
equipment
that
was
as
standard
as
possible
–
virtual
reality
headsets,
movement
trackers
and
3D
cameras.
In
the
future,
the
researchers
envision
being
able
to
use
mobile
phones
and
standard
headsets.

“Our
dream
is
to
make
this
accessible
to
the
mass
market,
so
we
have
to
bring
down
the
price
of
the
equipment
to
something
in
the
area
of
a
mobile
phone,”
said
van
der
Linden.

Disconnecting
to
connect

German
professional
dancer
Alexandra
Schmuklermann
is
a
research
assistant
on
the
CAROUSEL
project.
Her
experience
as
a
dancer
was
invaluable
to
the
team
effort,
and
she
is
enthusiastic
about
the
results.

“I
think
it’s
a
great
addition
for
the
dance
world
for
many
reasons,”
she
said. “The
fact
that
two
people
from
different
places
in
the
world
can
dance
together
in
the
same
digital
environment
is
amazing,
even
for
professional
dancers.”

If
you
want
to
collaborate
with
a
choreographer
who
lives
in
the
USA,
for
example,
it
can
be
very
expensive,
she
says.
“But
you
could
meet
in
this
online
space
and
dance
together,
even
choreograph
together.”

For
Schmuklermann,
the
ability
to
completely
disconnect
and
tune
into
another
world
is
also
important.

“You
are
transported
immediately
into
this
different
world
and
it
helps
on
a
psychological
level,”
she
said.
“Even
in
a
very
crowded
place,
I
could
personally
use
this
game
to
shut
the
whole
world
out.”

Not
too
easy,
not
too
hard

To
make
the
system
accessible
to
everyone,
whatever
their
age
or
fitness
level,
the
team
recognised
that
it
was
important
to
select
moves
and
dances
that
are
both
creative
and
easy
to
copy.

“It
is
a
challenge,”
said
Schmuklermann.
“The
dance
moves
need
to
be
creative,
but
they
also
cannot
be
too
crazy.
You
cannot
have
people
being
afraid
to
dance
with
the
system.”

To
avoid
accidents,
a
virtual
boundary
is
set
out
before
the
game
starts.
The
user
is
alerted
as
soon
as
the
boundary
is
crossed
and
the
game
instantly
switches
mode
so
that
instead
of
being
in
the
virtual
space,
the
user
is
standing
in
their
room
again
and
can
see
any
obstacles.

Users
can
also
choose
the
environment
they
want
to
dance
in.
Van
der
Linden
acknowledges
that
although some
people
love
going
into
the
virtual
reality
worlds
created
by
the
game,
this
is
not
the
case
for
everyone.

For
this
reason,
the
team
also
created
an
augmented
reality
mode
for
one
application.
Instead
of
being
immersed
in
an
imaginary
environment,
users
can
invite
others
into
their
own
room.

“For
a
number
of
people,
this
can
be
reassuring,”
he
said.

Avatar
on
the
dancefloor

As
well
as
real
virtual
dance
partners,
the
CAROUSEL
researchers
have
designed
dancing
avatars
powered
by
AI
systems.

“They
are
a
kind
of
digital
icebreaker,”
said
van
der
Linden.
“It
makes
dancing
with
others
feel
less
awkward.”

AI
dancers
were
trained
using
several
different
approaches
and
algorithms.
Getting
into
the
spirit
of
things,
the
research
team
even
recorded
themselves
dancing
a
salsa-type
dance,
as
well
as
a
group
of
semi-professional
lindy
hop
dancers.

These
sessions
were
recorded
in
the
experimental
motion
capture
studio
at
the
Max
Planck
Institute
for
Intelligent
Systems
in
Tubingen,
Germany.
Adam,
the
AI
avatar
disco
dancer,
was
trained
by
two
professional
dancers
in
Finland
wearing
motion
suits.

Healthy
mind,
healthy
body

While
much
work
remains
to
be
done
to
turn
this
into
a
mass-market
product,
researchers
are
already
contemplating
the
next
steps. The
vision
is
to
have
a
mass-market
application
that
can
be
used
by
anybody,
but
is
also
geared
towards
medical
needs.

The
aim
is
to
combat
isolation
and
loneliness
–
and
also
promote
a
healthy
lifestyle.
CAROUSEL
will
encourage
people
to
move
rather
than
just
sit
at
their
desks,
for
example.

“Moving
has
a
lot
of
benefits,
a
lot
of
studies
show
how
useful
it
is,”
said
van
der
Linden.
Dancing
is
also
a
great
mood
lifter,
helping
people
feel
happy
and
connected.
“That
is
basically
what
the
project
is
about,”
said van
der
Linden.

Research
in
this
article
was
funded
by
the
EU’s
Horizon
Programme.
The
views
of
the
interviewees
don’t
necessarily
reflect
those
of
the
European
Commission.

More
info

​This
article
was
originally
published in Horizon the
EU
Research
and
Innovation
Magazine.