Fingerdance

A choreographic exploration connecting meaning to movement.

Fingerdance was developed at the 8th Choreographic Coding Lab in Amsterdam in May 2017. The work is an exploration of different ways of relating bodily movement to computer technology and digital visual effects.

Background

The Choreographic Coding Lab (CCL) is an initiative born from Motion Bank, a four year research project of the Forsythe Company that aimed at the translation of choreography into online digital scores. In their own words, the project sought to:

invite artists with diverse approaches to dance to further explore the special qualities of computer-aided recording and design applied creatively to the documentation, analysis, and transmission of their choreographic ideas and processes to a wide audience.

The Motion Bank research took a number of forms, including working with motion capture technology and developing video annotation software specifically aimed at choreographers. The CCL series aims to introduce those findings to and invite further exploration and collaboration from computer artists, dancers, sociologists, architects, philosophers, and medical researchers. I attended the workshop in May 2017 with my collaborator Tatou Dede.

Over the course of the week in Amsterdam, we were introduced to a project that approached the integration of dance and digital technologies in a variety of ways. These ranged from motion capture and digital rendering through the use of virtual reality headsets to neural network analysis of movement and gestures. Considerable attention was given to the problem of how to translate dance performances into a form of data that could then be used as the basis of computer driven music or visuals. This could include motion capture, that is data as a series of points in space over time, or gesture analysis, where the computer is used to analyse gestures or shapes made by the body over time, which then drives a visualisation.

Theoretical Underpinnings

However what really stood out for me was a presentation from dance researcher Suzan Tunca on the work she had done with Emio Greco on a system they named "pre-choreographic elements". The elements are a solution to the problem of how to communicate and transmit contemporary dance choreography, and in particular, how to convey to dancers the physical animus of a dance work that goes beyond having dancers learn a sequence of steps or mimic the style of a choreographer. The pre-choreographic objects are imaginative concepts that the dancers use to imbue their movements with the physical sense or style intended by the choreographer. An example:

For me this was really interesting as it introduced me to a whole new level of analytical thought about dance that I hadn't encountered. I thought it would be interesting as a digital collaborator to harness this knowledge. For a start, I'm not particularly interested in working directly with motion capture data, and training computers to recognise gestures is challenging. But beyond this, I think the concept of pre-choreographic elements as the animating principle or emotion of a movement sequence offers the possibility to engage with dance professionals as intellectual equals. It's an opportunity to respect the analytical knowledge and approach that choreographers have about their own field rather than trying to reduce human movement to a stream of data points in time. Fingerdance is an attempt to bring this analysis into the computer and use it as the basis to drive visualisations.

Experimentation

Fingerdance attempts to combine visualisations driven by data captured from the body through sensors, and analytical data provided by a dance professional. First we developed a simple e-textile based bend sensor that was fitted to Tatou's finger. We then recorded a "dance performance" by the hand. The movement was an improvisation based on the simple instruction to move the hand as freely as possible without triggering the bend sensor to fall below a certain value (indicating too great a curve of the index finger). Tatou could see real time feedback on a screen for her actions.

The performance was filmed and then Tatou used the Piecemaker2 platform from Motion Bank to annotate the video according to the spatial position and movement of the hand, and also the emotional or conceptual level of the performance. The resulting annotations were analysed and mapped to varying computer generated graphics. Finally the performance video, bend sensor data, and generated graphics were combined into a single output using Processing.

While the results of this experiment are not visually spectacular, I think it's a promising pathway for further research, in particular how rule-based choreographic improvisation can be developed through a feedback look with annotations and visualisations in the computer.

Demo

2017
Choreographic Coding Lab Open Night
Amsterdam Netherlands

Role

Programming
Video editing