| |
|
Introduction
In these projects I intend to record a “live” performer’s movement with motion capture technology, and apply an inverted version to a 3D character to generate a reflective or echo effect for both the performer and audience. My goal is to digitally produce the illusion of classic human mimesis, or mimicry, as experienced in theatre and dance. Both my thesis and group project are conceptually related to Victor Turner’s anthropological analysis of theatre as a reflexive or “magic mirror” process, and Stephen Haliwell’s translation of mimesis in Aristotle’s Poetics as meaning “representation”. Other related theatre topics I will explore in my projects are Suzan Lori Park’s “Rep & Rev” technique, and Heinrich von Kleist’s concept of the “Uber Marionette”. In addition, I will synthesize these theatrical concepts with evolution and cyborg theory from the sciences and humanities to show how mimesis can be viewed as a broader cultural mechanism for technological “progress” in general.
Mimesis as a Reflective and Reflexive “Magic Mirror”
My use of mimesis is based on Haliwell’s “representation” translation, which includes both bodily mimicry in physical theatre by a human performer and virtual mimicry in a mathematically modeled 3D character. This broad translation of mimesis encompasses all forms of symbolic representation in the mind, and therefore all such forms reflected in culture. I view the mind as a symbolic mimesis, or computational model, of the physical world as experienced through the bodily senses. In this view culture is considered a behavioral mimesis or technique of the mind, and technology is a physical mimesis or materialization of culture. As technology becomes part of the physical world, it affects or re-makes the physical world experienced by our senses, thus creating a positive feedback loop driving change.
I am equating this subjunctive translation of mimesis to Turner’s “magic mirror” concept, since representation is not just a passive reflection, but instead also encompasses a reflexive process that is actively re-modeling our physical environment. Because of the reflective quality of this “mirror”, the nature of the mind can be deduced as technological or machinic. But since the mind is also undeniably an organ, on the flip side of the reflection, the nature of technology can be deduced as organic. Also, because of the reflexive or “magic” quality of the mirror, the process driving technological “progress” may be viewed as a natural evolutionary mechanism. These reflective and reflexive cultural processes can be compared to replication and mutation processes in genetic and memetic evolution theory. Although I will touch on these broader issues of mimesis in my projects, I will mainly focus on technologically simulating a simplified version in pantomime and dance.
“Mirror Gags” as a Simple Example of Mimesis in Theatre
A good reference example of reflective-type mimesis in pantomime can be seen in the mirror gags of the Marx brothers. The below YouTube clip shows two comedy skits: the first of Groucho and the other Marx brothers, and the second of Harpo Marx and Lucille Ball in a classic episode of I Love Lucy. The Marx brothers clip shows Groucho trying to outsmart his “reflection”, to expose it as human, as if it is a type of Turing Test. An interesting moment comes when Groucho spins, and the reflection does not follow, instead only mimicking the first and last part of the spin. But since Groucho’s back is turned during the middle of the spin, only the audience can see the inconsistency in the reflected movement. In the I Love Lucy episode, Lucy masquerading as Harpo reflectively follows the real Harpo’s movements, until he exposes her by using a trick hat. This skit plays with the subtleties of agency between reflective and reflexive mimicry, and calls to mind Bergson’s comedy of the mechanical in the reflective relationship and the slapstick movement.
My thesis application of this type of mimesis will be to have a 3D character reflectively follow a live performer most of the time, although with some micro variations coded into the repeated movement, except when the performer turns away from the 3D character which prompts improvisation. Producing this type of mimesis with technology adds another layer of meaning, as the digital character is simulating a human that is simulating the reflection of the human performer.
“Call and Response” as a Simple Example of Mimesis in Social Dance
A good reference example of echoing-type mimesis in dance can be seen in traditional African American “call and response” techniques inherited from jazz and blues music, as applied in the popular “street” dance “battles” of Breaking and Hip Hop. This technique originally evolved from African American spirituals, and is characterized by a delay in the repeated form, with improvisational variations added into the repetitions. In African American social dance, as personified in the jazz and swing dancing at the NYC Savoy ballroom from the 1930’s to the 1950’s, it was a cultural rule that a dancer could copy another dancer’s moves, but must vary them in some creative way, thus putting a unique and personal stamp of ownership on the movement. Over time this led to a variety of personal styles within the overall style of African American social dancing, with movements being customized to an individual dancer’s abilities and interests.
The cultural directive to vary copied dance moves can be viewed as a mechanism for change that not only increases the diversity and complexity of the individual dance, but prompts differences in styles that can lead to the formation of new dance forms. The swing dances East Coast Swing, West Coast Swing, Carolina Shag, and Balboa all likely evolved from New York Lindy Hop, but by different individuals in separated geographic and cultural environments, where the representation of the movements were accidently and purposely altered. When viewed as a cultural evolutionary process, mistakes in mimicry produce micro-mutations in the copied information or new moves, which may then be selected or discarded as part of the original style. On the other hand, intentional cultural appropriations or fusions of foreign movements to a given dance, such as adding acrobatic movements from vaudeville or gymnastics, may lead to macro-mutations in the form.
Analysis of the Mimesis Dynamic
As the dialogue between agents progresses in the mimesis process; reflective behavior fluctuates with reflexive behavior (“magic mirror” dynamic), or following fluctuates with leading (dance dynamic), or mutualism fluctuates with parasitism (microbial dynamic), or cooperation fluctuates with competition (gaming dynamic), or micro-mutations fluctuate with macro-mutations (evolution dynamic); responsive behavior is driven. When looking at creativity or improvisation as an evolution dynamic, creative and purposeful innovations in technique can be seen as a type of cultural symbiogenesis or transposition which may ultimately spawn new forms, or a speciation of technique.
Applying Park’s “Rep & Rev” Technique to Technology
The playwright Suzan Lori Parks uses a method of mimesis that is inspired by the call and response dynamic. She uses her technique, called Repetition and Revision or “Rep & Rev”, to reflexively rewrite the personal and collective history of her characters, to what “could” or “should” have been, and in so doing evolves a new history in the memory of the audience. For her, the technique of Rep & Rev creates “a drama of accumulation”, so that her “characters refigure their words and through a refiguring of language show us that they experience their situation anew.” By doing so, they also have the capacity to “refigure the idea of forward progression” in drama, and change the historical canon. She also relates this to ghosts and possession, by fusing the past with the present in a symbiotic relationship. As she puts it, “The relationship between the possessor and possessed is, like ownership is, multidirectional.” I am looking to explore a similar relationship with the “ghost” in the machine by using software to technological repeat and revise the movement of a performer, so that it affects human memory and behavior. My goal is to create an improvised dance dialogue between the performer and the digital representation, so that the borders between the real and the virtual become blurred through perception of shared agency. I will base my application of this technique on my experience as a social dancer, and how my mind appears to run pattern recognition and random algorithms when improvising.
Embodying Kleist’s “Uber Marionette” in a Digital Puppet
Another theatre concept I am exploring is Kleist’s “uber marionette”, as personified by a software generated 3D character in virtual Cartesian space. This type of character is a true marionette, as the software rig controlling the character’s movements represents physical rigs used for puppets, complete with virtual strings and handles. However, a digital character is “uber” because it can perfectly mimic human movement through motion capture technology, or also go beyond anything a human or physical marionette could accomplish. This fusion of human movement with digital technology qualifies the 3D character to also be a species of cyborg that is both familiar and uncanny in its movements. The technique has been evolving in culture from at least the Renaissance period, when time was mechanized and perspectival space was conceptually modeled, and then replicated and revised through the visual mediums of drawing, painting, theatre, film, television, and the computer. Human anatomy, kinetics, and dynamic forces are all mathematically represented in the 3D character and the digital environment it inhabits. In addition, the model can be copied and manipulated until it is a perfect mimesis of the director’s thoughts.
As much as Kleist and many other director’s would probably like this much control, the main issue is that the performance may become the product of a single vision, which limits the spontaneity and creativity that can come out of collaboration. Free play or improvisation is usually encouraged in live theatre rehearsals and repetitive movie takes, which allows for “happy” accidents and performing “magic” to happen. As noted previously, such “magic” can be seen as a cultural mechanism for change, and usually emerges from interaction between the humans involved in the environment of the scene. Instructions related by the director may be mutated by a performer through unconscious mistakes in the representation of the director’s “vision”, or through purposeful creative acts that reflexively re-route the direction of the action. My projects explore programming such spontaneity into the 3D character’s performance, by code that reacts unpredictably, so that the end result is the illusion of improvisation. However, if the performer gets the feeling of improvisation and reacts to it, even if it is an illusion, the dynamic in some sense becomes real.
Viewing Mimesis as a Competitive Gaming Dynamic
The “magic mirror” effect of mimesis as presented here also creates game-like competition. In the mirror gags the main performer tries to “trick” the reflection into revealing its nature, while in a dance “battle” both performers must match their opponent’s movement while also surpassing it with original content. The reflexive desire to win in both situations drives the loop, like facing mirrors that produce infinitely fractal reflections, which in turn escalates and changes the reflection each round.
Additional Theatre Influences
Throughout this project I will also explore other theatre concepts, such as Aristotle’s ideas on comedy and tragedy (reversals, recognition, pity and fear, plot structure, etc…), Bergson’s ideas on comedy (mechanical), and Turner’s analysis of theatre as a ritual process (breach, redressive, liminal, etc…).
For instance, both projects can be viewed as comedies of the mechanical (with a hint of the uncanny), while at the same time are tragedies as we recognize that the human performer is ultimately going to lose in the game against the “uber marionette” or evolving cyborg, because we simply cannot evolve at the same rate as technology, either genetically or memetically.
Thesis Details: Using Motion Capture to Simulate the “Mirror Gag”
A library of thematically related theatre or dance movements would first be motion captured into computer memory and categorized. These clips would represent learned movements in a human performer’s memory, which would then be algorithmically blended into the live motion captured movement of the represented performer. For instance, in the case of the “magic mirror” mimesis, a mirrored version of the captured data will be applied to the 3D character with no delay, in a Cartesian space that represents the real space inhabited by the performer across a 2D plane that represents the mirror surface.
Minor variations will be created in the repeated movement by randomly blending small values into the captured data, so that it is barely noticeable to the performer. This is meant to catch the performer’s attention, and can be done in Motion Builder using custom Python scripts. In addition, major variations in the movement will be implemented when the performer turns away from the mirror a specified amount, causing the 3D character to move unexpectedly. This will also be done in Motion Builder through Python scripts by querying the head rotation data, and running conditional statements that retrieve a random pre-recorded motion clip. If the performer turns back towards the mirror, the character will resume following with only minor variations.
The performer will only be able to see the major variations out of the periphery of their vision, so it will hopefully create a perception of anthropomorphism in the performer to give agency to the character. A successful result would be for the performer to engage in a sense of play to “catch” the character moving behind their back. This project conceptually addresses relations of movement, memory and agency between humans and technology. In creating the illusion of time and space in a 3D animation, computer processes and memory can reflect human processes and memory, so that mimesis is achieved, and the controlling agent is blurred in the relationship.
A virtual camera will be set up in Motion Builder to frame the top half of the character, and coincide with the mirror surface. This camera view will be back-projected on a proportional screen that has a sheet of clear plexiglass mounted between the performer and screen, so the surface of the “mirror” can be touched by the performer. If using an optical motion capture system, cameras will be placed above the plexiglass, and around the performance space. For this project, only the upper half of the body, head, and hands need be captured, not requiring a full motion capture suit. Since this is an art installation in a gallery, and the performer is a gallery visitor, it is preferable not to use a spandex suit, which is hard to wear. A more practical solution is to use a custom fabricated jacket, hat, and gloves that are easy to wear.
Group Project Details: Using Motion Capture to Simulate a “Dance Battle”
Details coming soon…
Thesis and Group Project Milestones (Summer-Fall)
- Load Motion Builder 2009 to CS computer, then install the PhaseSpace-Motion Builder plugin, and use the PhaseSpace system to motion capture a performer’s movement directly to an actor in Motion Builder (MB). Thesis and Group.
- Use MB software or Python coding to apply the reversed Cartesian values of the captured motion data to another actor in realtime, so it mirrors the first actor, which should then be hidden. Thesis and Group.
- Use Python random functions in MB to code realtime variations in the mirror effect, by delaying the frame timing or slightly varying the marker values. This will cause random minor mistakes or micro-mutations in the repeated movement, such as a slight lag in the following, giving the impression that the following actor is capable of human error. Thesis and Group.
- Record a library of categorized slapstick sequences into memory, and use Python conditional and random functions to retrieve and then blend or layer the clips onto the following actor whenever the recorded actor turns his head away from positive Z in Cartesian space. These triggered random sequences or macro-mutations is meant to give the appearance that the following actor is intentionally improvising in direct response to the movement of the recorded performer. Thesis.
- Determine a hidden mirror plane in Cartesian space, and experiment with dynamic simulations and collisions between the two actors, and possibly with props. Thesis.
- Store a library of human and “robotic” dance sequences in computer memory. Use Python conditional functions to create a delay in the following actor, so it waits in a moving hold until the performer stops in a pose. When the data registers that the performer poses or holds for a short time, use Python random functions to have the following actor repeat part of the motion while blending in one of the pre-recorded clips, ending with another held pose. This echoing sequence of movements is meant to simulate the “call and response” or improvised dance dialogue between human dancers. Group.
|
|
