Day 5: July 19, 2010

Activities:

  1. Discussions
  2. Began assembling the robotic 'arm' from purchased components (Jim Ruxton)
  3. Commenced programming of control application for robotic components in PureData (Jim Ruxton)
  4. Motion Capture Session (@ CAFA Mocap Lab, 4 hours)

Discussions:

  • Robotic 'head' mechanism:
    • Jim Ruxton performed basic demonstration of the linear actuator component (purchased at electronics market in Zhong Guan Cun district)
    • Servo mechanism would consist of two servo motors mounted perpendicularly to eachother, enabling a large degree of rotational movement in two dimensions
    • Projector would be mounted on the protruding end of the robotic 'head'
  • Robot movement/ behavior:
    • Would be able to move around a space and project images/ videos on screen(s)
    • Projects video from the Beijing environment, up and down the wall or across a series of screens - movement w ould correspond to panning and tilting shots filmed in the physical environment
    • Possible projection surfaces: walls, screens, scrims, people's bodies
    • Should allow movement along various axes and trajectories
    • Camera could be mounted onto the robot's 'head' to enable motion tracking of users in the space
    • Projection space could be divided into various zones, and different images/ videos could be shown depending on the zone currently being projected upon
    • Experiment with use of a grip stand, to securely hold the robotic components
  • Planning for the Motion Capture session:
    • Discussed the types of physical motions/ gestures used by factory workers during their work, and associated postures (e.g. seating or standing position)
    • Referred to early chronophotography by Etienne-Jules Marey, which studied the sequence (narrative) of human bodily movement while performing various tasks
    • Spectral Bodies - 19th Century Meat Packing Plant:
      Study of bodily movements of butchers as they passed meat carcasses down a 'disassembly line'
      http://www.readingpictures.com/slideshow.html
    • Sketched typical motions employed in an assembly line, using found video documentation as reference:
      http://v.youku.com/v_show/id_XMzUwNDI3MTI=.html
      http://video.sina.com.cn/v/b/28653462-1664117624.html
    • Wildlife (2009), SNIFF (2010), by Karolina Sobecka:
      Artworks that employs projection of motion-captured animals (rendered in Maya/ Unity 3D)
      http://vimeo.com/6400445
      http://vimeo.com/6400266
    • Confirmed that motion capture suit cannot record movement of individual finger joints - instead, a motion capture glove would be required - ShapeHand (from Measurand), and 5DT Data Glove (from MetaMotion)
  • Factory-related (assembly line) gestures:
    Stamping, cutting, ramming, moving widgets from one location to another, stirring/mixing, screwing, lifting, dropping
  • Farming gestures:
    Threshing, picking up objects, cutting, lifting heavy loads

Motion Capture Session (4 hrs):

We recorded physical gestures which were predominantly related to labour (factory and construction work, and gym park exercise). These recordings will be converted into BVH files, which we will import into Second Life to create avatar animations.

Gestures performed by Judith Doyle:*
1. Tubes and Levers
2. Tubes and Levers 2
3. Sowing Materials (Standing)
4. Sowing Materials (Sitting)
5. Screwing on Jar Lids
6. Farming - Take 1 (without props)
7. Farming - Take 2 (without props)
8. Farming - Take 3 (with wooden stick as prop)
9. Exercise ('Elliptical Trainer')
10. Exercise ('Pulling)
11. Exercise ('Bicep Curl')
12. Exercise ('Chest Crunches')

Gestures performed by Fei Jun:*
1. Bricklaying - Take 1
2. Bricklaying - Take 2
3. Sawing Wood - Take 1 (no prop)
4. Sawing Wood - Take 2 (using stick as prop)
5. Sawing Wood - Take 3 (using table as prop)
6. Hammering Nails - Take 1
7. Hammering Nails - Take 2 (using stick as prop for 'wall')
8. Walking with a load on his back
9. Walking with a load of his back - Take 2
10. Walking with a load on his back - Take 3
11. Threshing - Take 1
12. Threshing - Take 2
13. Tilling the ground
14. Picking up objects from the ground
15. Washing Clothes
16. Sweeping - Take 1
17. Sweeping - Take 2
18. Sweeping - Take 3
19. Operating Crane
20. Smoking in crouched position - Take 1
21. Smoking in crouched position - Take 1

* Listed in the order in which they were performed

View images of the Motion Capture Session

Second Life Activities:

  • Test BVH files (from previous motion capture of Tai Chi movements) were supplied by Wang Li Min, and imported successfully by Ken Leung into Second Life 'Gesture Warden' avatar (as Animations)
  • More research required on:
    • How to loop avatar animations
    • Whether BVH files can be applied to objects (e.g. animals) as well as avatars
    • How animations can be strung together in sequence (e.g., via scripts)

References:

Projects:

Kronografije moza, ki hodi in tece, ter ptice v letu (1880)
Etienne-Jules Marey
http://www.kiberpipa.org/gallery/album82/Etienne_Jules_Marey_Kronografije_mo_a_ki_hodi_in_te_e_ter_ptice_v_letu_1880.jpg
http://en.wikipedia.org/wiki/%C3%89tienne-Jules_Marey

Spectral Bodies (2003)
Judith Doyle
http://www.readingpictures.com/slideshow.html

Wildlife (2009)
Karolina Sobecka
http://vimeo.com/6400445

SNIFF (2010)
Karolina Sobecka
http://vimeo.com/6400266

Videos:

Assembly Line Videos:
http://v.youku.com/v_show/id_XMzUwNDI3MTI=.html
http://video.sina.com.cn/v/b/28653462-1664117624.html

Motion Capture Gloves:

ShapeHand (Measurand)
http://www.finger-motion-capture.com/

5DT Data Gloves (MetaMotion):
http://www.metamotion.com/motion-capture-hardware-gloves-Datagloves.htm

Contacts:

Shang Weng
Teaching Assistant, Central Academy of Fine Arts
Friend of Fei Jun

Ding Dong
Student of Fei Jun
(Assisted us with various tasks and photo documentation)

Wang Limin
CAFA Digital Media Lab
(Helped to create BVH files for us - initial test BVH files, and final BVH's from MOCAP sessions)

Tian Yue
CAFA Digital Media Lab
yuetian.cn@gmail.com
(Operated motion capture software during the mocap session)