Reinforcement learning based simulation of ostrich locomotion using a whole-body model


SOCIETY FOR INTEGRATIVE AND COMPARATIVE BIOLOGY
2021 VIRTUAL ANNUAL MEETING (VAM)
January 3 – Febuary 28, 2021

Meeting Abstract


P41-2  Sat Jan 2  Reinforcement learning based simulation of ostrich locomotion using a whole-body model La Barbera, V*; Tassa, Y; Richards, C; Daley, M; Hutchinson, J; Royal Veterinary College; DeepMind; Royal Veterinary College; University of California Irvine; Royal Veterinary College vlabarbera@rvc.ac.uk

To achieve agile and efficient locomotion, animals must integrate neural control with the physical and physiological properties of the musculoskeletal system, subject to the dynamic constraints of the environment. Ostriches are the fastest bird on land, yet achieve this impressive performance with slender lower limbs and only one major toe. Prior research has studied hindlimb biomechanics in detail but the question remains: how are head/neck and wing motions are integrated with these in complex three-dimensional and unsteady behaviours? Furthermore, some robots resemble ostriches’ morphology; for example, the robot Cassie produced by Agility Robotics. Here we build a complete musculoskeletal model of an ostrich and use it to analyse locomotor dynamics. The model has been assembled in the MuJoCo physics engine, which provides fast computations for physics simulations. In recent years there have been rapid advances in solving complex dynamical simulations using Reinforcement Learning, even involving muscle simulations in the NeurIPS challenges. Using this kind of simulation, another question arises: what is the relationship between musculoskeletal morphology and the learning process? This study provides a complete open source model of the major components of a whole ostrich’s musculoskeletal system. We present our preliminary results from control simulations based on Reinforcement Learning (e.g. normal walking/running; obstacle course maneuvering) which have direct applications in robotics and comparative neuromechanics.

the Society for
Integrative &
Comparative
Biology