XROMM Tools for DeepLabCut Bringing deep learning to XROMM marker tracking


SOCIETY FOR INTEGRATIVE AND COMPARATIVE BIOLOGY
2021 VIRTUAL ANNUAL MEETING (VAM)
January 3 – Febuary 28, 2021

Meeting Abstract


P33-7  Sat Jan 2  XROMM Tools for DeepLabCut: Bringing deep learning to XROMM marker tracking Laurence-Chasen, JD*; Manafzadeh, AR; Hatsopoulos, NG; Ross, CF; Arce-McShane, FI; University of Chicago; Brown University; University of Chicago; University of Chicago; University of Chicago jdlc700@gmail.com

We present XROMM_DLCTools, an open-source workflow that integrates XMALab and DeepLabCut to dramatically increase XROMM (X-ray Reconstruction of Moving Morphology) marker tracking speed. The process resembles typical deep learning pipelines; the user tracks a subset of frames in XMALab, which are used to train a deep neural network with DeepLabCut. The network then predicts 2D marker locations, in both cameras, for novel frames. The predictions can be evaluated and corrected in XMALab, where final triangulation and rigid body reconstruction is performed. The workflow shows special promise for cyclic behaviors (e.g. chewing or treadmill locomotion) where range of motion is constrained and relatively few training frames are needed to capture the majority of the variation in the whole dataset. Our hope is that this new workflow will enable large-scale, multi-taxon studies that were previously precluded by the XROMM marker tracking bottleneck. We provide full instructions and code at github.com/jdlaurence/XROMM_DLCTools.

the Society for
Integrative &
Comparative
Biology