Deep learning, a branch of machine learning, can serve as a powerful toolkit for studies in ecology and evolutionary biology. In the era of big data, gaining more insight into complex patterns within a dataset can be transformative for conducting comparative analyses. The field of comparative phylogenetics has become increasingly interested in exploiting massive datasets to uncover factors that have played a role in the macroevolutionary diversification of taxa. One such factor is the evolution of color pattern in fishes. With fish representing half of all vertebrates, and reef-fishes comprising one of the most diverse assemblages of vertebrates on Earth, mapping color pattern evolution onto the tree of life of fishes will enhance our current knowledge of their diversification through time. Yet, carefully curated datasets comprised of high-quality fish images with transparent backgrounds are required for accurately quantifying features of color and pattern within these taxa (Alfaro et al. 2019). A rate-limiting step in preparing these fish images for comparative analyses is the cost associated with accurately and consistently digitizing these fish images manually. We sought to implement robust deep learning models to more efficiently curate our datasets, however, a steep gap in the deep learning model space for performing high-quality continuous image segmentation exists as these popular models have been previously trained with common household objects in common contexts. We therefore present a deep learning pipeline to perform high-throughput phenoscaping by accurately generating high-quality, digitized fish images through continuous, instance segmentation using regions with convolutional neural networks (R-CNN). Our model extends previous work by Microsoft for large-scale object detection (Lin et al. 2015) while preserving essential morphological characteristics of the fish body necessary for comparative color pattern geometric analyses.

Authors: Shawn T. Schwartz*, Michael E. Alfaro, UCLA Department of Ecology and Evolutionary Biology