In this work, we address the problem of trait segmentation for fishes, where given an image of a fish specimen, our goal is to annotate each trait in that image with a unique color. Trait segmentation has been traditionally addressed by manual annotation on each species-image, which requires expert knowledge on trait anatomy, ontology, and phylogeny, and is slow and unscalable to a large dataset of biodiversity images that we have recently gathered from several museums. One possible solution to automate trait segmentation is to use artificial neural networks (ANN), which can detect non-linear patterns from any image. To leverage ANN, we formulate the trait segmentation problem as a semantic segmentation problem where we annotate each pixel of an input image according to the anatomical trait-class. However, a black-box ANN model learns solely from training samples and requires a lot of annotated observations. Moreover, black-box ANN ignores external biological knowledge in the training phase, which sometimes results in inconsistent outputs. To address these challenges, we develop a novel approach that incorporates biological knowledge into the black-box ANN model. In particular, we extract the inter-trait relationships from the fish-ontology as biological constraints and add a penalty term in the loss function of the ANN for each time a constraint gets violated. This forces the ANN model to train its parameters such that they follow the biological relationships in its prediction. Experimental results demonstrate that using biological knowledge guidance helps us to learn the ANN model from a much smaller number of annotated samples (~400 training samples for our experiments).