Human-in-the-loop Hybrid and Automated Pre-processing for Zero-shot Aerial Part-matching via Analogical Reasoning

Abstract

Accurate and efficient object identification under occluded imagery conditions remains a core challenge in common machine learning tasks such as image recognition and autonomous vehicle navigation. We introduce the Part-annotated Fine-Grained Visual Classification of Aircraft (“Part-annotated FGVC-A”) dataset consisting of over 2,700 images with labels for four to nine pre-identified airplane parts. We use the visual probabilistic analogy mapping (visiPAM) model to demonstrate two complementary data pre-processing pipelines for zero-shot part matching between different aircraft. First, a human-in-the-loop procedure achieves 64% accuracy but requires 204 hours of manual annotation. Then, we automate this pipeline using semantic segmentation and clustering, resulting in a 76.9% accuracy, 22% higher than the human-in-the-loop approach, while cutting pre-processing time by 97%. These results demonstrate the potential of analogical reasoning as a zero-shot solution for part-based identification tasks for various computer vision applications dealing with minimally-labeled or occluded data.