Cross-condition comparisons on neurodevelopmental conditions are central in neurodiversity research. In the realm of visual perception, the performance of participants with different category-specific disorders such as developmental prosopagnosia (problems with faces) and dyslexia (problems with words) have contributed to understanding of perceptual processes involved in word and face recognition. Alterations in face and word recognition are present in several neurodiverse populations, and improved knowledge about their relationship may increase our understanding of this variability of impairment. The present study investigates organizing principles of visual object processing and their implications for developmental disorders of recognition. Some accounts suggest that distinct mechanisms are responsible for recognizing objects of different categories, while others propose that categories share or even compete for cortical resources. We took an individual differences approach to estimate the relationship between abilities in recognition. Neurotypical participants (N = 97 after outlier exclusion) performed a match-to-sample task with faces, houses, and pseudowords. Either individual features or feature configurations were manipulated. To estimate the separability of visual recognition mechanisms, we used representational similarity analysis (RSA) where correlational matrices for accuracy were compared to predicted data patterns. Recognition abilities separated into face recognition on one hand and house/pseudoword recognition on the other, indicating that face recognition may rely on relatively selective mechanisms in neurotypicals. We also found evidence for a general visual object recognition mechanism, while some combinations of category (faces, houses, words) and processing type (featural, configural) likely rely on additional mechanisms. Developmental conditions may therefore reflect combinations of impaired and intact aspects of specific and general visual object recognition mechanisms, where featural and configural processes for one object category separate from the featural or configural processing of another. More generally, RSA is a promising approach for advancing understanding of neurodiversity, including shared aspects and distinctions between neurodevelopmental conditions of visual recognition.