Studies of object similarity have focused on the relationship between different physical objects and their mental representations or between instances of the same physical object and its mental representation. The present study is the first to investigate the structure of within-category psychological space. We provided evidence that large objects and frequently mentioned objects are perceived as less similar to each other compared to small objects or less frequently mentioned objects. Further, similarity judgments were higher for manipulable objects compared to non-manipulable objects. The relevance of these data to the isomorphism between physical and psychological spaces is also discussed.