Abstract: There is much interest in studying animal personalities but considerable debate as to how to define and evaluate them. We assessed the utility of one proposed framework while studying personality in terrestrial hermit crabs (Coenobita clypeatus). We recorded the latency of individuals to emerge from their shells over multiple trials in four unique manipulations. We used the specific testing situations within these manipulations to define two temperament categories (shyness-boldness and exploration-avoidance). Our results identified individual behavioral consistency (i.e., personality) across repeated trials of the same situations, within both categories. Additionally, we found correlations between behaviors across contexts (traits) that suggested that the crabs had behavioral syndromes. While we found some correlations between behaviors that are supposed to measure the same temperament trait, these correlations were not inevitable. Furthermore, a principal component analysis (PCA) of our data revealed new relationships between behaviors and provided the foundation for an alternate interpretation: measured behaviors may be situation-specific, and may not reflect general personality traits at all. These results suggest that more attention must be placed on how we infer personalities from standardized methods, and that we must be careful to not force our data to fit our frameworks.