Difficulty recognizing negative emotions (NEs) in children is linked to increased antisocial traits and externalizing problems. However, crucial aspects of this relation remain unclear, such as: whether NE recognition is associated with externalizing problems in general or only a particular subcomponent (i.e., aggression); whether subcomponents of NE recognition (i.e., insensitivity and misspecifications) are relatively more important; and how these relations change over the course of development. We assessed emotion recognition, overt aggression, attention deficit hyperactivity disorder (ADHD) symptoms, and oppositional defiant disorder (ODD) symptoms in an ethnically diverse sample of Canadian children (N = 150; 4-year-olds, N = 148; 8-year-olds) and followed up with them 1 year later (86.9% retention). Emotion recognition was assessed using a behavioral task and caregivers reported on children's externalizing symptoms. Children with lower NE recognition had higher initial, but not subsequent, overt aggression, even when controlling for nonaggressive externalizing symptoms (i.e., ADHD and ODD symptoms). NE recognition was not concurrently or longitudinally associated with nonaggressive externalizing symptoms. Age and gender did not moderate these findings. Both higher NE insensitivity (e.g., reporting a sad face appears neutral) and misspecifications (e.g., reporting a sad face appears angry) were significantly associated with higher concurrent overt aggression. In conclusion, both NE insensitivity and misspecifications were found to be uniquely important for children's overt aggression. These findings highlight the importance of different forms of NE recognition and differentiating between aggressive and nonaggressive externalizing problems in children.