This study examined the roles of verbal and nonverbal sources of information in the ability of persons with and without autism to recognize emotion. Child, adolescent, and young adult participants in four groups [Lower Functioning Autism (LFA) (n = 17), High Functioning Autism (HFA) (n = 18), Lower Functioning Comparison (LFC) (n = 18), and High Functioning Comparison (HFC) (n = 23)] identified emotions shown (happy, angry, sad, surprised, or neutral) in video clips of individuals expressing emotion verbally, nonverbally, or both. Verbal expressions of emotion were either Explicit, Implicit, or Neutral, whereas nonverbal expressions were Animate or Flat (3 × 2). Pairwise ANCOVAs indicated no group differences between HFA and HFC groups or between the LFA and LFC groups, and indicated instead group differences between higher and lower functioning persons. With groups collapsed into High Functioning (HF) and Lower Functioning (LF), significant group differences were found. Performance of LF individuals suggested they had difficulty inferring how a person felt based on what the person said, if the emotion was not explicitly named. Performance of HF individuals suggested they relied more on nonverbal than on verbal information to determine a speaker's emotion, except where the emotion was explicitly named. Results suggested that persons with autistic spectrum disorders can use affective information from multiple sources in much the same ways as persons of comparable developmental level without autism.