Institutional care has been shown to increase the risk of attention problems in children, but some children are more sensitive to their environment, both for better and for worse. With this in mind, the current study examined the moderating role of temperament (falling reactivity) between early adversity and attention skills. Six- to 15-month-old infants residing in institutions (n = 63) and infants reared by their biological families from low socioeconomic environments (n = 59) were recruited. The infants' attention skills were measured by calculating the length of time they spent looking at toys. The infants' temperaments were measured by a subscale of the Infant Behavior Questionnaire (falling reactivity/rate of recovery from distress). The findings were in line with the differential susceptibility theory. Compared to infants with high levels of falling reactivity, infants with lower levels of falling reactivity had better attention skills if they were in a family group, but they had lower attention skills if they were residing in institutions. The attention skills of the infants who had higher scores for falling reactivity did not appear to be affected by the adverse environment.