How does aging affect facial attractiveness? We tested the hypothesis that people find older faces less attractive than younger faces, and furthermore, that these aging effects are modulated by the age and sex of the perceiver and by the specific kind of attractiveness judgment being made. Using empirical and computational network science methods, we confirmed that with increasing age, faces are perceived as less attractive. This effect was less pronounced in judgments made by older than younger and middle-aged perceivers, and more pronounced by men (especially for female faces) than women. Attractive older faces were perceived as elegant more than beautiful or gorgeous. Furthermore, network analyses revealed that older faces were more similar in attractiveness and were segregated from younger faces. These results indicate that perceivers tend to process older faces categorically when making attractiveness judgments. Attractiveness is not a monolithic construct. It varies by age, sex, and the dimensions of attractiveness being judged.