Religious affiliation is generally associated with better mental health. The nonreligious, however, currently constitute one of the fastest-growing religious categories in the United States. Since most of the nonreligious were raised in religious homes, their growth raises important questions about the mental health of those who consider dropping out of religion. In this article, I use longitudinal data from the Portraits of American Life Study to examine the impact of religious affiliation on mental health. Specifically, I compare individuals who dropped out of religion (leavers) with individuals who considered dropping out (stayers) and individuals who are more consistent in their religious (stable affiliates) and nonreligious (stable Nones) affiliations. I find that stayers experience more depressive symptoms than any other group and that they experience a greater increase in depressive symptoms over time. My findings are consistent with identity theories in sociology, and they provide evidence that a strong religious or secular identity is an important contributor to mental health.