This essay analyzes racialized exclusions in sociology through a focus on sociology’s deep epistemic structures. These structures dictate what counts as social scientific knowledge and who can produce it. A historical analysis of their emergence and persistence reveals their connections to empire. Due to sociology’s initial emergence within the culture of American imperialism, early sociological thought embedded the culture of empire’s exclusionary logics. Sociology’s epistemic structures were inextricably racialized, contributing to exclusionary modes of thought and practice along the lines of race, ethnicity, and social geography that persist into the present. Overcoming this racialized inequality requires problematizing and unsettling these epistemic structures by (1) provincializing the canon to create a transformative epistemic pluralism and (2) reconsidering common conceptions of what counts as “theory” in the first place.