There is much scholarly and public debate over how slavery should be remembered, especially in the southern United States. We have seen this recently with the case of Charlottesville, Virginia, where protest ensued over a statue of Robert E. Lee. However, attention should also be paid to the history of slavery in the northern United States, particularly in places such as New England, where attempts were made to silence this history. The author analyzes the archives of the Royall House Association to study the historic preservation and presentation of the Isaac Royall House and slave quarters in Medford, Massachusetts. Royall and his connections to Harvard University have recently gained more attention with nationwide protests against racism on college campuses. Drawing on critical race theory, research on racism denial, and scholarship on history and memory, the author analyzes public narratives on how the Royall House should be remembered. The author teases out the various purposes battles over public memory serve and what their outcomes are and addresses whether these contests are largely symbolic or if they play an important role in dismantling racism. The author finds three main outcomes of these memory projects: to make invisible a racist past and uphold white supremacy, to serve as an illusion of progress with an attempt to assuage guilt, and to make an impact against racism.