In the two weeks since white supremacist Dylann Roof allegedly murdered nine people in a South Carolina church, activists, politicians, and everyday anti-racists have taken up the cause of removing Confederate flags and monuments from the public landscape. The campaign is a noble one—the flag was, after all, created explicitly for Southern white people to start a war to preserve slavery, and Roof posed with it in photos before reportedly trying to incite his own race war—but for some liberal New Yorkers, it has served as a self-congratulatory reminder that the South is a uniquely racist place with a disgusting past that has nothing to do with them.

Of course, the problem with this line of thinking is that the whole country was built on slavery, not just the South, and we have yet to deal with its horrors on anything approaching the way, for instance, South Africa has reckoned with apartheid, or Germany has atoned for the Holocaust. Today, Americans can visit a national Holocaust museum in their own country, but not a national slavery museum.

An 1895 Harper's New Monthly article described slavery as "part and parcel of [colonial New York's] economic organization," and at one time New York was home to the most slaves of any city except Charleston. Yet, it wasn't until last month that Wall Street got a marker memorializing the municipal slave market where, from 1711 to 1762 prospective buyers made enslaved men flex to assess their fortitude, and grabbed women's genitals when considering a purchase. The African Burial Ground near City Hall didn't get a memorial until 2010, and it almost didn't get one at all because the developer building a tower on top of it tried to breeze through the regulatory process without acknowledging the site's historical significance. Few other reminders of this past exist in New York.

Monuments to prominent slaveowners from New York's history, on the other hand, are plentiful, though they're seldom presented as such. For example:

Delancey Street (Joanna_Pan/Flickr)

James De Lancey (1703-1760), lieutenant Governor of New York and provincial Supreme Court judge. Delancey Street is named after him.

He presided over the 1741 trials of dozens accused in a supposed slave revolt plot. One hundred fifty-two enslaved and free people of African ancestry, plus a handful of Europeans, were arrested, of whom 13 black men were burned at the stake, 21 people hanged, 84 sold into slavery, and seven exiled from New York. De Lancey's slave Othello was among those hanged. Historians now doubt that a plot existed in the first place.

De Lancey is not to be confused with Westchester County Sheriff James De Lancey (1746-1804), owner of at least six slaves, who famously sued a man in Nova Scotia, where slavery wasn't allowed, to be reimbursed for wages earned by his former slave, Jack, who had escaped north.

The Chambers Street subway station (Daniel Krieger/Flickr)

John Chambers (1710-1764), New York alderman and provincial Supreme Court judge. Chambers Street is named after him.

He also presided over the 1741 slave revolt trials, including that of his slave Robin. Robin was burned at the stake.

(Public domain)

Cadwallader Colden (1688-1776), colonial governor. PS 214 the Cadwallader Colden School in Flushing is named after him.

He sold his slave Gabriel to keep him apart from the enslaved woman Gabriel was in love with. He also sold a 33-year-old woman and one of her children to someone in Barbados, because to allow her to be near the rest of her enslaved children would "spoil them."

Pierre Van Cortlandt (Public domain)

The Van Cortlandts. Van Cortlandt Park is named after Stephanus Van Cortlandt, and the Westchester towns of Cortlandt and Cortland Manor are named after the family.

Olof Van Cortlandt came to the city in 1637 as an employee of the Dutch West India Company, a leading African slave trader that brought the first enslaved Africans to New Amsterdam, as the city was then called. His son Stephanus was New Amsterdam's first locally born mayor. Stephanus's grandson Pierre brought slaves to staff the family manor in what's now Croton-on-Hudson. The family owned vast tracts of land in what is now Westchester County, and operated sugar refineries that processed slave-picked sugarcane.

New York state abolished slavery in 1827, but trade in slave-made goods remained a staple of the local economy and wealthy and powerful city leaders continued to reap huge profits from the institution. They included:

Citibank in Chinatown (Wikimedia Commons)

Moses Taylor, founder of the bank that is now Citibank, who traded in slave sugar.

The former Domino Sugar factory (Edward Crimmins/Flickr)

William Havemeyer owned what came to be called the Domino Sugar factory, which refined slave sugar cane and molasses. Havemeyer Street in Brooklyn is named after him. (Last year, artist Kara Walker created a critically lauded installation, A Subtlety, inside the old factory that commented on the sugar trade and slavery.)

John Jacob Astor made his fortune trading fur, slave cotton, and for a time, opium. The Waldorf-Astoria hotel chain, Astor Place, Astoria, Queens, and a host of other New York places are named after his family.

Emmanuel Lehman (Public domain)

The Lehman brothers Henry, Mayer, and Emanuel founded Lehman Brothers in 1850 as a slave-cotton brokerage in Alabama, then moved it to New York in 1870, post-emancipation.

Their investment firm famously folded during the 2008 financial collapse, but Lehman College, named for Mayer's son Herbert, a New York governor and U.S. senator, remains in the Bronx.

Charles L. Tiffany founded Tiffany & Co. in 1837 with cash from his father, owner of a Connecticut cotton mill.

(Greg Wass/Flickr)

The bank J.P. Morgan Chase, founded in 1799 by Aaron Burr as the Bank of Manhattan Co. (PDF), seized 1,250 slaves from plantation owners over debts, and between 1831 and 1865, did deals in which 13,000 slaves were used as collateral.

Still don't see it? How about this: as a commenter notes, New York itself is named after England's James II, Duke of York, who headed the Royal African Company, formed specifically to supply Virginia planters with captive labor. We could keep going. And as The Root wrote in an article on the subject:

The very name "Wall Street" is born of slavery, with enslaved Africans building a wall in 1653 to protect Dutch settlers from Indian raids. This walkway and wooden fence, made up of pointed logs and running river to river, later was known as Wall Street, the home of world finance. Enslaved and free Africans were largely responsible for the construction of the early city, first by clearing land, then by building a fort, mills, bridges, stone houses, the first city hall, the docks, the city prison, Dutch and English churches, the city hospital and Fraunces Tavern. At the corner of Wall Street and Broadway, they helped erect Trinity Church.

We should of course acknowledge that New York came to be a hotbed of abolitionism even as conservative slaveowners and profiteers continued to hold sway. Names of abolitionists of European descent like Robert Murray and Lewis and Arthur Tappan live on in the New York landscape. Enslaved and free people of African descent struggled for liberation in New York, too, as they did throughout the South and the Americas. The Weeksville Heritage Center in Brooklyn, preserving part of the 19th-century free African American community of Weeksville, serves as a potent acknowledgment of that.

Still, we may not think about slavery as we move through New York in our day-to-day lives, and there are few reminders, but tributes to its beneficiaries are inscribed all around us.