In 1965, the Brokaw Mansion, which stood at 1 East 79th Street, was destroyed to make room for this high-rise apartment building. The mansion was built during the years of 1887 and 1890 by Rose & Stone, for Isaac Vail Brokaw, who later built more houses nearby. To the east of the Brokaw Mansion, at 7 East 79th Street, was a building he designed as a wedding gift for his daughter Elvira. Later he built twin Gothic houses at 984 and 985 5th Avenue for his sons Howard and Irving.
According to this website, "the rooms of the Brokaw Mansion were huge and unusual for a house of that period, airy and well lit. The library had a seven-foot tall safe concealed behind a panel opened by pressing a hidden catch in the moldings." And that's not all, early on there was even a moat! It was enclosed with stone after a horse fell into it.
In 1946 the Institute of Radio Engineers (IRE) purchased the Brokaw Mansion (and later acquired two other related buildings)—it had been vacant for about 8 years at that point. Demolition of the buildings—by new owners Campagna Construction Corporation—began in 1965, and was started on a weekend so that officials wouldn't be able to stop it. Their destruction (along with that of Penn Station) played a significant role in advancing landmarks legislation in New York City. In 1962 the Landmarks Preservation Commission had already recognized the structures as landmark buildings—but there was no legislation to back up the Commission's authority.
The New York Times ran a scathing editorial called "Rape of the Brokaw Mansion" which "decried the ‘weekend stealth...[of] the despoilers’ who demolished these buildings and noted that if the city did not pass pending landmarks legislation there would be no landmarks left to save." In 1965, TIME ran an article called "The Gargoyle Snatcher," which shared an anecdote about Ivan Karp, founder of the Anonymous Arts Recovery Society. He allegedly offered to buy two copper finials perched on the roof of the mansion, but "was told by the wreckers that removing them with care was too dangerous and would slow up the job of razing the building. Said the sympathetic foreman, 'Sure it's a shame, but something should have been done about it before we got the job.'"
Bonus link: Here's an interview with someone who worked inside the mansion during the IRE years. In it he discusses what it was like inside, and recalls, "My office ended up to be what had been [George Tuttle Brokaw's wife] Clare Booth Luce's dressing-room. It was probably four times the size of my current office. It had an Italian marble fireplace in it. The safe that she kept her jewelry in became the petty cash safe."