On Madison Avenue, no less. Further research has revealed this unexpected edifice to be the façade of an old armory built in 1894, so it never saw any actual battles, but nonetheless you can imagine my surprise when I came across it during my futile Friday afternoon quest to locate a coffee shop that was not a Starbucks but that nonetheless encouraged use of their electrical outlets. [As a side note, those wishing to hop into the coffee shop game might want to consider this as a selling point, since I ended up overpaying for coffee that I knew I would not enjoy just to get some laptop juice out of the deal.]
According to The City Review, the Squadron A Armory, which is between 94th and 95th Streets, was built to resemble "a fourteenth-century French fortress, complete with square towers, round turrets and a crenellated parapet," but it never really served a…fortifying…purpose. The New York Times explains that Squadron A (a division of the National Guard) only ever fought "in Puerto Rico, in France and along the Mexican border."
The actual armory building mostly served (per NYT) as an "immense riding ring with elaborate social quarters, which it used for polo, dances, badminton and horse shows." In 1969, most of the armory was torn down to build the Hunter College Elementary and High School, but the Madison Avenue façade was declared a historical landmark right before it was destroyed, and now protects the playground behind it (The City Review has an interesting shot of the playground, towards the middle of their article on the armory).