There’s one elite group propping up the Manhattan real estate market right now — everyone else is sitting on their hands