Focus, what do you mean the piece that come upwards, the big old tower looking thing that the armrest/storage piece screws onto? Because that thing does come off, I've done it.
Because if it didn't come out then he couldn't just swap a non-armrest console into the space... I agree that's the way to go, 'glassing the hole where the tower used to come through would work too, but sounds much more labor-intensive. You could also try to find someone without an armrest that wants one, and trade.