While it is safe to remove the protective eyeware only during totality, the moment the sun peaks out from behind the moon it can again cause irreversible eye damage.
Prevention is key: When in doubt, keep the glasses on for the entire solar eclipse to avoid damaging the eye.