No, a USC spouse cannot withdraw the I-485 because it’s not their petition, it’s the immigrant’s petition.
However, a USC can withdraw the I-864 affidavit of support and if that happens before the interview/approval, then the I-485 will be toast (=denied). And no, you can’t prevent a USC from withdrawing I-864 if they so wish.