It seems that
var guid = Guid.Empty;
var hr = Session.RemoveAllSetExceptions(ref guid);
resets state of MDAs (and potentially other not-.NET exceptions?).
Since setting exceptions without RemoveAllSetExceptions does not work for some mysterious VS/COM reason, I'll have to record state of MDA, and restore it after the Remove call.