I’d guess “To whom it may concern” would be the most gender neutral/inclusive way to rephrase that.
What is your issue with DEI?
I mean some of them did, probably. It’s a diverse group, can’t really lump everyone together outside of being not straight and/or not cis.
Do you care if others see you as an asshole? Remove morality and other social obligations from it, if you don’t like someone, you don’t like someone. If spending time with this person upsets you, don’t. Hell, even if you force yourself to go there’s a chance he’ll get under your skin and you’ll end up causing a problem.