There is an important distinction here. You, I presume, have a ‘safe space’ of your own, your home, or even just your room if you share. That is your personal space. It makes sense that only those you trust should come in. However, when you put up a sign, it changes a space. A sign, such as one announcing a conference or symposium, even with a barrier like ticketed/preregistered entry, says it is a public event. Not a ‘safe space,’ but a space specifically for encountering and engaging with others, the public. The public is a group that is supposed to include everyone. Excluding people from that group effectively designates them as unpeople. If it were to be invite only, a private space, there would be no argument if, say, the invitees were all women, but the transition to a public event, combined with the discrimination based on unchosen characteristics creates the offense of sexist discrimination. It is one thing to demand relevance (e.g. no entering a feminist conference to shout about misandry) but it is another to treat everyone who, through no choice of their own, happens to be some type of person as an unperson. It’s prejudicial bigotry.