Have you seen Disney's 'Mulan' (the animated version at least - not sure about the live action version)?
The main character, Mulan, goes off to war dressed in armor, and everyone assumes Mulan is a man. But then they change their minds when they see Mulan's body, without armor on.
In other words, they used social queues as a heuristic for guessing that Mulan was a man, but they ultimately deferred to Mulan's biological body as the final source of truth, and decided that she should be thought of as a woman in the end.
Does that make sense? Yes, we use social behavior as a clue to a person's gender, but that is usually, historically, ultimately in lieu of more detailed information about their body.
I think we could probably go back and forth about this all day, but do at least understand why, to me, it seems like biology is the fundamental basis for gender, and why I'm having a hard time deferring to your argument?