

Generally, women are going to have boobs and wider hips and men generally have bigger feet. Obviously there’s a massive spectrum of human body shapes, but this is the main differentiation.
If I wear my (male) partner’s t-shirts, they’re just straight with no space for boobs and they don’t come in at the waist. Same with trousers, there’s a massive pouch in the crotch, too narrow on my hips and too loose at the waist. People like to wear things that best fit their bodies.
It isn’t about style. Women’s sizes (e.g. 8, 10, 12) originally came from codes based on the variations between hip, waist and bust measurements.