It depends on the type of fish, their age, --- ratio in the wild population, and probably all sorts of other things.
In short, it is not often easy to determine. Sometimes there are color differences, size differences, etc...but other times, or if they are immature, it is tough to tell.