I'd educate yourself on the specific issue by reading this article (to start):
http://blog.philadelphialuthiertool...ing-the-correct-knob-for-your-guitar-or-bass/
As for why, it's because there is enough demand for various type of pots that it is profitable to produce them. Different companies and different projects call for different specs, and use different systems of measurement. The why of this is a long and frustrating story.....but you have to learn to work with it, because it's probably not going to change.
You know which is which because you read the specifications of the parts you are using, and/or you figure out that if something from one system doesn't quite fit, then the right part must be from the other system. Knowledge gained through experience will also allow you to see the differences with your eyeballs. You can usually tell very quickly whether a pot is metric or customary simply by looking at it.
Contrary to your statement, all pickups do not adhere to a single standard. They come in annoyingly various sizes, in
both commonly used systems of measurement. Off the top of my head, I can think of having read pickup polepiece spacing specs that state (in millimeters) 48 (DiMarzio), 49 (Duncan), 49.2 (Gibson), 50 (various imports and Suhr), 51 (DiMarzio), 52 (various imports), 52.4 (Gibson), 52.6 (Duncan), and 53 (various imports and Suhr), as well as (in inches) 1.915 (DiMarzio), 1.93 (Duncan), 1-15/16 (Gibson), 2.01 (DiMarzio), 2.07 (Duncan), and 2-1/16 (Gibson). Confusing things more, some of those are conversions of each other, and which you read depends on where you're reading it. This is clearly far from a single standard, as you stated was in place.