This is only partially true.

RSA public keys consist of two parts:

1. The exponent e, which is usually 65537, for the reasons you described.
2. The modulus n, the product of the two primes used for generating an RSA keypair.

If the public key were always completely the same, then there would be no point in making different private keys.