Because the phrase ‘by the penny’ is implied that whatever it is, will squeeze all the value out down to the singular digit, because the penny has a value of 1 cent as we all know. However, with the penny no longer in production, how soon can we expect a time where nobody will say “by the penny”?
Probably two or three generations after those who actually handled pennies, but then again it wouldn’t be the first time an old phrase or usage fossilised and remained in a language long after its full meaning has been obscured by time.
One example that springs to mind is the “ut” of “utmost”. It’s an old spelling of “out”, but the word has stuck around because “out(er)most” means something else.