Because the phrase ‘by the penny’ is implied that whatever it is, will squeeze all the value out down to the singular digit, because the penny has a value of 1 cent as we all know. However, with the penny no longer in production, how soon can we expect a time where nobody will say “by the penny”?
I’ve heard that one as well as penny wise and pound foolish, penny for your thoughts, here’s my 2 cents, etc.
I’ve heard your other examples but not “by the penny”.