This has been a problem forever, the googleization of CS where everything is assumed to scale to gigabytes and therefore all that matters is big-O.
In systems that's meaningless, what really matters is memory locality, loop placement, caching/lookaside and other features.
The JDk is an excellent example of both large scale and small scale optimization, the GC systems and much of the low-level features like locking use microoptimizations while the higher order data structure features use algorithmic optimizations.