🖙Statements about Software Speed and Quality are Still Vibes, But Here's Some Data
I'm Alys, reporting for Alys+, live from the programming discourse
Phoronix benchmarked a number of recent Firefox releases. (Hat-tip to The Register's Liam Proven, who linked to it in a good piece on Mozilla's issues). Their benchmarks found Firefox got significantly faster and less memory-heavy over the past 21 releases, which sounds like it covers a ton of time until you remember how often Firefox releases (it only covers through 2023). So this doesn't rule out the possibility that Firefox has experience a huge drop in performance since Firefox 4 or Firefox 2.0 or Firefox beta or Firebird or whatever that has only recently been partly corrected, but it does offer hard data about recent performance.
I did notice that a 2019 Phoronix article also benchmarked Firefox using one of the same benchmarks, albeit on a different system. While definitely not an apples-to-apples comparison, it's interesting. Firefox 70 scored in the 40 ms range on ARES-6, where as Firefox 140 scored under 10 ms. This definitely doesn't prove Firefox hasn't gotten slower; it could be all hardware, especially if ARES-6 is highly parallelizable. What it does show is that Firefox is not getting so much slower that we're getting worse or even stagnant performance. Using the highly imprecise metric of Passmark scores, the CPU used on the newest test has double the scores of the one used on the earlier test. So significantly faster, but not the 4X improvement. That suggests Firefox might have actually gotten faster in this time but without testing on the exact same hardware, it's tough to be sure.
Overall, I consider this evidence for my assessment that software performance over time is highly dependent on the particular software you're measuring and isn't reducible to a simple narrative about a broad slide downwards.
Back to you in the studio, Alys.