You don't need to tell me Hz are not everything. But still, it matters. A lot. And going from PII 266 MHz -> P4 2.0 GHz we did see an IPC increase too. In addition to an 8.6x CPU frequency increase.
Going from Yorkfield->IB we've perhaps seen more IPC increase. But only 2.83 GHz (Q9550) -> 3.5 GHz (3770K) => 1.23x frequency increase. So the IPC increase going from Yorkfield->IB is nowhere near enough to compensate for the higher frequency increase we saw going from PII->P4.
To sum it up: Show me benchmarks where the relative CPU performance increase going from PII 266 MHz (1997) -> P4 2.0 GHz (2001) is lower compared to going from Q9550 (2007/2008) -> 3770K (2012), then I'll believe you.
There is a Q9650 that runs at 3.0 Ghz.
http://www.anandtech.com/bench/Product/49?vs=551
Q9650(3.0 Ghz) vs. i7-3770k(3.5 Ghz)
Performance is between 1.5 times to 2 times the performance of the Q9650, all while having a measly-looking clock increase of 500 Mhz.
http://www.anandtech.com/bench/Product/49?vs=701
Q9650 to i7-3570k
I use the 3570k as well because Hyperthreading can affect IPC.
Since we're considering 4-year periods, then there is the Pentium II -> Willamette Pentium 4s and the Coppermine Pentium III -> Northwood Pentium 4s.
Pentium II benchmarks are hard to find, but the following at least compares some vintage CPUs to the top-of-the-line Northwood released on November 25, 2002.
http://www.tomshardware.com/reviews/benchmark-marathon,590.html
This is the closest I can find. Too bad the only Pentium IIs they had were Celeron Mendocinos, it seems.
But if it's a Mendocino vs. the Athlon XP 2100+, then see an increase of approximately 7 times that of the Mendocino at its lowest clock.
There does seem to be a pattern of diminishing returns with regards to clockspeed increase though. After accounting for Coppermine's IPC increase, the performance gain is 4 times that of the Mendocino for only 600 Mhz for the MP3 Maker Platinum. Even having a clockspeed jump of 2.0 GHz only triples the performance compared to the Coppermine at 1.0 Ghz, even with Hyperthreading.
Comparing the Mendocino to the Pentium 4 2.0A, the performance gain is about five times that of the Mendocino 400 Mhz.