Moore's Law: Moving the Goalposts?
This posting at NextBigFuture has a fantastic set of slides from Intel about the "long" run, up to 2029 (which will still be before I retire, even optimistically).
There is a lot of material for thought in there, but the big picture is that they are shooting for a million-fold improvement in FLOPS applied to a single problem by 2029, to about a zettaFLOPS, 10^21 FLOPS. I haven't been through the numbers yet, but that seems plausible if you postulate a 1000x improvement in VLSI density (which is around the upper bound where you're building out of individual atoms), one to two orders of magnitude improvement in clock cycle, and make up the rest in increasing parallelism. Overall, it seems to be about the upper bound postulated by deBenedictis, but a detailed check (of both) is in order.
As the article notes, communication is the big deal, all throughout the system. I refer to it as, quite directly, a problem in special relativity. Networks exist both inside and outside the chip, making the physical boundary almost meaningless. This is part of what we are researching as an "All-IP Computer" here in the Murai Lab. (Sorry, the web pages are undergoing revision right now, and you might run into some rough spots; noticeably, the All-IP project doesn't have a web page at the moment...)
At any rate, Intel seems intent on keeping the goalposts as far away as possible, and continuing to grind downfield, a few yards at a time, so that progress over years is astonishing. Leapfrogging their progress is probably impossible, but I hope to be sitting by the side of the road (to mix a few metaphors) with a road sign pointing the way to some interesting areas when they get to atomic and quantum levels...