I agree that using one technique for an infinite world is probably going to get boring pretty quickly, and you can probably get more leverage out of a finite world.
But even with an infinite landscape, I think you could improve the variety by using a generator at different scopes - if your landscape is just one constant type of terrain, like a field, things will get boring. You might have a wide variety of rivers and trees, but the basic idea is the same. But if you have another generator controlling the terrain type - going between plains, desert, mountains, etc., that would be really interesting. At the small scope, variation in trees and water is still the same level of change, but there is also a larger scope that changes the overall feel at a much larger radius.
Oh definitely. Minecraft does this in a simple but effective way. Though the interestingness of the landscape lasts until you've seen a few instances of each type, and then you have a model of the system in your mind, because it's just a few simple functions composed together. There is a lot of fun stuff to do so the terrain is only just the backdrop to that fun.
Instead of {noise(x,y)>0.5 = desert} let's simulate eons of weather, geological events, floods, and then feed that back into the lore of the game world. Dwarf Fortress is one of the few games I'm aware of that attempts this level of world generation. Unfortunately it's quite impenetrable as a game, due to its complexity.
But even with an infinite landscape, I think you could improve the variety by using a generator at different scopes - if your landscape is just one constant type of terrain, like a field, things will get boring. You might have a wide variety of rivers and trees, but the basic idea is the same. But if you have another generator controlling the terrain type - going between plains, desert, mountains, etc., that would be really interesting. At the small scope, variation in trees and water is still the same level of change, but there is also a larger scope that changes the overall feel at a much larger radius.