The difficulty is arbitrary and set socially (mechanistically it is currently set by a formula in the software; it could be set differently if people decided to use different software...).

So having a lower difficulty at time t is a terrible reason for a given miner to move to a given chain, they have to also believe that it will be the consensus chain that maximizes their value/opportunity in the future.

Also, if you don't care about distributed consensus (discarding chains outside the network to make it keep working), POW mining is an expensive and inconvenient way to keep a ledger.