They might never work for novel discovery but that probably can be handled by outside loop or online (in-context) learning. The thing is that 100k or 1M context is a marketing scam for now.