Is a custom core required to support the full standard instruction set? I thought I could license the rights to have my own, say, A53, and then I’d be able to customize it (as long as I don’t call it an A53 core). Is that not the case?

I wouldn't be surprised if their licensing terms require your chip to pass some kind of compliance test for the standard ISA, but probably only the licensees know that. I'm sure you can't use the ARM trademarks if it isn't compatible thought - that's pretty standard.