I'm using 2.8.10, and this behavior seems fixed. If you open a JPEG or PNG, there's now a menu option in File called "Overwrite file.jpg". If you do an export, this menu option is replaced with "Export to file.jpg" that repeats your last export without a dialog box. The shortcut is Ctrl+E.
The Overwrite option has been in there a long time and is essentially just an alias for the Export funtionality. Although it does help alleviate the situation, I totally agree with what Drdrdrq is saying - just make Ctrl-S work properly!
I don't think it's really clear that this is proper behaviour, though? If we're talking JPGs, every time you go through opening that file and "saving" it, it's going to be more and more mangled by the compression. (Also, each time you have to ask yourself "how much do I want to mangle it this time?" - which doesn't make sense). Effectively, the file you save definitely does not preserve what you have in the editor. On the other hand, with the current behaviour, Ctrl-S provides a certain guarantee that the output will be lossless.
And, sure, one can go off and appeal to nature and say "it's standard," but, I really don't buy that argument. There really isn't that much software that deals with the same stuff as GIMP. This isn't like a text editor or something. They are very much in a position to set their own standards when the defacto behaviour is wrong.
It's not just lossy compression you lose things like layers and stuff when you flatten the file to a bitmap.
In the general sense this is the correct behaviour but in the specific instance of doing simple bitmap manipulation like cropping "export as save" is expected and since thats the most common use case for many people it's probably frustrating.
To be perfectly honest people using gimp for image cropping should use something else, gimp gui is clunky, it starts slowly and there is no benefit.