It's not about git conflicts, it's about behavior of the resulting repository and merging changes safely. You can absolutely have changes two changes A and B that have no conflicts to resolve textually, but can result collectively in a bad repository state (i.e. build is broken, tests fail) when both A+B are applied.
For example, say A is a patch that renames the function foo() to foobar(), and then B is a patch that calls the function foo() at a completely new callsite. The original repository is green, and both A and B are individually green too. You merge A into main, then merge B without rebasing or re-merging main, and the build is broken. Each of these changes passed the tests in isolation, but together they will result in a repository that is broken. This has nothing to do with whether the codebase is a mess or not; it's just a simple rename of a function. B and A are simply mutually exclusive, and most be ordered concretely between each other.
To fix this case manually, you have to merge A to main, then rebase B onto main (or merge main into B), which would then result in "function foo() not found", or your tests failing or build exploding. Now that the CI has caught it, you can change B to use the new function foobar() and re-attempt a merge again. Except you also have patches C-through-Z written by 5 other developers that might also conflict. This kind of example is everywhere in a large codebase; imagine that you're renaming files, changing parameter types to a function, reworking test output from debug statements, etc. And now imagine every CI run is 15 minutes long. If anyone merges in that 15 minutes before you, you have to start all over again. This also applies to all 5 developers of all other 20+ patches, too.
The long and short is that there are a lot of cases where, to be safe, you need to just rebase your change on top of the latest tip first, before you can be 100% sure the build passes. Codebase cleanliness has nothing to do with it; that's just too pessimistic.
The Merge Queue solves it a different way. Just queue up A and queue up B to be merged in series. Actually, the order doesn't matter at all. Let's say B is up first, then A, then a new patch C that is totally unrelated. The build passes with B applied, because it worked originally so it gets merged. Next up is A. A now fails, because even though it applied the patch successfully, there's a new call to the old function named foo(). So it gets kicked out. Now C is up immediately after. It succeeds, so it gets merged. At this point, the author of A is now responsible for rebasing their change and fixing the build. At no point did the author of either B or C have to be responsible for re-merging or rebasing their changes on top of main, as they triggered the happy case.
The best way I can describe merge queue versus manually rebasing is this: the merge queue is optimistic locking, while manual rebasing is pessimistic locking. The time-to-merge a change is the latency. In an optimistic lock strategy, you always try to do the thing, but just detect if it fails and safely abort. The pessimistic case requires strict serialization of the operations to ensure no conflicts, but it needlessly holds up many concurrent writers. It has nothing to do with "messiness" of the data structures, to use an optimistic lock; you might just have a really writer-heavy system on your hands! If we keep putting it in latency/locking terms, this results in a much better "p90 time-to-merge latency", in other words.
In a large team of developers, working on a big codebase, the "optimistic locking" approach of the merge queue is very effective at getting PRs merged faster, and has very few downsides.
> For example, say A is a patch that renames the function foo() to foobar()
Why are you allowing commits to land that break interfaces?
If this is a public function it should go through a deprecation cycle.
If this is a private function then it shouldn't be accessed from outside its module/class/whatever in the first place. If it is, then you've got the "messy codebase" I referred to in my original comment.
They can both be private methods in a single module, the example still applies fine? You can for example rename a private function in a single rust crate, which is used by other functions within the crate, which is not exported.
I'm not sure if you're being intentionally obtuse here, it's a very simple scenario that has nothing to do with public interfaces or deprecation cycles.
> They can both be private methods in a single module, the example still applies fine?
It does, but you're now looking at an edge case that rarely arises in practice in a well-maintained codebase, and certainly not one to design your entire branching/merging/CI strategy around.
If your codebase is a spaghettified nonsense then the problem arises much more often, and so I can understand the use of merge trains/whatnot.
My situation: small team with around 10 simultaneous projects at any one time, all depending on a framework that we use for tons of different things. Everyone is constantly modifying the same parts of the code because that’s where the features should be added. It’s way cleaner than when the projects were separated and didn’t share code.
In this case, we often have conflicts in private modules. (And everything is private since we don’t provide any libraries to anyone.)
For example, say A is a patch that renames the function foo() to foobar(), and then B is a patch that calls the function foo() at a completely new callsite. The original repository is green, and both A and B are individually green too. You merge A into main, then merge B without rebasing or re-merging main, and the build is broken. Each of these changes passed the tests in isolation, but together they will result in a repository that is broken. This has nothing to do with whether the codebase is a mess or not; it's just a simple rename of a function. B and A are simply mutually exclusive, and most be ordered concretely between each other.
To fix this case manually, you have to merge A to main, then rebase B onto main (or merge main into B), which would then result in "function foo() not found", or your tests failing or build exploding. Now that the CI has caught it, you can change B to use the new function foobar() and re-attempt a merge again. Except you also have patches C-through-Z written by 5 other developers that might also conflict. This kind of example is everywhere in a large codebase; imagine that you're renaming files, changing parameter types to a function, reworking test output from debug statements, etc. And now imagine every CI run is 15 minutes long. If anyone merges in that 15 minutes before you, you have to start all over again. This also applies to all 5 developers of all other 20+ patches, too.
The long and short is that there are a lot of cases where, to be safe, you need to just rebase your change on top of the latest tip first, before you can be 100% sure the build passes. Codebase cleanliness has nothing to do with it; that's just too pessimistic.
The Merge Queue solves it a different way. Just queue up A and queue up B to be merged in series. Actually, the order doesn't matter at all. Let's say B is up first, then A, then a new patch C that is totally unrelated. The build passes with B applied, because it worked originally so it gets merged. Next up is A. A now fails, because even though it applied the patch successfully, there's a new call to the old function named foo(). So it gets kicked out. Now C is up immediately after. It succeeds, so it gets merged. At this point, the author of A is now responsible for rebasing their change and fixing the build. At no point did the author of either B or C have to be responsible for re-merging or rebasing their changes on top of main, as they triggered the happy case.
The best way I can describe merge queue versus manually rebasing is this: the merge queue is optimistic locking, while manual rebasing is pessimistic locking. The time-to-merge a change is the latency. In an optimistic lock strategy, you always try to do the thing, but just detect if it fails and safely abort. The pessimistic case requires strict serialization of the operations to ensure no conflicts, but it needlessly holds up many concurrent writers. It has nothing to do with "messiness" of the data structures, to use an optimistic lock; you might just have a really writer-heavy system on your hands! If we keep putting it in latency/locking terms, this results in a much better "p90 time-to-merge latency", in other words.
In a large team of developers, working on a big codebase, the "optimistic locking" approach of the merge queue is very effective at getting PRs merged faster, and has very few downsides.