> Could they have at least minimised the blast radius? For example, make all of the text/Unicode changes for Python 3 and thats it.
Except "that's it" does a ton of work here, as the entire stdlib had to be audited and updated to correctly work with the new text model, plus other semantics change (e.g. the removal of total ordering).
That those were so large and wide-ranging in the first place is why the core team decided that fixing a bunch of other issues could go in as well e.g. reorganise the stdlib, move keywords to functions, remove deprecated stuff, etc...
Those ancillary changes were mostly pretty easy to implement.
> I had a hand in moving from 2.7 to 3.6 for a 1M LOC repo and it was not pleasant. So many backward compatible changes that that all had to be accounted for rather than being able to focus on the major breaking change of a particular version.
It wouldn't have been more pleasant if the breaking changes had been spread over multiple versions, you'd still have eaten all of those, and the early adopter would have gotten even more pain as they'd have needed to implement breaking changes on every update.
> We couldn’t have done it without six.
We did just fine without six migrating a hundreds-kloc-scale repo from 2.7 to 3.5, we basically copied what Werkzeug did, with our own "minisix" module for just the bits we needed (some lifted from werkzeug's and others bespoke e.g. we needed a compatibility layer for CSV).
We could not, however, have done without the 3.0 - 3.3 improvements, the existence of 2.7, or the community experience.
And tests. Having a well-tested codebase was absolutely essential to a comfy migration, the less a corner of the codebase was tested, the more migration bug kept popping up afterwards.
> It seems that a bunch of changes were made all at once because it was already going to be incompatible so let’s take the opportunity to clean up as much as possible. I think that was the big mistake.
No, doing all those breaking changes version after version would have been a lot worse. IME the vast, vast majority of the pain (short and long term) came from the semantics changes, not the syntactic ones. Those were relatively easy: do the syntactic change, enable the corresponding lint (so colleagues don't reintroduce the old construct out of habit), commit. A few of the API changes were frustrating (like base64 which was clearly mis-converted by the core team but what do you do).
- the text model stuff took by far the longest and had by far the longest tail (as in things we missed during the migration), this is in part because not all of the changes or Python 3 APIs are ideal, but mostly it's just a lot of runtime things in a lot of paths if your software leverages strings a lot
- total ordering removal broke a surprising amount of things, and also had a pretty long tail of bugs, mostly in that `None` would get mixed in with other values and get fed to comparison operators, sorting, or min/max
- `round` was a surprise contender for pain in the ass, not just because it changed the rounding method (to banker's rounding, from I don't remember what) but also because P2's round() always returned floats while P3 returns integers if precision is missing / None
- changes to dict iteration also broke a few things, there were multiple pieces of code with implicit reliance on iteration order, although for the most part they were issues in numerical stability of floats, which broke tests using strict equality on floats
- division changes also broke a few things as it wasn't always easy to recognise whether operands were always floats or could be int, also `//` being floor division rather than integer division led to a few behavioural changes
We also did a few things like banning the builtins which had changed behaviour (open, map, filter, ...) for the duration of the migration, the issues they caused were subtle enough as to not be worth it. They were re-allowed once we dropped P2.
Except "that's it" does a ton of work here, as the entire stdlib had to be audited and updated to correctly work with the new text model, plus other semantics change (e.g. the removal of total ordering).
That those were so large and wide-ranging in the first place is why the core team decided that fixing a bunch of other issues could go in as well e.g. reorganise the stdlib, move keywords to functions, remove deprecated stuff, etc...
Those ancillary changes were mostly pretty easy to implement.
> I had a hand in moving from 2.7 to 3.6 for a 1M LOC repo and it was not pleasant. So many backward compatible changes that that all had to be accounted for rather than being able to focus on the major breaking change of a particular version.
It wouldn't have been more pleasant if the breaking changes had been spread over multiple versions, you'd still have eaten all of those, and the early adopter would have gotten even more pain as they'd have needed to implement breaking changes on every update.
> We couldn’t have done it without six.
We did just fine without six migrating a hundreds-kloc-scale repo from 2.7 to 3.5, we basically copied what Werkzeug did, with our own "minisix" module for just the bits we needed (some lifted from werkzeug's and others bespoke e.g. we needed a compatibility layer for CSV).
We could not, however, have done without the 3.0 - 3.3 improvements, the existence of 2.7, or the community experience.
And tests. Having a well-tested codebase was absolutely essential to a comfy migration, the less a corner of the codebase was tested, the more migration bug kept popping up afterwards.
> It seems that a bunch of changes were made all at once because it was already going to be incompatible so let’s take the opportunity to clean up as much as possible. I think that was the big mistake.
No, doing all those breaking changes version after version would have been a lot worse. IME the vast, vast majority of the pain (short and long term) came from the semantics changes, not the syntactic ones. Those were relatively easy: do the syntactic change, enable the corresponding lint (so colleagues don't reintroduce the old construct out of habit), commit. A few of the API changes were frustrating (like base64 which was clearly mis-converted by the core team but what do you do).
- the text model stuff took by far the longest and had by far the longest tail (as in things we missed during the migration), this is in part because not all of the changes or Python 3 APIs are ideal, but mostly it's just a lot of runtime things in a lot of paths if your software leverages strings a lot
- total ordering removal broke a surprising amount of things, and also had a pretty long tail of bugs, mostly in that `None` would get mixed in with other values and get fed to comparison operators, sorting, or min/max
- `round` was a surprise contender for pain in the ass, not just because it changed the rounding method (to banker's rounding, from I don't remember what) but also because P2's round() always returned floats while P3 returns integers if precision is missing / None
- changes to dict iteration also broke a few things, there were multiple pieces of code with implicit reliance on iteration order, although for the most part they were issues in numerical stability of floats, which broke tests using strict equality on floats
- division changes also broke a few things as it wasn't always easy to recognise whether operands were always floats or could be int, also `//` being floor division rather than integer division led to a few behavioural changes
We also did a few things like banning the builtins which had changed behaviour (open, map, filter, ...) for the duration of the migration, the issues they caused were subtle enough as to not be worth it. They were re-allowed once we dropped P2.