“Telling the truth” implies two things: that the actual effort needed is fixed, and that you know it. In practice I’ve found neither of those things to be anywhere close to true for large projects. Furthermore the pursuit of accuracy by doing finer and finer-grained estimates and then rolling them up can waste a lot of time because it provides fodder for bike shedding while missing the forest for the trees. I can’t count the number of times I’ve seen this process lead to a total train wreck.
What works better is to be very clear on the big picture goals, do a speculative high level breakdown to inform an initial deadline and staffing commitment, but keep the precise scope flexible. Then get right into prototyping and building, attacking the areas of largest risk and refining the requirements as you go. You only should have fine-grained plans for the next 2-3 months with the rest of the roadmap intentionally being flexible, save for any key milestones that are needed to ensure progress to the high level goal.
Of course this requires deep expertise with cross-functional influence and bi-directional trust with management, but if you don’t have those things big projects are fucked regardless. In that case your best bet is hunkering down with some agile methodology as a shield while looking for a better job.
I find my biggest hurdle with estimation is often one small requirement which turns out to require a large amount of work. Like, "have this element behave in [unique way] on this page." Ok, well, that element was never even built to be touched individually, let alone have custom behavior. So now it's actually doubled the development time.
This is something which can usually be caught during estimation, but occasionally it isn't something you're going to notice until you're actually digging into the code and implementing it. And if you're going that deep during estimation, you're already practically building the thing, so estimation is now taking up a significant amount of time and effort. I am not a super experienced dev, but I haven't really seen a way around these issues cropping up often enough to make estimation feel close to worthless. (that is, it's unreliable often enough that it's unwise to couple any timeline too closely to any estimate)
> What works better is to be very clear on the big picture goals, do a speculative high level breakdown to inform an initial deadline and staffing commitment, but keep the precise scope flexible.
For this part I find that separating out the aims from the goals is a helpful, e.g.
Aim: to be sheltered from the rain
Goal: build a roof
Then you can adjust the goals (what kind of roof; materials etc) while keeping in mind whether the goals meet the aim, and whether the aims are still appropriate, and their priority too. Analysis is more difficult when the concepts are conflated.
What works better is to be very clear on the big picture goals, do a speculative high level breakdown to inform an initial deadline and staffing commitment, but keep the precise scope flexible. Then get right into prototyping and building, attacking the areas of largest risk and refining the requirements as you go. You only should have fine-grained plans for the next 2-3 months with the rest of the roadmap intentionally being flexible, save for any key milestones that are needed to ensure progress to the high level goal.
Of course this requires deep expertise with cross-functional influence and bi-directional trust with management, but if you don’t have those things big projects are fucked regardless. In that case your best bet is hunkering down with some agile methodology as a shield while looking for a better job.