It’s pretty simple. Most projects fail to meet estimates. Typically we are estimating so we can determine:

• do we possess the resources to complete a project?

• can we finish on time?

• is the project even worth it?

None of this can be calculated before the start. The information we possess before we've actually done something is nothing so we are making guesses based on the WORST information. What’s more, if we wait to test the accuracy of our fictional estimates we'll end up too far behind to make mid-course corrections.

So the solution is to measure all the "risk" upfront through lean experimentation. And by "risk" I mean the unknown bits that defy accurate estimation.

So, identify the unknowns, slice them into thin experiments, dedicate resources, time-box each experiment, and complete only enough until it is "knowable" then scale estimates accordingly.