estimating | The Merchant Stand

Let’s talk about #NoEstimates

Should we expect estimates for software development?

There’s a movement called “No Estimates” that suggests there is a better way to develop software without using estimates as part of the process. In an interview with InfoQ, Woody Zuill states, “Personally for me, what No Estimates is about, is alternative ways to make decisions, that don’t require that we use estimates.” As I read various articles on the idea, they suggest that the need for estimates is reduced (or eliminated?) by incrementally providing working software that can be shipped or delivered to the customer. The counter argument says that estimates are needed to provide input to scheduling, budgeting, and prioritization activities..

I read the debate with great interest since software delivery and estimates are part of my everyday life. I think proponents of both sides would agree to that the business of writing software is non-repetitive and could be classified as art. Estimates by definition are not completely accurate. In my mind, the dilemma between the programmer and stakeholder is not only how the estimates are delivered, but how they are interpreted by each. Project stakeholders look for estimates to provide working accountability for schedules and budgets. While the programmer is just trying to give a forecast of what he or she thinks based on what they know at the time.

The answer isn’t whether we estimate or not, but why we estimate.

Zuill continues his thought, “But the problem isn’t with how easy they are to do, or how accurate they are. Is has more to do with, why do we need the estimates. What is our motivation for having these estimates?”

It’s a fair question and a good one. The value of getting the answer lies with discussing this line of questioning with project stakeholders to come to a mutually acceptable answer. Completely getting rid of estimates is hard because estimates are part of our life and culture. It spans all industries and services. We use estimates for our personal possessions as well as business processes at work to help us make decisions. Zuill asks if we need the estimate to make the decision.

Think about the first principle in the agile manifesto that says “Individuals and interactions over processes and tools.”

You see the value in the estimate is the interaction it creates with the customer and other project stakeholders. Estimating could be a nuisance to the programmer, but estimating creates an opportunity for an interaction with an individual. Customers like and expect estimates. But I think many customers and stakeholders would be open to the question of “What value does this estimate provide?” What we may find is that some estimating steps could be eliminated while others should remain. The answer may be in the middle-ground. Software development, after all, is a custom build each time through.

Classic IT Challenges – Understanding Project Complexity

Posted on July 27, 2013 Posted in Business Process Management Leave a comment

This is the fifth post in a series of writings to discuss issues facing Information Technology (IT) departments. If you have not read any of the first four challenges you can find them under project work vs production support, prioritizing work, operations vs innovation, and estimating work.

IT shops vary in size, budget, and processes. But all IT shops share a common set of challenges that shape how they interact with their inter-company customer base as well as the clients of their organization. There isn’t a single answer to these common challenges and they don’t disappear after a process is put in place to address them. The manner in which IT management chooses to respond to the challenges is interwoven in the culture, processes, and services of the group. My thoughts are more around framing the problem than presenting solutions.

Challenge #5 – Understanding Project Complexity

It’s not uncommon to be in the midst of a project and to have new discovery that significantly increases the complexity of the solution. I’m not referring to intentional scope change requests. New discovery is when a detail of a scoped feature becomes known and the team has to redesign or create new alternatives and flows to existing designs. This might include a business rule for taxes, pricing discounts, or a manufacturing limitation. New discovery often leads to a scope change request document. But I classify this type of work differently because it is not a requested change in scope but more of a clarification or additional rule to an existing piece of high level scope.

Projects are usually scoped from documents such as RFPs, sales contracts, and project charters. Those are great documents, but they aren’t produced by a business analyst that has been through a series of questions to discover the finer details of project requirements. Specific use cases and functional requirements are often absent during the initiation stages of a project. In a perfect world, project teams would know every possible detail that could affect their solution set. But we don’t live in a perfect world.

The scoping phase of a project is where budgets, timelines, and tasks are defined. These three areas (triple constraint) comprise the working guidelines and constraints of the project. So if the team encounters a new discovery then it presses the working guidelines and usually requires communications with stakeholders. Where’s the fun in being over budgets or late in the deliverables?

Example 1

A new client is signed and is in the process of onboarding. While setting up the invoice process a discovery is made related to special taxing rules for orders. Depending on the customer classification they may or may not be taxed.

Example 2

During the implementation of a new customer new business rules surface about who can order product and who cannot order. The right-based system was in the original scope but the new rule set creates additional criteria to define the rights of a group or individual.

The Challenge

The challenge for a business analyst is to uncover as much of the project details before project execution begins. But the bigger challenge for project teams is how to react to new discovery and how to manage the expectations of the stakeholders. The best scenario is if new discovery does not lead to an increase in budget or an expanded timeline. But in many cases it’s not possible to finish the project without expanding the original guidelines.

I’ve framed this challenge more in terms of a traditional waterfall software development cycle. Agile methodologies also encourage some level of discovery up-front with artifacts such as user stories. But the premise of the overall challenge is the same. If we misunderstand the true complexity of a project in the early stages it can have large consequences to the work during later stages of implementation.

Next Week

Challenge #6 – Reducing Costs (SG&A)

Classic IT Challenges – Estimating Work

Posted on July 13, 2013 Posted in Business Process Management 1 Comment

This is the fourth post in a series of writings to discuss issues facing Information Technology (IT) departments. The first three challenges were project work vs production support, prioritizing work and operations vs innovation.

Challenge #4 -Estimating Work
Estimates provide the basis for developing budgets, creating timelines, and setting customer expectations. I have suggested before that estimating is the most powerful step in the software development cycle because it determines if projects are approved or rejected.

There are a couple of challenges with estimating work. The first is that despite processes that call for initial estimates and then re-estimates when more requirements are defined, most of the important decisions are based on the high-level estimates. Project budgets, timelines, chargebacks, and approvals are most often formed at the beginning of a project and often before the requirements are truly understood.

The second challenge is trying to keep the amount of time spent on an estimate within reason and to keep the time spent on the estimate from impacting active work. Creating an estimate requires time from an analyst. So there is a cost! Organizations should be mindful of this because spending money on a project that is not yet active or approved lengthens the the time to recognize return on the investment and it could possibly take people away from working projects that are already approved.

Example 1
A team receives a high level description of a project and is asked to provide an estimate. A timeline is built from the estimate and shows a final delivery in 90 days. 15 days into the project a discovery is made that two systems are not compatible and that a major assumption about equipment availability is not true. The new information will require an additional 30 days on the project timeline. How does the team and react to this information? Is the budget revised? Is the timeline revised?

Example 2
A portfolio management organization has five active projects. Three new project requests arrive. The same people working on the five active projects are the estimators for the new work. If they stop work on the active projects it will create delays in the project timelines, but executive management wants estimates on the new requests to determine what to do with the requests.

The Challenge
Estimates are by definition inaccurate. Yet, important decisions are made from them. IT team members feel the pressure not to underestimate because budgets are created from the estimates and it’s hard to secure additional funding for budget overruns. But IT team members are also pressured not to overestimate because stakeholders may cancel the project, complain that estimates are unreasonable, or take the work to an outside agency. Business stakeholders want accurate estimates delivered in a timely manner and with a minimal amount of detailed project requirements.

Next Week
Challenge #5 – Understanding Project Complexity

The most powerful step in software development

Posted on December 15, 2011 Posted in Agile Organization, Business Process Management, eCommerce 2 Comments

What’s the most powerful step in a software development process?
It’s not uncommon for analysts to label steps in a process. The critical path, bottlenecks, waste, and non-essential steps come to mind. So I would say that common wisdom agrees that all steps in a process do not hold equal weighting of importance. Maybe there isn’t a single most powerful step in the software development process you follow or maybe it depends on the context of the situation.

For what it’s worth, a few weeks ago it occurred to me that the act of estimating was perhaps the most powerful step. Estimating is completed at the ground level, by those that do the work, not by the executives or managers. Most decisions to do or not to do a project are within the hands of executives and managers. But it’s those closest to the work that influence the decision with their estimates for cost and return.

Mike Cottmeyer, a leading Agile development thinker and trainer, acknowledges that estimating for numbers is a tough step in software development in his post The Real Reason We Estimate. But Cottmeyer’s main point is that “Estimating is about creating a shared understanding of the requirements, and a shared understanding of the solution. When teams have problems estimating, its almost never an estimating problem, it’s a shared understanding problem.”

I like that line of thinking and agree that estimating provides the benefit of shared understanding. But most shops I know use estimates to determine if a project will even exist.

Estimating determines if work will begin
The power of estimates is clearly visible when a large estimate for work cancels a project or a particular solution to problem before it’s even started. To be fair, that’s part of the purpose of the exercise. If a work effort to bring a solution to a business problem is so large that it doesn’t make sense to do then you would want to consider alternatives or move on to the next business problem. That’s just sound financial discipline.

But I’m thinking so much about the extreme estimates as I am about the typical project. The number of hours estimated is eventually turned into a cost and the cost is compared to the dollar figure from the estimate of benefit. That’s basis for the return on investment calculation. Bottomline, estimating is used as a decision point if the project will begin or not.

Estimating determines the budget
If the project is considered to be a good business investment then the estimates are used to set the budget. That’s important for obvious reasons of people payments or equipment purchases. But the budget also determines if additional work is required in the future. If the team goes over budget it triggers other processes, meetings, and decisions to get in alignment with a budget. That’s additional work, which while necessary, is also expensive and not directly related to creating the solution for the customer.

So that’s my case. Estimation is the most powerful step in the process because it directly influences if the project will exist and if it does exist it influences the budget.

Let me know if you have any other thoughts on this topic.