A Business Technology Place

The speed of software releases

How fast do you want software released?

I was part of a discussion this week week about the velocity of production software deployments. I didn’t use velocity in the context of estimating software but rather the benefits of a regular cadence to releases and the focus it brings to the testing effort. The velocity of the team affects the frequency of production changes and thus in a sense affects the overall risk to the organization for disruption to existing services.  If we release software too quickly, I believe we increase the risk of service disruption and create inefficiencies in system-level testing. If we release software too slowly then we increase the risk of losing competitively in the marketplace as well as losing customers.

Boris from GoldenEye knew what he liked.

Boris from GoldenEye knew what he liked.

This wasn’t a discussion about agile software development principles.

For my part, I was curious about the relationship of production software releases,production defects created by those releases, and the testing effort required for approval. Maybe velocity was the wrong term to use based on its agile methodology definition.  My focus was on the frequency of change events to the production environment. In the most basic form, software releases can arranged by individual unit (feature/bug).  Each time there is a production release it requires at a minimum unit testing for the bug or feature that is new or has changed. But when we don’t do some level of full system testing we may miss items that are broken as result of the code change. All production releases by nature have some element of risk to disrupt current services.

What’s the right software release frequency?

Obviously the answer to this question depends on a variety of factors. Software and business teams make choices and decision about how they answer the question.  I look for a balance to provide support in the following areas:

  • Quality Assurance – The testing effort should have time  focus on feature and system-level testing.  Full system testing should reduce the risk of introducing new bugs into production as result of the new code.
  • Maintenance window – Excluding full system outages, the maintenance windows provide an agreed-to time for production updates. So if the maintenance windows are every other week then the software release schedule should not exceed every other week.
  • Priority – Not all production defects require immediate resolution.  The balance is to fix priority defects in a timely manner but not so frequently that it increases the risk and pressure associated with quality assurance thoroughness.

Remember what it is all about.

The point of velocity is not to tick a check-list that we have a process. The velocity does have benefits with estimating. But the real value is to provide a regular cadence of features with the highest quality to the customer. Most of all, the customer should understand the flow and be part of the process.

Onward and upward.

Policy on supporting multiple browser versions

Time travel back 15 years.
I recently had a conversation with an IT product manager that wanted to post a statement on a website about which browsers are officially supported by the site. There was some angst with the testing group as well as the technical support desk about phone calls from customers that had browsers that were recently released. It gave me flash backs to the early days of the internet when I would see statements on websites like:

“This site works with …..”
“The site requires …..”
“This site is designed to work with….”

I know it’s not possible to test with every combination of browser version and operating system.
The root of the request was that the quality assurance group was not setup to test with every browser version. I get that. Unlike the early days of the internet, there are more options than Internet Explorer and Netscape. Plus add years of build-up and internet browsers now have multiple versions in the market. But this problem isn’t unique to my colleague’s products. It’s part of doing business on the internet.

What customers expect.
Customers expect your internet site to work. Whether they are shopping, price comparing, or researching product information, they expect to find information and not think about their choice of browser. Most of my friends (non-technical) don’t think about Chrome, Firefox, Internet Explorer, Opera, Safari, etc. Most of them use the browser that was installed with their system (Internet Explorer usually) and don’t even know the version number installed on their system. Their expectation is that they can transact with information without thinking about the tool.

A Better Way.
I recommend to look at your internet analytics to see what the most common browsers are for your customer base. Pick the top two or three and make sure the QA group tests with those browsers. Make sure the developers are programming with industry standard techniques and then handle incidents on a case-by-case basis. This type of approach will minimize the number of issues and not alienate any of your customers that have chosen a different browser from what is officially tested.

Make sure the technical support staff collect the browser type and version on incident calls. Issues may or may not be related to the browser. But this gives your organization a better chance of replicating issues and determining if the root cause is related to the browser.

When you know about a browser incompatibility.
If you determine there is an issue with a particular browser type and version then do two things:
1. Notify the technical support staff so that they can inform and customers experiencing this issue.
2. Determine if you need to resolve the issue. The answer is determined based on many factors such as:

  • Is it a complete block in the order flow?
  • Is there a suitable work-around for the customer?
  • How common is the browser type and version the customer is using?

A good example in 2011 is the continued use of Internet Explorer v6. The browser is now ten years old and does not support many of the programming features of the internet today. In a B2C environment it may be easier to walk the customer through an upgrade. In a B2B environment this may be more complicated depending on the number of installations and compatibility with other software.

The Big Idea.
The big idea though is to have a policy that states it’s your intention to support all browser types and versions. Really, this is an unwritten policy that is expected by internet customers. You can test with the most common versions and then examine issues as they arise and treat them on a case-by-case basis.