A Business Technology Place

The Merchant Stand

Tag Archives: usability

A/B testing shows more than conversion rates

Posted on by Posted in eCommerce 1 Comment

I recently tested two different versions of an updated landing page to a site that I manage. During the design process the usability analyst received feedback from people not associated with the site on several manual wireframe prototypes. With this input, she developed a couple of rough mockup designs using powerpoint and went through another round of user group feedback. Ultimately she narrowed down the designs to two leading options.

The two leading page designs were then developed and placed on the site as the new landing page in a 50/50 A/B split. Our initial thought is that we would compare each design against the baseline conversion rate established by the existing landing page (our control). What we found was the the conversion rate was flat to slightly higher for each design but that page B in our test had a higher instance of customers forgetting to select a required radio button choice.

Here’s a quick comparison of the three field design for options A and B. The page had other elements but the form layout varied by a horizontal and vertical stack of the required fields.

A/B testing form layout

 

So our conversion rate metric improved slightly but we learned that the usability of design A was much greater than design B. At a 2x rate users of the site would not choose a selection for the the radio button question on the form.

Usability is no stranger to A/B tests on eCommerce sites. It’s typical to see it with colors, button styles, form types, etc. In my case, I was so focused on the conversion rate metric prior to the release that I forgot about the usability test of the two design. The two options were setup perfectly for a usability test because their content was the same while we varied the layout pattern of the form on the page that has fields for user input.

Use case example – removing the ‘how’ from requirements

Posted on by Posted in Agile Organization, Customer Focus, eCommerce Leave a comment

I met with members of a project management office this week to plan an upcoming software project. Do understand that this is not an agile software shop. So when I suggested we create use cases in lieu of a more formal requirement document suite I received some inquisitive stares. Really more like stares that silently said ‘huh?’

This particular software release does not include any new functional capabilities. It’s more targeted for user interface and site flow types of enhancements. I wanted a set of project artifacts that were more focused on context, flow, and results rather than something filled with sections of  pre-written project governance language.  The point of my suggestion to them was not start a full discussion on software development methodology, but get through the requirements phase  by being less document centric.

Of uses cases and user stories.

Use cases are an early form of user stories. For waterfall and traditional project management shops, the user story provides a nice alternative from a set of fully loaded requirements. I like them because they are written in logical sequenced steps which aid both development and testing staffs.  User stories take the simplification of documentation to another level, but that’s not the scope of what I writing about today.

A use case is part of the eCommerce Organization responsibilities in the experience and usability area.  This document often works as a piece of the overall usability design including mock-ups and process maps.

Removing the ‘how’ from requirements.

I like uses cases because they show how interactions with software are observed by the user. This allows the author to focus on ‘what’ rather than ‘how’.  A use case creates a dialogue with the solution owner on what they expect to happen under specific situations. So a use case does not include internal algorithms or details about the UI. Those are taken from other documents such as mock-ups or design documents.

This type of project artifact and conversation offers a project team several key features:

  1. Allows business users to talk about ‘what’ in their language. This is more natural to business and solution owners and prevents them from being burdened with IT system speak about capabilities and system constraints.
  2. Keeps the business analyst from over analyzing. The point is to talk more in natural language and flow.
  3. Simplifies the process of collecting fully loaded requirements which may not be appropriate for the scope of the project.
  4. Produces a document that is easy to understand for development and QA in the design and build-out of code and test cases.

A sample use case.

Use-Case Specification:  Select shipping method

Version:

1.0 – Initial version

Goal:

The goal is for the customer to select a shipping method for their purchase and to proceed with checkout.

Scope:

Select shipping method covers the first page within the checkout process which collects the shipping method and shipping address of the customer.

Actor(s)

Customer

Assumptions

  1. The customer has least one item in their cart.
  2. The shipping method logic will enforce rules regarding class of service and delivery address

Trigger

The shipping selection page is triggered by:

  1. The customer selecting checkout from the shopping cart
  2. The customer selecting to return to previous step from the final order summary

Pre-Conditions

  1. The customer must maintain at least one item in their shopping cart for the shipping selection page to be a valid option.

Flow of Events

Primary Flow

  1. The customer enters their delivery address
  2. The customer chooses an available shipping method
  3. The customer selects to continue with checkout

Alternate Flow #1 Step 3

  1. The customer selects to go back to the shopping cart

Alternate Flow #2 Step 3

  1. The customer changes their delivery address and/or shipping method
  2. The customer selects to continue with checkout

Post-Conditions

  1. The customer will either return to the shopping cart or proceed to the final order summary.

Business Rules

  1. Non-USPS carriers do not deliver to PO Box addresses.
  2. Show the class of service rather than the carrier (i.e. 2nd day, Overnight, First Class)