Nov 16 2007
WCBA - Delivering Effective User Acceptance Testing
In the Modeling, Testing and Design track at the World Congress for Business Analysts (WCBA), Jonathan Kupersmith (Kupe) of B2T Training (and formerly of Turner Broadcasting) described how to deliver effective User Acceptance Testing (UAT).
UAT represents the final approval by the customer and is typically conducted by users with assistance from the project team with tests derived from day-to-day operations, use cases and process workflows. UAT validates that the final solution meets the needs of the customer.
First, Kupe pointed out that UAT is not the same as usability testing. And a project is doomed when it substitutes or replaces usability testing for user acceptance testing.
In the traditional approach, project team members, business analyst (BA) or quality assurance (QA) write test scripts, users get a quick demo of the new application and then walk through test scripts step-by-step.
Kupe described the shortcomings of the traditional approach including:
- project team members pressed for time
- users were not fully vested in UAT
- users did not fully understand how new functions should work
- tests are often generic
- high pass rate (because the BA wrote the scripts), but risk of major issues not being discovered until production
Kupe confessed that he came up with his more effective approach based on a bad experience he had with the traditional approach. He described the more effective way to approach UAT as:
- involving key users early
- providing users with hands-on system training
- facilitating sessions that create the test plans
The end results from this revised approach were that:
- users took responsibility for project success
- users were comfortable with new functions
- test scripts involved real life scenarios
- issues formerly caught in production were caught in UAT
- no major issues made it to production
- users felt they were part of the team and helped champion the release