10. Define what success looks like and publish performance data

This guidance will help you apply standard point 10.

Everyone is responsible for meeting the Service Standard. This standard point is most relevant to:

Performance analystsBusiness analysts

Why it's important

Having clear objectives, a definition of what success looks like, and appropriate metrics can help you know whether your service is solving the problem it's meant to solve.

Collecting the right information and interpreting it will alert you to potential improvements you need to make and help you know if changes have the effect you intend.

Discovery

Things to consider:

  • define what a definition of success looks like for discovery. This should include both quantitative measures, such as specific research outcomes, and qualitative indicators, like user satisfaction, or conversion rates
  • how a service may add value to users in your problem space
  • access whether that value could reasonably be realised by developing a service, and only proceed if you think you can
  • some data collection to inform baselining

Alpha

Things to consider:

  • define what you want to measure, why, and how these measurements will be obtained
  • identify analytics that describe time spent on pages, heatmaps, if users start something but don't finish it
  • use performance data and reporting to capture baseline measurements on service efficiency, timescales, service level agreements (SLAs)
  • collect and use qualitative insights from user research and feedback on the current as‑is service to inform decisions
  • provide evidence of baseline data, or explain why it is not available, and set out how you will measure future success
  • show how you have iterated and improved metrics and data collection as you learn more about user needs
  • explore metrics you could analyse to support or improve your service

Things to avoid in alpha

  • putting measurements against data that does not exist in your as-is service. You could explain why certain measurements could not be included when describing the measurement choices you've selected

Beta and live

Things to consider:

  • capture and collate measurement data agreed in the KPIs during alpha
  • combine quantitative and qualitative data to measure where benefits are being realised
  • demonstrate how the collated data provides evidence to show how your new service is performing for users
  • evidence that decisions to make improvements and fix problems are based on performance data
  • engage with business owners and stakeholders to help make decisions using performance data
  • iterate and improve ways to collect metrics and data as the team learns more about user needs
  • regularly publish performance metrics; these must include cost per transaction, user satisfaction, completion rate, and digital take-up
  • apply learnings from metrics in beta phase when moving to live

Things to avoid in beta and live

  • not capturing or analysing data needed to assess service performance

  • failing to engage with stakeholders when interpreting performance data