Measure What Matters

A Practical Approach to Association Performance Metrics

Summary

Well-designed metrics can measure any and every aspect of an association. This article provides an overview of performance as it relates to value creation (known in the product community as product development). 

In doing so, I define performance and frame its importance, distinguish between different types of metrics, provide guidance on using metrics in one’s association, and give a brief overview of data-driven storytelling.

The product community is a product development learning community designed specifically for associations. 

The Importance of Performance Metrics

"In God we trust, all others bring data."

W. Edwards Deming

Performance is a widely-used (and often misused) concept in associations. From a tactical or daily management lens, it is the action or process of carrying out or accomplishing an action, task, or function. Performance can also be defined more broadly as the ability of an association to implement strategy to achieve organizational objectives.

Measuring performance takes an investment in, and commitment to, building and executing a comprehensive (but not overly complex) metrics program. A metrics program will help us set direction, guide our actions, and help us gauge how well we’re living up to our promises.

From the most practical sense, metrics result in data. Associations excel at collecting data; we are less good at:

• Truly knowing what’s important (a.k.a. truly knowing what to measure).

• Aligning our data and metric practices to our strategic priorities.

• Developing (and sticking to) data standards as a critical org-wide practice.

• Identifying the right metrics to guide member journeys or outcomes.

• Using metrics holistically to determine association-wide performance.

This article won’t cover these topics in depth, but I want to make a pitch for investing in good, consistent data and an association-wide metrics program so we can position ourselves for a more sustainable path to superior performance, organizational health, and steady growth.

I recommend approaching metrics in a way that we can consistently and confidently answer these questions:

• How is a particular product performing (annual conference, publication, etc.)?

  • Is it meeting its intended target? 

  • How is a particular product performing in comparison to other products?

  • How is our overall product portfolio doing?

• What do our metrics or key performance indicators actually measure?

  • How do we determine satisfaction, learning, impact, or return on investment? 

  • How can performance data be used to change practice, bridge gaps, or improve outcomes? 

• How is our association doing as an organization?

  • How do know if our association is meeting its strategic goals?

  • What compelling story can we tell with the performance data?

Among the many angles to approach the topic of performance, this article focuses on measuring product performance. In this context, products are the indispensable value that we use to identify, draw, motivate, excite, and serve our members. 

This value can be anything as simple as a podcast, webinar, membership, or short course; it can be as complex and multifaceted as a piece of software or larger, more complex product like an annual meeting. 

The product community is deliberately designed as outcome-based, thereby requiring us to invest in metrics as a system of practice that informs, guides, and elevates. 

Product metrics are used to guide pattern analysis, measure performance, drive impact, and clarify direction. A comprehensive metrics program will incorporate quantitative, qualitative, and mixed methods. This is true for formative milestones (measurement during the journey) and summative outcomes (measurement at the end of the journey).  

Over time, a strong metrics program can be foundational for investing in program architectures such as competency badges and certificates. These signify successful program completion but also hold the responsibility of keeping the community alive. In this way, data becomes something more than gauging performance or progress, it becomes an indicator of a healthy culture.

Different Types of Performance Measures 

"It is an immutable law in business that words are words, explanations are explanations, promises are promises, but only performance is reality."

Harold S. Geneen

For any association, there are endless metrics that we could use to track performance. Which should we use to help guide our journey? Though there is some obvious variation, most associations could utilize four fundamental metrics: 

1. Acquisition – How many prospects are our marketing programs or community engagement efforts driving to our association?

2. Conversion – What percentage of prospects become a member, take a course, or engage meaningfully in our community?

3. Retention – What percentage of our customers remain active over time?

4. Revenue – How much money do our members generate?

A related, non-association example Google’s HEART Framework, which measures the success of digital products and stands for Happiness, Engagement, Adoption, Retention, and Task Success.

The framework is widely used by product teams allowing them to leverage quantitative and qualitative user data in the product design and delivery process

As growth in the product community is multifaceted and cultural in nature, performance can also used to gauge how an association’s culture evolves in tune with your product portfolio and strategic priorities.  

Tools and frameworks can help reinforce strengths and provide guidance for areas of professional growth or organizational health. They are also instrumental in sales, storytelling, and making a sound case for making investment decisions.

Product performance is a huge topic. Let’s distinguish among three related concepts:

• Metrics – Quantifiable measures used to track processes in order to judge a particular product’s (or your overall association’s) performance.

• Key Performance Indicators (KPIs) – A subset of metrics, KPIs are measures that tell you how your association is doing: your website uptime, your customer service satisfaction, the quality of your product, etc. As opposed to OKRs, KPIs are often considered BAU (business as usual).

• Objectives and Key Results (OKRs) – Measurements used to describe and measure where you want to be, not where you are. We use OKRs to lead; we use KPIs to manage. The below graphic lays out the common format of an OKR. The reason why it is hard to build great OKRs is because they require developing, committing to, and carrying out, a sound strategy. 

What’s the difference between OKRs and KPIs? First of all, OKRs can be used with KPIs. 

Let’s say you have an objective pertaining to a new association market. You might have the following key result (KR): Grow the new market by 20% by the end of Q1.

We emphasize “grow the new market” because it represents the KPI for this specific Key Result. Enhancing your reach to new markets is one you can stay relevant and to scale your association.

Here are some additional OKR examples:

• Increase gross profit margin to 10% for Q1. (KPI: gross profit margin)

• Increase revenue to $5M in Q3. (KPI: company revenue)

As you can see, Key Results intrinsically encompass KPIs.

In some cases, you’ll find that some Key Results may not directly include a KPI or a metric. In this instance, you’re more likely working with a milestone goal; for example, launching a new association management system which can be measured based on whether the implementation has been completed (or not). 

Creating a Meaningful Metrics Program

“If you don’t collect any metrics, you’re flying blind. If you collect and focus on too many, they may be obstructing your field of view.”

Scott M. Graffius

How can you make your association metrics more meaningful?

In the context of a product community, performance should focus on the right metrics; the wrong metrics can cause confusion or misguided decision making. Here are six steps to make your metrics more meaningful.

1. Start with strategy metrics. Everything you do, and everything you analyze, should be connected to your association’s overarching business strategy. Focusing solely on performance metrics without considering strategy can result in your metrics being disconnected from your desired outcome. Instead, use your strategic goals to help define measurable objectives. Then identify performance metrics that help to achieve these objectives and by extension the larger strategy.

2. Develop interconnected metrics. Don’t develop your metrics in isolation. In social media, for instance, reach, click-through rates, common trends, and conversions are all separate metrics. They should, however, be tracked in connection to each other. Just as your metrics should be connected to larger strategic goals, they also need to be horizontally integrated, especially as changing one metric might influence another. It also helps ensure your metrics meaningfully measure your efforts and help you tangibly improve.

3. Link your metrics to the future. Do your metrics just measure past performance (lagging indicators) or do they forecast the future (leading indicators) as well? Defined simply, lagging indicators describe results; they tell you about ‘the what’ but not ‘the why’. Leading indicators, in contrast, focus on desired outcomes and help uncover what leads to these outcomes. It’s common for associations to collect and analyze lagging indicators, although leading indicators are more likely to predict future performance. 

4. Avoid an over-reliance on goals. Goals can be great, especially if your focus is on leading indicators. They can help you achieve common direction for everyone to work toward. Goals can also be a trap as they are often hard to define and can narrow everyone’s focus to the exclusion of a healthy operating culture. Risks can increase and cooperation can decrease. So what’s the balance? The key is to find common ground. Use your future-facing metrics to set realistic expectations, but be careful not to become over reliant on goals. Balance can help achieve lasting success in which metrics improve holistic organizational performance.

5. Define and track desired end states. Meaningful metrics should focus on shared business outcomes. Associations often fail to achieve their strategic goals because they are not connected to the metrics. Just as you define metrics with your larger strategy in mind, you also need to ensure the data you collect and analyze maintain a consistent connection to your strategy. Define your outcomes then build your metrics. You can then establish benchmarks to help track your progress toward your goal and increase the connection among your strategy, objectives, and performance metrics.

6. Provide external context to internal metrics. Thus far, we’ve largely focused on internal metrics. These measures can come alive in comparison to data from the external environment. For instance, linking market needs with internal capabilities helps you develop metrics that are realistic, achievable, and growth minded. They can also provide the context and benchmarks you need to reliably measure success over time. They allow for responsible goal setting, as well as an in-context evaluation of your place in the marketplace. Only an external consideration can make your BI metrics truly meaningful.
   

Tell a Story – Data Doesn’t Replace Intuition

“The purpose of visualization is insight, not pictures”

Ben Shneiderman

I recommend developing a core set of metrics to help paint a picture, tell a story, and drive impact. This is important as product leaders interact with all parts of the association and each stakeholder has different motivations, ways of thinking, and communication methods. 

Here are a few high-level tips for telling a story with your data. 

• Be mindful of your audience

• Be short, simple, and accurate 

• Use language everyone can understand

• Use simple graphics to provide context, show growth, or demonstrate impact. 

• Let the data speak for itself; don’t twist the data to force your point.

And, of course, use data to support or add context to your story. This evidence will strengthen your claims or help you make connections.

Without the right metrics, your efforts can be frustrating at best and misleading at worst. That's why it's important to define metrics that actually matter to your association. This takes time, can be complex, and requires a number of steps. 

It doesn't make your commitment, process, and outcomes any less essential. 

Remember, product-led growth fuels connection. Join the product community and flip your destiny. 

About the Author

James Young is founder and chief learning officer of the product community®. Jim is an engaging trainer and leading thinker in the worlds of associations, learning communities, and product development. Prior to starting the product community®, Jim served as Chief Learning Officer at both the American College of Chest Physicians and the Society of College and University Planning.

Please contact me for a conversation: james@productcommunity.us.