Measuring the Performance of Association Products
Create Growth Paths By Measuring What Matters
Summary
Well-designed metrics can measure any and every aspect of your association.
This article provides an overview of performance as it relates to value creation (known in the Product Community as product development).
Performance is one of six association product competencies; the other five are: culture, vision, customer, design + build, and go to market.
In doing so, we define performance and frame its importance, distinguish between different types of metrics, define and discuss the concept of a maturity model, provide guidance on using metrics in your association, and give a brief overview of data-driven storytelling.
The Product Community is a product development learning community designed specifically for associations.
How Are We Doing? How Do We Know?
Associations collect a lot of data. We recommend approaching this data in a way that we can consistently and confidently answer these questions:
How is a particular product performing (annual conference, publication, course, etc.)?
Is it meeting its intended target?
How is a particular product performing in comparison to different products?
What metrics or key performance indicators are used to determine satisfaction, impact, or return on investment?
How can performance data be used to change practice, bridge gaps, or enhance features?
How much of our performance data do we actually use?
What compelling story can we tell with the performance data?
Among the many angles to approach the topic of performance, this article focuses specifically on measuring product performance.
In this context, products are the indispensable value that we use to identify, draw, motivate, excite, and serve our members.
This value can be anything as simple as a podcast, webinar, simple game, or short course; it can be as complex and multifaceted as a piece of software or larger, more complex product like an annual meeting.
The product community is deliberately designed as outcome-based, thereby requiring us to invest in metrics as a system of practice that informs, guides, and elevates.
Product metrics are used to guide pattern analysis, measure performance, drive impact, and clarify direction.
A comprehensive metrics program will incorporate quantitative, qualitative, and mixed methods. This is true for formative milestones (measurement during the journey) and summative outcomes (measurement at the end of the journey).
Over time, a strong metrics program can be foundational for investing in program architectures such as competency badges and certificates. These signify successful program completion but also hold the responsibility of keeping the community alive. In this way, data becomes something more than gauging performance or progress.
Different Types of Performance Measures
For any association, there are endless metrics that we could use to track performance. Which should we use to help guide our journey? Though there is some obvious variation, most associations could utilize four fundamental metrics:
Acquisition – How many prospects are our marketing programs or community engagement efforts driving to our association?
Conversion – What percentage of prospects become a member, take a course, or engage meaningfully in our community?
Retention – What percentage of our customers remain active over time?
Revenue – How much money do our members generate?
Another non-association example comes from Google. It’s called the HEART Framework and stands for Happiness, Engagement, Adoption, Retention, and Task Success. The HEART framework measures the success of digital products. The framework is widely used by product teams allowing them to leverage quantitative and qualitative user data in the product design and delivery process.
As growth in the product community is multifaceted and cultural in nature, performance is also used to gauge how an association’s culture evolves in tune with your product portfolio and strategic priorities.
Tools and frameworks can help reinforce strengths and provide guidance for areas of professional growth or organizational health.
Though not covered in depth here, measure for growth is also instrumental in sales, storytelling, and making a sound case for making investment decisions.
Product performance is a huge topic. Let’s distinguish among three related concepts:
Metrics – Quantifiable measures used to track processes in order to judge your association’s performance.
Key Performance Indicators (KPIs) – A subset of metrics, KPIs are measures that tell you how your association is doing: your website uptime, your customer service satisfaction, the quality of your product, etc. As opposed to OKRs, KPIs are often considered BAU (business as usual).
Objectives and Key Results (OKRs) – Measurements used to describe and measure where you want to be, not where you are. We use OKRs to lead; we use KPIs to manage. The below graphic lays out the common format of an OKR. The reason why it is hard to build great OKRs is because they require a real strategy to be implemented.
What’s the difference between OKRs and KPIs? First of all, OKRs can be used with KPIs.
Let’s say you have an objective pertaining to a new association market. You might have the following key result (KR): Grow the new market by 20% by the end of Q1.
We emphasize “grow the new market” because it represents the KPI for this specific KR. Enhancing your reach to new markets is one you can stay relevant and to scale your association.
Here are some additional OKR examples:
Increase gross profit margin to 10% for Q1. (KPI: gross profit margin)
Increase revenue to $5M in Q3. (KPI: company revenue)
As you can see, KRs intrinsically encompass KPIs.
In some cases, you’ll find that some KRs may not directly include a KPI or a metric. In this instance, you’re more likely working with a milestone goal; for example, launching a new association management system which can be measured based on whether the implementation has been completed (or not).
Adopting a Maturity Model
Measuring performance helps product community participants achieve clarity of mission while ensuring learners demonstrably gain critical knowledge and skill in the product community competencies.
Assessing product performance is an important part of the product community fabric, integrated at all stages of the experience: in creation of expectations, development of goals and outcomes, measurement of learning, and tangibly connected to growth, impact, and product performance.
A maturity model gauges an organization’s maturity in a number of competencies. It is used to better understand an organization's current state and helps determine areas of improvement. Maturity models are also a useful tool to help create longitudinal learning journeys.
The product community competencies — culture, vision, customer, design + build, go to market, and performance — anchor the maturity model (and are represented in the middle column of this graphic).
A maturity model can also help an organization understand its limitations or gaps, develop a vision, or set strategic goals.
Maturity models are structured as a series of levels of effectiveness. Associations (or teams) will typically pass through levels in sequence as they become more capable.
This particular model is generalized. That is, these six competencies typically comprise the elements of most association product strategies.
Working with a maturity model begins with a general self-assessment. This can be accomplished by simply identifying relevant levels for each competency.
It is common for some respondent choices to be sporadic or inconsistent (you will want to circle more than one box in a row). This is normal.
Try to select the best choice and please reflect any identified contradictions during the report out.
Once you've determined your level for each competency, you can prioritize the capabilities your organization needs to get to where it wants to be.
The important outcome of a maturity model exercise isn't determining your current level, but determining the actions to take (or capabilities you need) to reach your ideal destination (represented in blue in the graphic).
A successful product community is grounded in broad ownership and shared execution of the association’s strategic aims, values, vision, and other aligned plans.
In this way, the successful execution of the product community becomes a powerful vehicle to position the association durably for growth.
Advice on Using Metrics at your Association
How can you make your association metrics more meaningful?
In the context of a product community, performance should focus on the right metrics; the wrong metrics can cause confusion or misguided decision making. Here are six steps to make your metrics more meaningful.
Start with strategy metrics. Everything you do, and everything you analyze, should be connected to your association’s overarching business strategy. Focusing solely on performance metrics without considering strategy can result in your metrics being disconnected from your desired outcome. Instead, use your strategic goals to help define measurable objectives. Then identify performance metrics that help to achieve these objectives and by extension the larger strategy.
Develop interconnected metrics. Don’t develop your metrics in isolation. In social media, for instance, reach, click-through rates, common trends, and conversions are all separate metrics. They should, however, be tracked in connection to each other. Just as your metrics should be connected to larger strategic goals, they also need to be horizontally integrated, especially as changing one metric might influence another. It also helps ensure your metrics meaningfully measure your efforts and help you tangibly improve.
Link your metrics to the future. Do your metrics just measure past performance (lagging indicators) or do they forecast the future (leading indicators) as well? Defined simply, lagging indicators describe results; they tell you about ‘the what’ but not ‘the why’. Leading indicators, in contrast, focus on desired outcomes and help uncover what leads to these outcomes. It’s common for associations to collect and analyze lagging indicators, although leading indicators are more likely to predict future performance.
Avoid an over-reliance on goals. Goals can be great, especially if your focus is on leading indicators. They can help you achieve common direction for everyone to work toward. Goals can also be a trap as they are often hard to define and can narrow everyone’s focus to the exclusion of a healthy operating culture. Risks can increase and cooperation can decrease. So what’s the balance? The key is to find common ground. Use your future-facing metrics to set realistic expectations, but be careful not to become over reliant on goals. Balance can help achieve lasting success in which metrics improve holistic organizational performance.
Define and track desired end states. Meaningful metrics should focus on shared business outcomes. Associations often fail to achieve their strategic goals because they are not connected to the metrics. Just as you define metrics with your larger strategy in mind, you also need to ensure the data you collect and analyze maintain a consistent connection to your strategy. Define your outcomes then build your metrics. You can then establish benchmarks to help track your progress toward your goal and increase the connection among your strategy, objectives, and performance metrics.
Provide external context to internal metrics. Thus far, we’ve largely focused on internal metrics. These measures can come alive in comparison to data from the external environment. For instance, linking market needs with internal capabilities helps you develop metrics that are realistic, achievable, and growth minded. They can also provide the context and benchmarks you need to reliably measure success over time. They allow for responsible goal setting, as well as an in-context evaluation of your place in the marketplace. Only an external consideration can make your BI metrics truly meaningful.
Tell a Story – Data Doesn’t Replace Intuition
In the Product Community, we recommend developing a core set of metrics to help you paint a picture, tell a story, and drive impact.
This is especially important as product leaders interact with all parts of the association and each stakeholder has different motivations, ways of thinking, and communication methods.
Here are a few high-level tips for telling a story with your data.
Be mindful of your audience
Be short, simple, and accurate
Use language everyone can understand
Use simple graphics to provide context, show growth, or demonstrate impact.
Let the data speak for itself; don’t twist the data to force your point.
And of course if you have data supporting your story always include that, as it is evidence to help make your claim.
Without the right metrics, your efforts can be frustrating at best and misleading at worst. That's why it's important to define metrics that actually matter to your association. This takes time, can be complex, and requires a number of steps.
It doesn't make your commitment, process, and outcomes any less essential.
Remember, product-led growth fuels connection. Join the product community and flip your destiny.
About the Author
James Young is founder and chief learning officer of the product community®. Jim is an engaging trainer and leading thinker in the worlds of associations, learning communities, and product development. Prior to starting the product community®, Jim served as Chief Learning Officer at both the American College of Chest Physicians and the Society of College and University Planning.
Please contact me for a conversation: james@productcommunity.us.