Measuring An Online Community: Master Your Data to Gain an Unfair Advantage

You have a truly remarkable advantage over offline community builders. You can track every single action your audience makes. You should know exactly what stage they are at in the membership life cycle process and which stages need to be optimized.

I'm always amazed by the number of organizations and community managers who have either:

a) No strategy for the community beyond maintenance; or
b) A strategy built upon guesswork and assumptions when the data is so close at hand.

You shouldn't be guessing what is or isn't working in a community. You should be religiously gathering and analyzing what the data. You should measure the following:

New visitors. This shows whether your outreach is successful. Always compare it to the previous month and six months ago. You should also analyze where these visitors arrived from and track how many of each progressed into active members. You can also track the success of each different source of members (where does the best quality traffic come from?)
New visitors to new registered members. This shows whether your website is optimized for converting a curious visitor into a member and whether you're attracting the right sort of visitors. You can go further and measure their progress through each stage of the registration form.
Percentage of members who make a contribution. This shows whether you are converting those that register into participants within the community. If this is low, you might be just collecting lurkers.
Members active within the past 30 days. This shows whether you are gaining or losing active members. When this number starts to drop, you have a serious problem and a limited amount of time to correct course.
Contributions per active member per month. This is an activity per member ratio. If this drops, members are less engaged in the community and this could lead to more members leaving. This might also show if a small number of members are dominating the discussions.
Visits per active member per month. This shows how often members visit the community. The less frequently members visit, the more likely the contributions will drop and the number of active members will depart. This may also show the popularity of events held in the community.
Content popularity. Each piece of content can and should be measured. How many people read it and how many responded to it. This will indicate which content items are most popular and which should be discontinued.

You should also use sampling to understand the following:

What percentage of newcomers remain members for more than a month. Select 10 newcomers from three months ago and analyze their journey through the community and specifically where they dropped out of the process. Did they make a contribution? Did they not make a second contribution? You can adjust and tweak your community for this.
Speed of replies to discussions. How quickly are discussions receiving a reply? The faster the responses, the higher the level of social presence within the community and the greater the level of participation.
The percentage of newcomers who initiate a discussion. This highlights whether newcomers may be unmotivated or intimidated to start discussions.
Language and tone of voice. What language do members adopt when they address each other? Is it formal and polite? Is there friendly banter? Is there a sense of familiarity? This will let you know what stage the community is in.
Sense of community. Ask members every year to participate in your specially modified version of the sense of community index.
Number of volunteers. This will indicate the number of people moving on to the highest levels of engagement within the community. Low numbers usually limit the scalability of the community.

Each piece of data will tell a story. If the number of active members is decreasing but the level of contributions continues to rise, it might indicate a core group is dominating discussions and other members are unable to break into the circle. As a result you might provide core members with a separate place to chat, or work to break newcomers into the group or talk directly to group members about the problem.

Create a spreadsheet and a graph showing all this data. Update this monthly. Watch for numbers that dip and take a corrective course of action.

When you gather data you can set objectives, strategy and targets for each of the areas of community management (growth, moderation, relationships, activities, content etc...).

In practice, if you notice the number of volunteers has dropped, you can set a relationships strategy to focus on fewer bring and offer opportunities to be involved in areas of the community they are passionate about.

(Image: Measuring time, a Creative Commons Attribution (2.0) image from aussiegall's photostream)