One of my favorite parts about open collaborative tools is the ability to observe the evolution of collaborative projects. Platforms like wikis, blogs, social bookmarking, and even discussion forums enable any observer to come in and view the history of any given article, compare versions, and review the progress and process of that page. But at the same time, it’s important to remember that there’s a lot of work that goes into a collaborative projects that can’t be gleaned from simply watching the changes to wiki pages. So in the name of getting my thoughts down, here’s an incomplete list of some things that you can learn and what you cannot learn from outsider observation (I see some follow up posts to this topic in the future).
What You Can Learn
Who’s literally made the change to the page: You can certainly measure raw, quantitative participation: i.e. Justin has made 20 edits or has posted 7 blogs or has tagged 14 items with 24 tags. This has a place in evaluating collaborative projects, obviously: in most cases of open collaboration, more participation is better.
You can tell literally what–qualitatively–a user has contributed to the project. The transparency of contributions means that you can see exactly who and when an idea or contribution was made. So I can see that Justin added 2 paragraphs of content to the wiki page and 3 blogs worth of ideas today. While this in isoluation is not really important, these contributions can be qualitatively evaluated. Likewise, you can see who is playing a more facilitative or leadership role in projects.
Social Networks. Certain elements of social networks become visible when you look at collaborative projects in open environments. Social bookmarks reveal who is tagging content for a project, thereby linking those participants. Blogs with comments clearly link people to their thoughts, but also to the people behind the contributions at some level. Similarly, you can discern the strength and weakness of links by looking at actual participation.
What you can’t learn
However, there’s a lot of work that goes into collaborative projects that goes unnoticed and undetected, even in an open collaborative environment.
Triggers, positive and negative. One of the most interesting aspects of projects that is simply not visible to the outside are triggers for participation. You may be able to literally see that a person who should be a prolific contributor only contributed once to a big collaborative project; however, what’s not visible is why they haven’t paricipated. Technical hurdles? Lost interest? Managerial interference? Too Busy?
Rallying. As I’ve said before, collaborative projects require a good amount of leg work behind the scene in order to get off the ground. So while some of these are visible–blogs, announcements in the wiki, etc–many are not. Without actually engaging the participants of the projects, you cannot, as an outsider, understand the work done behind the scenes. So I can’t tell that the project’s leader called colleagues on the phone from 5 different organizations, visited 3 seperate offices to build the network, and spent 3 hours talking to leaders face to face. All of this activity is important to the success of a project, but it’s not really visible from the outside.
Ghosting Participants. Technology is changing quickly and some folks just can’t or don’t want to keep up with the latest developments and newest tools. So, from my experience, many times there ends up being one “stuckee” from an organization or office that ends up doing the lionshare of the actual posting of information. In this case, what looks to be a single prolific contributer may be 5 people’s contributions via a single point of entry.
The Key
As a consultant, nothing’s asked of me more than to provide best practices. And the best practice for advising clients is that quite simply, you have to look beyond the numbers so that you can actually tell the story of the collaboration. Evaluating and understanding a collaborative success or failure not only takes leg work to track participation and contributions, but also to talk to the contituency, both active and non.
Social Consulting 