Terrell Russell: This Old Network

In my last post, I wrote about Liz Lawley’s short piece concerning anonymity in review sites concerning teachers and students. She also posed the question of checks and balances in these systems.

The checks and balances needed in these systems are straightforward. They fall out of the required list of elements for a review system to be successful.

1. Require a verified reviewer – Whatever this means in context. Over time we will see OpenID being used to build these systems and identities can be used across systems and potentially aggregated. For now, the walled garden is probably the de facto standard – but that is changing. Systems should definitely not allow anonymous comments to have the same ‘weight’ in whatever scoring or rating system is put in place. Perhaps a side area for anonymous chatter that fades quickly from view.
2. Require a comment/rating – This really does require full text for full explanation and context, everything else is too simple. However, that being said, a simple Lykert scale for a few attributes makes available the generation of statistics and otherwise easier analysis – full text is much harder to count (but much richer).
3. Require a reviewee – The item being reviewed should be a unique entity in the system. For systems where the reviewee is actually a person, and not an event or location or album, the reviewee should be a first class entity as well – able to leave their own feedback and comments/reviews. Here is where the power lies for future development.
4. Make available notifications for all parties involved – This is probably the most important. There should always remain the potential for further conversation about any review/rating. The system should assist in that regard – removing any/all barriers to further discussion and digging to the real facts and circumstances. RSS, email (opt-in or otherwise) should always be made available to all parties; reviewer, reviewee and others.
5. Administrators of the system should always have the right to remove – Probably the most contentious point, but necessary. The administrators should always reserve the right to edit what is said in the system, assuming it’s held in a single place and arguably a full record of what was edited. With a distributed system, this is nigh impossible, but we haven’t seen any like that yet. Distributed systems change the calculus dramatically for all elements of reputation, so this is a fair concession at this point in time.

All systems of this type benefit from the network effects they can generate. The value in the network definitely follows Metcalfe’s Law as the utility of any review area will be significantly greater the greater number of users and reviews posted. It might go without saying, then, that none of the above pieces really matter if you don’t have a proper threshold of activity. The long tail of networks doesn’t win in this regard. Only the big part of the curve gets to play this game today if the goal of the site is global significance (eBay, Amazon, etc.). If a network you are building has a lower bar for success (only your friends, or your school/job), then success can be reached much more easily with less users and activity.

Tags: lawley - metcalfe - rating - reputation