I really love this language:

"we see Rayt as a place to debate, discuss, or expose anything being broadcast on the net - that we envision Rayt users as activists who are the world's volunteer watch dogs for the wild, wild web."

Also, do you really envision 5 red stars?  I think the zero star rayting is simpler. 

I think the five-star system certainly has it's drawbacks - and it can be a bit too much if you treat the "stop sign" as the sixth star. That's a lot of stars. Could you elaborate on the zero star rayting and how that would work?

Also - double thanks for changing the stop sign!

Matt, thanks so much for weighing in.  I like the red light green light idea, but my concern is that it is too much like the StumbleUpon thumbs-up/thumbs-down.  The difference between the two is that Rayt shows the various gradations between green and red (yellow, orange, etc) depending on how many red or green rayting the site receives.  However, it would be really easy for StumbleUpon to change to a gradation system.  Then what would set us apart?

That being said, I like the idea of Rayt as the anti-StumbleUpon.  We don't just tell users what is good.  We tell them what is bad... and everything in between. That is what sets us apart - we give richer site data.  We need to preserve that distinction.

Hey folks. Great discussion. Joe and I started this conversation so I thought I would weigh in.

In trying to understand the purpose of Rayt I thought to myself what information I would want the moment I arrived at a new website. I came up with 3 things.

1. Is it safe for me to be on this site? Is it a fraud, will it put spyware on my machine etc.

2. Is this a good site? The criteria can be drastically different for different sites. Moreover, the relevance of these criteria can vary not only from site to site but from user to user. For example, someone may not like a site because its hard to use while someone else will like it because it has great deals and I may agree with one or the other. I would like to see that distinction quickly and easily. Also, how much of the rating is attributable to the site and how much to the company or person? For example, a great bank may have a terrible site. So what is the scope of the rating and how does the system monitor and communicate it?

Alternatively, is this the best site for me to be doing what I'm trying to do? Could Rayt give me suggestions about where else I should be looking based on ratings and my preferences (ease of use person vs. great prices person)?

3. How accurate is the rating of the above two things? If there have only been two people who rated the site I will be less inclined to trust the rating than if 1000 people had. How much disagreement is there (whats the spread on the rating). Why do people feel the way they do, what should I watch out for? This is where the comments are really crucial.

So when I think of what would make Rayt super useful to me, I think of three things.

1. Communicate the above information to me very quickly and unobtrusively.
2. Create the easiest way possible for me to contribute to the system.
3. Ensure enough participation and security so that the rating system is accurate. In other words create a system that has the characteristics suggested in the book "The wisdom of crowds".


In brainstorming how one could visually communicate all three things very quickly I thought of the following.
Have a simple view that has n number of stars with a rating which is the average of the stars per category. Also if enough people complained about the authenticity, legality or scruples of the site then a stop sign could show up beside the ratings as a warning to stay away.

Finally, You could also support an advanced view that instead has letters, single words or small icons denoting different categories (ie. V for value, U for useability etc). People could even create their own categories. The color of the letter could denote the rating on that parameter (possibly a gradient between red and green). Finally, you could set font sizes or icon sizes to denote how many people rated the site along that category. It would be intuitive because categories that had been rated few times would be hard to see. Finally, in terms of doing ratings yourself the categories themselves could be drop downs into a gradient who's level you choose or simply red, yellow, green. People could choose to only rate along the categories they felt were important to that site. On aggregate this would also communicate to other users what characteristics are important to watch out for and also it would be a form of intelligent filtering in that it would literally be difficult to see those categories that were not, on average, deemed important. Categories could also be sorted from most rated to least etc. To simplify the UI you could filter out those categories that seem to be irrelevant.

Anyway, those are my thoughts. I hope they are helpful and thanks for all the effort you all have put into this idea. Its great and sorely needed by the entire web community. Wow, this was a long comment. My bad.

Carlos - I've read your comments a few times and found them very insightful - thank you for continuing the conversation!

In particular, I think your ideas for user-generated categories for rating a site combined with larger font sizes for those categories which get more ratings is fantastic! The way you envision this working is very well thought out and could be applied to numerous rating systems. It would be interesting if a site like newstrust.net, which rates news articles on various gradients (i.e. - accuracy & style) integrated this kind of system.

Your ideas go a long way towards answering your second question, "Is this a good site?" That question, which so many people can answer and respond to in so many different ways, certainly requires a careful answer as we develop potential prototypes for Rayt.

My only concern with developing the system you've described would be ensuring that our barrier of entry remained as low as possible. If we tell people that they can judge sites on as many or as few gradients as possible, is that asking too much? Is giving a site a rating between 1&5 stars too vague to be useful to the extent that we would have to add another layer of user-generated gradients, which could make it too complicated to be usable? It's a bit of a catch-22.

This may be just a UI issue, though. Perhaps it could be brilliantly easy to present the different categories. Then again, making anyone click anything more than once could be conceived as asking too much. And asking someone to rate a site on numerous criteria requires numerous clicks.

Maybe there is such a way as to present the single gradient five-star rating system as the default option with extra gradients under the hood if people wanted to rate sites on a more specific scale. Perhaps that's how you've envisioned it?

So ... now this comment is getting a little bit too long, too. Your ideas definitely deserve more exploring as does the question: How do we make sure people understand that Rayt is about discussing, exposing, and protesting? And not about how pretty or usable a site is. Tricky indeed.