MEMBER LOGIN   |   Username: Password:
Twitter: (The Twitter thing is temporarily down due to a Twitter-related bug or something!)
Recent Updates: The New Site is now open! (6 years ago) Which of [these facial express... Vulpin Adventure OST| "Blast to the past!", a review of Plazma Burst 2 by Rating Orb Duogduogduog
...
Pseudolonewolf`s Avatar Rating System Ideas (EDIT 2) Mon 15th Aug 2011 11:02pm

Category: Site Updates

I'm going to be redoing the rating system for the new site. But I'm not sure HOW to redo it...

One thing I'm sure about is that ratings will be for posts only; no longer will it be possible to rate accounts directly. There'll be a Friends List sort of thing for that, as well as the ability to report people, and the 'anonymous reviews/comments' thing that I may end up doing.

There are many different ideas for how posts could be rated though. I'll go over a few ideas here, but obviously I'd like to hear suggestions as well.

1. There's the current system, with three positive, one neutral, and three negative. This one should already be familiar, so I won't explain it. It has its drawbacks, but gives a relatively wide range of potential values to rate a post.

2. There's another idea that I suggested before, where ratings consist of three parts; essentially yes/no values for 'Attitude', 'Language', and 'Contribution', where the values would be positive unless there was something wrong with the post. You'd have to specify a value for all three before a single rating (in a form like 010 or 011 or 110 or whatever) would be submitted.
It's a clunky system though since it'd take three clicks to make a single rating, and it's not very subjective; why bother letting people give such ratings if they're all going to be very much the same anyway?

3. There's the idea of having Agree/Disagree on posts... This could be added *in addition to* a rating system, or instead of one.
It would show how many people are on your side, and it might be easier to actually give such ratings, but there are certain contexts where they couldn't apply at all, like simple greetings or statements that aren't an opinion.

4. There's the idea of a simple Like/Hate system for comments, but that's too simple for my tastes.

5. Another concept I came up with was to have a sort of '5+1 stars' scale... There'd be six orbs (not stars) rather than the seven we have now, but five of them would function sort of like a standard five star rating system, with an additional negative orb reserved for comments that were abusive or completely inappropriate; the sort of comments that'd get red orbs currently.
The negative orb would count separately to the other ratings, perhaps, so people would have an Average Rating which would be n/5 'stars', but also a value which showed how many 'negatives' they'd got. Or something. The details aren't really clear about this idea.
But it's sort of like changing the negative end of the scale to only one value, and expanding the positive end by one orb, with brown being one star. Maybe?



EDIT 1:

6. How about a rating system with TEN levels of quality? Or eleven orbs... It's sort of like the '5+1 stars' thing except that you'd have a scale from 1 to 10 (plus the additional '0' or '-1' orb), which would each have their own colour, and none of which would be *negative* as such... Perhaps it'd result in far more variety in orb colours?

7. A potentially interesting but likely stupid idea came to mind, where comments wouldn't necessarily be rated 'positively' or 'negatively', but on a different scale (perhaps of 10, as above), with ends like 'law and chaos' or 'serious and casual'. Maybe the latter. Long, stuffy, 'logical' posts would get very 'Serious' ratings while shorter posts with bad grammar would get 'casual' votes... and as such a person's orb wouldn't necessarily reflect how 'good' a member they were, but what *type* of member they were.
This idea came to mind when I was thinking about how people might not rate if they don't want to hurt others or don't feel that posts deserve praise or something, and also due to the idea that there's probably some 'in-crowd' now, and everyone's just all in the same group, essentially; just some are 'cooler' than others.
I've been sort of wanting to lessen the strictness of the rules here in order to be a more inviting place (since I'll need lots of members to make lots of money), so if I did something like this, I could perhaps split the forum in two; one bit would be tailored towards the 'serious' crowd who want 'intelligent discussions', while the other part would be more laid-back and entertaining or something (though I'd never allow it to get to the point where people casually flung around abuse or communicated entirely in memes).
Anyway, it's just a raw, unrefined idea, this, presented here for the sake of variety.

With that idea, there might be a scale of three, five, or even nine or ten orbs, with 'casual' being at one end and 'serious' at the other. None would be considered negative as such; some people might take greater pleasure from serious votes than they would from casual ones, or vice versa. It all depends on the image that they want to present to others and the reasons that they are here.
Oh, and I'd likely think more about the names of them; 'casual' and 'serious' are just the two that came to mind readily. They could be things like 'entertaining' to 'thought-provoking' (though that would suggest that a post couldn't be both)...

8. Alternatively, rather than ratings, there could instead be a group of buttons for certain values which you could effectively turn on and off... Things like 'Useful', 'Entertaining', 'Thought-Provoking', 'Friendly', 'Strongly Agree', 'Strongly Disagree', 'Abusive', 'Poor Grammar', and so on. They'd basically work like 'checkboxes' in the sense that you'd be able to choose as many or few as you liked on any given post. That way it'd be possible to see a person's specific strengths based on their totals for these buttons. They could even have a secondary use for determining the colour of an orb beside a person's name; since some would be positive (like 'Useful' or 'Entertaining') and some would be negative (like 'Poor Grammar' or 'Abusive'), the orb would represent the ratio of positives to negatives (perhaps with a scale of 10 different colours).
The main problem with this would be whether or not people could be bothered to use it... though it might actually be easier than the current system to use? In terms of deciding what to give a post, anyway.
The interface might be difficult to make elegant though.

That system may have ten possible 'criteria', four of which are positive, two neutral, and four negative. The neutral ones wouldn't contribute to the person's orb colour; it'd be based on the ratio of their positives to negatives. Each user could give each post up to four positives and four negatives, and a post's overall orb would be the ratio of the total. The criteria might be things like:
Positive:
+ Useful
+ Entertaining
+ Friendly
+ Thought-provoking
NEUTRAL:
= Agree
= Disagree
NEGATIVE:
- Poor grammar
- Abusive
- Thoughtless
- Spam

Of course, they're just *ideas*; I'd need to decide properly on which criteria would work best. There needn't be a balance between positives and negatives.
This would make it harder to misunderstand or misuse the system though, I think.



EDIT 2:

9. As an extension of number 8, instead of having ten categories like that, with some positive and some negative, there could instead be fewer categories, and on each you could choose to either rate that criteria positively or negatively. The criteria may be as follows, each represented by a single letter on the interface to save space:
* U - marked positive if you thought a post was particularly USEFUL, or negative if it was useless SPAM.
* F - marked positive if the post was FRIENDLY, or negative if it was ABUSIVE.
* T - positive for THOUGHT-PROVOKING posts, negative for posts that show no thought put into them by the poster ('thoughtless').
* E - positive for ENGAGING or ENTERTAINING or funny posts, negative for boring posts that you could barely get through.
* G - negative for POOR GRAMMAR, positive for eloquence or something.
* A - positive for AGREE, negative for DISAGREE.

That reduces the number of criteria to six, and it means that you can't vote with contradictory values like marking something as both friendly AND abusive, or agree AND disagree...
The interface might look something like this:

(99 U 99) (99 F 99) (99 T 99) (99 E 99) (99 G 99) (99 A 99)

Except it wouldn't just be text; the ( ) things are meant to represent the edges of rounded button things, which would have the letter in the middle, a green edge, and a red edge. You'd click the green edge to mark that criterion as positive, or red for negative. Rolling over each one would display a tooltip which would explain the thing so you wouldn't have to memorise the letters or anything. The buttons would be grey or brown or black by default, except when you rolled over the edges, when they'd highlight either red or green, or if you'd already voted on any criteria, in which case the edge you'd voted on would be a shade of green or red. The numbers for the negatives and positives would be green and red, or brown/grey for no votes.
Though obviously it'd be easier to SHOW that than to explain it...
Anyway, it seems like potentially an improvement over the last idea, though I'd like to know what people think, of course.

Each post would have an overall Value determined by the ratio of positives to negatives (with agree/disagree not contributing to that value), which might be represented by an orb, and it's the average of these Values maybe that'd represent a person's orb next to their name (if they had one).

And again as with the other system, you'd be able to specify as many or as few criteria as you wanted. You could mark a post as just friendly, or as friendly/agree, or as thought-provoking/disagree/abusive, or any combination of them. You'd also be able to undo any votes you'd applied by voting the same way again (like if you'd voted + friendliness, you'd just click the green edge of the F bit again and it'd get rid of that part of your vote).
Your submitted rating would be altered every time you clicked a coloured edge; it'd refresh this interface with the updated numbers, meaning you'd need no more than one click if you only wanted to rate one aspect of the post.




I think that some other concepts have been suggested, but I'm not really in the mood to read through a load of stuff right now, so I'll instead ask for you to provide your favourite concepts in the comments section... Sorry if it'll mean essentially repeating something you've said before, but at least this will put everything together in the same place.

So yes! Ideas! Suggestions!

I'm also still looking for feedback on [the previous post], so if you've got something to say about the site in general that isn't about ratings, please mention it there.
19 comments

 

19 Commentson 17 roots

Esoto`s Avatar
Rating Orb β Esoto 20 Brazil 192C 27F
6 years ago | (1)
The rating system should be based on fancy temperament faces! So I could like rate people "totally !!!" but if a comment/post was nice and happisy and made me feel warm on the inside I could rate it a or . The rating would differentiate intelligent and well-thought posts, or just any post where you can imagine the poster making a face at all.

Hopefully that would help set the mood for the way you read posts, aswell as bring more confusion over the temperament system. After all, everyone loves fancy temperament faces and nothing could go wrong whatsoever!
JiQi`s Avatar
Rating Orb JiQi 99 Montserrat PhlegmaticMelancholic 41C 9F
6 years ago | (0)
Hello again...
I'm totally for multicriterial rating, esp. the "UFTEGA" system (numero 9) - this sounds good!
Only, maybe instead of "POOR GRAMMAR" use more general: "POOR LANGUAGE"?
Good luck with your work on the new site.
ScintillaTinge`s Avatar
Rating Orb ScintillaTinge 18 United States 59C 0F
6 years ago | (3)
9 appealed to me at first, but remember on Newgrounds when commenting on a flash entailed scoring the piece out of ten on a bunch of criteria? People didn't like giving a good piece, say, zero on Interactivity just because it was a movie and not a game, and sometimes just gave a piece straight nines or tens if they really liked it. Eventually the system was replaced with a one-number overall score. I feel the same would happen over here; there'd be great informative posts that don't deserve the positive Entertainment value, and funny userpage comments that don't deserve the positive Usefulness value. And because these factor into people's averages, unlike the old scoring criteria on Newgrounds, people would again be discouraged to post except for annoyingly long, rambling, falsely friendly, trying-too-hard-to-be-funny posts.

Oh, please don't do 7! As neutral as it sounds, I think people would make an even wider distinction between 'serious' and 'casual' posts than they already do, and pretty much everyone would look down upon the casual ones. Most good members are a healthy mix between 'serious' and 'casual,' in that they post their more academic, 'left-brained' posts as well as their friendly or silly ones, and if we made a dichotomy between people who sometimes appreciate silly little things, and the ELITE INTELLECTUALS, we'd get a lot of pretentious academicism from the now-good members, as well as pseudointellectual children who don't space after commas.

2, 5, and 6 are my favorites so far!

And lastly, could we do away with orbs next to people's names? I think that's the major source of the 'in-group' social organization we have now. Even without personal ratings, I think orbs next to names would have the same bad effects. It's not that hard to tell good members of a society from bad ones, and it's probably something we do in any society, naturally--
Pseudolonewolf`s Avatar
Rating Orb A β Pseudolonewolf 23 United Kingdom MelancholicPhlegmatic 2257C 559F
6 years ago | (2)
With the Newgrounds system, wasn't it necessary to specify a value for each of the criteria when submitting a review/comment thing? With system number 9, you'd only need to specify as many or as few as you liked; you could just mark a post with +Friendly without marking it with anything else. It seems like it'd result in fewer forced ratings where you HAVE to give a value but don't know what to give...

I also think it'd be a nice thing to get an idea of peoples' personalities, since you could look at their userpages and see the ratios and totals for each of the factors, so see that they might have, say, friendly and useful posts but grammar issues, or stuffy logical posts that were 'useful' but boring... (Since 'boring' (-Engaging) would be a valid rating for unnecessarily long ramblings... Trying too hard to be funny and failing might similarly get a negative on the 'entertainment' thing.)

Something that's come up since I introduced rating systems was the desire to explain why certain ratings were given, like with a snippet of text submitted alongside the rating itself. It seemed impractical, but it's understandable why people would want to be clear. This system would solve that issue, I think.

Anyway, I'd try and encourage people not to strive to achieve EVERY positive rating in every post they made, but rather to just 'be themselves' or something, during which they might get different positives for different posts. Of course, me trying to 'encourage' that wouldn't make it so...
But I suppose I'd like to see *why* people like my posts, if they do at all, and such a system would be a good way of seeing what I'm doing right or what I'm good at or something.

If I did the serious/casual thing - and chances are that I won't - then the ideal value to strive for would probably be somewhere around the middle... And if the site split into two groups over it, I could see each group looking down on the other rather than 'everyone' looking down on casual types... I know I'd prefer some 'casual' types to the pretentious children who try hard to sound all 'mature and intelligent' but it comes across as superficial...

I like orbs next to names just because, well, I like how they look. Names look naked without them, I think... Perhaps I could consider replacing them with something visual, like the current orbs, but not necessarily an indicator of the person's performance? Hmm...
NONEXITIALIS`s Avatar
Rating Orb NONEXITIALIS 19 United States CholericPhlegmatic 8C 1F
6 years ago | (1)
Seven and Eight both intrigue me. Seven because it's completely new, and I like shiny new things; Eight because it brings the normally boolean "like" system to a whole new level... both systems seem good in theory, to me at least. Honestly I like all the options except Four, but Five, Seven and Eight really jump out at me as "unique" and "interesting". That is my only criteria, really. It's why I like the current orb system, and why I really wouldn't mind any of those systems except something boring like a "Like" button.
Well that's my opinion. I know it's not very helpful, but it is my opinion all the same, and I figured I'd voice it on such an important topic.
Eleanor Rigby`s Avatar
Rating Orb E Eleanor Rigby 15 United States PhlegmaticCholeric 1184C 365F
6 years ago | (1)
Idea 7 sounds really intriguing. It seems like it would be safe from rating inflation that has made the current system practically useless, since none of the ratings on that scale are necessarily positive or negative. Very interesting concept. More ratings sound like an improvement, they would bring more nuance to the system. I don't like the checkbox idea (information but too clunky), but the agree/disagree part would be great for certain types of forum threads.

I'm not sure I really understand the benefits of the 5/+1 or 10/+1 scales. Would they make it possible to get a good and bad post at the same time?

If you do decide to stick to a rating system similar to the current one, have you considered switching from orbs/stars/etc. to faces? I use flash card software, and recently switched from one where I had to rate the difficulty of a card 1, 2, 3, or 4, to one where I instead had the options of six different faces. Instead of having to try and add meaning to numbers, I now just go with the emotion on the face that feels most "right". I think that would be good for the rating system here, as even with your guidelines, one of its main problems is that everyone has different interpretations of what sorts of posts deserve which colors. I think there is even a thread about it floating around somewhere.

I still miss value. +/- was excellent, even if it was binary. It worked because one only had to give a rating to posts that deserved one, not all posts.
EricLightscythe`s Avatar
Rating Orb EricLightscythe 15 India MelancholicPhlegmatic 53C 51F
6 years ago | (1)
Well, Somehow, I'm not comfortable with having to do away with personal ratings. Honestly, keeping it would be much better, so we can get an 'Idea' about people in general.
Also, if we rate someone, couldn't we have a poll thingy when we do that? Like, for example, if we rate someone yellow, a small box would pop up with options as to why we rated that person up/down. Like troll, uses foul language, etc. in the case of bad ratings or niceness, English, amiable character, etc in the case of good ratings. This would be only when we give someone a personal rating. And then the answers given could be, er, 'averaged' and shown in the form of a bar graph or something?

Hope you don't scrap the personal rating.
Good luck....
Abbx901`s Avatar
Rating Orb Abbx901 15 Pakistan SanguineCholeric 149C 20F
6 years ago | (1)
I would prefer the last one, to be honest.

It is quite clear that negative comments will either only be abusive, rude, off-topic, or be in another language, or be with terrible, terrible grammar and spelling.
In these cases, a group of two orbs would suffice.
I would like greater variety for positive ratings, since I have this urge to give someone neither a cyan nor green, then I end up giving him nothing, because I can't make up my mind.
I would vote against the Agree/Disagree thing, but maybe Pseudolonewolf can temporarily install each item to the site and get feedback then on what people think of it.
hlbeta`s Avatar
Rating Orb hlbeta 22 United States CholericMelancholic 144C 131F
6 years ago | (1)
You may want to take another look at this thread: [LINK]
It's pretty text-heavy, but it's also some of the best discussion on possible ratings schemes and voting mechanics I've seen on the forums. I stand by my original proposal with the additional mechanisms for integrating it into the current orb system. The planned break from the current statistics should make implementing most of those proposals significantly easier. I'm greatly in favor of rebalancing the voting structure to minimize the damage an abusive or naive member can inflict and believe that those mechanics are the best way to address the problem.

As for ratings in general, I like the aesthetics of the current system. It's small, eye-catching and easily read. My only real complaint with the orbs is that they don't provide much information on users because so many are blue or brown and greens are effectively impossible to attain. Whatever comes next, I'd like to see a greater degree of granularity and have the whole thing balanced so that the full scale can be properly represented.

Lastly, you need to remove the ratings or vote potency of banned users. Letting their abusive ratings hang around and distort the system seems sloppy.
DeNovo`s Avatar
Rating Orb E β DeNovo 24 United States MelancholicSanguine 244C 201F
6 years ago | (2)
The hardest systems to game are the simplest ones; for that reason I recommend the +1/Report system. Users only have two choices, upvoting a post or reporting it for violating the rules. Any user who is reported has all their votes negated, and any post which is reported no longer contributes to the member's rating. Member ratings consist of the total number of times their contributions have been upvoted.

The down side is that this system will need some amount of policing. The good news is that it rewards people whose posts are recognized by many people to be valuable, and people who are ranked highest are by definition the most recognized and respected posters. :D

You could make reporting someone more complex to make it more difficult to misuse, such as requiring a quorum of people/rating power reporting a post or member before it loses value, or bite the bullet and find a large enough group of people to police the community.

Hope this helps! :D
coreyj`s Avatar
Rating Orb coreyj 18 United States MelancholicPhlegmatic 105C 152F
6 years ago | (4)
I like most of these systems, except number 4.
Numbers 1 & 2 are my favorites, but number 3 would be nice, in addition.
I'm not so sure about number 5, though. It seems a bit awkward, and would take a while to adapt to...
Though I agree with removing the direct personal rating system.
coreyj`s Avatar
Rating Orb coreyj 18 United States MelancholicPhlegmatic 105C 152F
6 years ago | (4)
Oh, and could you make it possible to edit comments and posts? It can be quite a bother otherwise.
Page 1 of 2: