Blog
Mass rating tool changes to Platinum users
Mon 28 Dec 2009 09:12:18 AM UTC — Deborah S.Over a year ago, WOT introduced a mass rating tool for our Platinum and Gold level users. The mass rating tool is designed for multiple submissions with matching ratings and comments, and has proven to be quite handy.
Unfortunately, the tool has been abused. Specifically, some users have been copying published lists and have not double-checked for accuracy. Additionally, corrections are not made in a timely way, and previous ratings are not being reviewed. Our experienced, trusted reviewers work far too hard to validate and assess individual ratings for us to allow their work to be tainted by those with less-than-reliable processes or motives.
Beginning today, we are limiting the mass rating tool to our Platinum users. Current Gold-level members will also be able to continue using the tool, but new Gold-level members will not have access until they reach Platinum status.
WOT's goal, and that of the majority of our community, is to maintain high quality, reliable ratings so that everyone will have a safer experience while shopping, browsing and searching the Web. We appreciate your comments and suggestions.
Fair Enough
Makes sense, platinum users wouldn't have as much motive to abuse the system as gold.
Glad to see this change..
People copying hosts files from other sources is really not necessary..
Peace
---------
Jared Gray
This kind of restriction is too crude
It's a very powerful tool and It seems to be misused way too often (not necessarily abused) e.g. by just copy-pasting all sorts of blacklists. This way, the proportion of rating errors is very high. This poses a risk to the quality of ratings and hence to WOT itself; therefore the use of the mass-rating tool should be very restricted, but in a fair and nuanced way.
Offering this tool to someone just because of a certain 'status' is not a good idea. It should not be seen as a reward but just as a tool. People may be prolific raters, but it does not mean they know how to use the mass-rating tool right, and the opposite might be true.
I suggest revoking the access to the tool for everyone and granting the use on an individual basis (people can request the right to use it). The amount of people that *really* has use for this tool is small enough to do this (there may be a one-time-only administration spike). It's also a good idea to monitor the use of the tool. You don't have to exclude 'gold-members' this way.
Education of users wanting rights to the tool is necessary. Also clearly define what the tool is *not* for.
Re: This kind of restriction is too crude
That's a good idea and I'm sure it's the next step we'll take if the current restrictions won't stop people from misusing the tool. Of course, we do take away access to mass rating from users who don't seem to understand it's purpose already.
Restriction seems to be discriminatory
One can not generalize an entire group of users just because of a few members who misuse a privilege. The better course of action should have been to exclude people from this privilege on a case by case basis rather than generalizing the Gold Status members as a whole. I agree with the comments posted in the above post.
Misuse of mass ranking - an example
I've just come accross a site that by most standards would be considered pretty trustworthy, the "Achaeology Data Service" of "The University of York" (ads.ahds.ac.uk).
It gets a very bad red WOT score, based on what appear to be copied entries from third-party blacklists. I checked out one of the mentioned blacklists and the site is NOT listed in that blackist. Perhaps it once was and has since been removed by the blacklist manager. Maybe the blacklist manager realised that it was entered in error. But there has been no corresponding withdrawl of the negative WOT ranking.
It appears that the original entry was copied by a Platinum WOT member and because of their Platinum status it appears that their ranking carries more weight, so several positive individual ratings by casual but consciencious WOT users do little to correct the ranking.
I think WOT is a great service, but it's users will lose trust in WOT itself if you don't do something to prevent blacklist copying by members, regardless of whether they are platinum, gold or, like myself nickel-alloy.
I'm sure that there are a lot of dubious web-site owners out there who would like nothing more than for WOT to be discredited. I value WOT and don't want this to happen.
Stopped using it anyway
While I'm "entitled" to the mass rating tool, I've stopped using it for the very reasons that some have articulated here. Is basically like using a sledge hammer to kill a fly.
Same here
I think at this stage where there's been a change of mind here, I think I can now say that I too have stopped using the tool a long time ago, because of just those reasons.
--
Per aspera ad astra
Stopped using it anyway
While not having access to the mass rating tool myself I am not sure that mass rating is the way I would choose.While it may get you into the top 100 very quickly I think that there is no substitute for individually visiting a site and seeing exactly what is their.Then you know exactly what you are rating and why.
Makes sense
I agree to your point of view
there are instances
where the mass-rating tool is handy.
Examples:
Pharma Bonus
Canadian Online Pharmacy
Authorized Software Resellers aka Online Store
CDC emails on H1N1 vaccines and personal profile
to list a few...
-------
WOT Services Ltd. - gives us safety through Web of Trust.
WOT Community - gives us security through unity.
Thank you all
- G7W
I agree. I've stopped using
I agree. I've stopped using the mass-rater for lists like hpHosts, but when people like you make big verified digs, I'd certainly rate and comment them, and the mass-rater is useful there.
@G7W
I have no objection with someone of your apparent skills using the Mass Rating Tool.But there are obviously some users whom appear to be more interested in getting to the top spot rather than the integrity of their ratings.
Some users appear to be using old lists without checking if the ratings are still valid.To the detriment of site owners and the reliability of WOT itself.
Perhaps it would be a good idea if WOT only gave this tool to members it trusts and not at what rank they are.It is not something I would want to use myself but I can see the advantage in only but the most experienced and trusted hands.
Or people could request the use of the tool at platinum level and then WOT decide themselves if a user is right for it or not.
Just a opinion,but reading this thread a lot of users seem worried about certain users having this tool.
Too much emphasis on activity level
Some users place entirely too much emphasis on activity level.
-
Yes that is surely a major problem, and I'm quite sure that is the only reason why a lot of mass ratings have been made...
The activity score is a good way of motivating people, but that can sometimes go too far and then the WOT staff have to react, as it has happened now.
--
Per aspera ad astra
'Speriment
First of all, your post is excellently put.
"The activity score is a good way of motivating people"
Yes, I agree, but that concept may be obsolete now that the WOT membership is high.
Would be interesting to see if elimination of activity scores/levels would cause a decrease in rating activity. If so, that would seem to indicate that some users are rating simply to increase their activity scores. Those continuing to rate at the same frequency would be those that rate for altruistic reasons, which IMO is the legitimate reason.
Haven't thought this idea through yet, so there may be disadvantages to it too.
Just thinking out loud at this point. Seems like at the very least, this might be an interesting 'speriment.
re Speriment
It would be interesting if mass raters like G7W stopped posting their mass ratings so that other mass raters would have to rely on their own skills,or lack of to compose their own lists to submit to WOT.
Also they would have to stand by their own ratings instead of putting source wxwmywotG7W so as to pass back the responsibility of their ratings to poster.
As it is they get the points but avoid the responsibility.Cut Copy Pasting your way to the top means nothing.Its a bit like cheating on a exam card.That is my observation at any rate.
-
Would there actually be a technical way to make ratings done through the mass-rating tool less rewarding of some kind, so that there is less potential of abusing it just for the activity score? I mean, if we're talking about how to prevent people from doing that, that would probably be the most effective way wouldn't it?
--
Per aspera ad astra
Easier
Whichever way would be easier for the WOT folks . . . eliminating activity scoring or reducing activity points for mass rating . . . seems like the 'speriment result would be the same.
-
mmm at least at the lower ranks, the activity score is a good way of starting to motivate people to rate in the first place, so I guess if it was completely removed, the overall rating activity on WOT (even for "honest" ratings) might suffer mightn't it?
--
Per aspera ad astra
Still thinking on it
Perhaps . . . your point does have some merit. Haven't yet refined the idea in my head. Probably something that should be "brainstormed" for a while.
Re: -
Yes there is, and that's a good idea. We'll think about it.
my opinion
The Mass rating tool could be applied to Gold members as well as platinum members, but with one restriction.
The MassRatingTool could be only used for the category of sites that g7w already mentioned:
Pharma Bonus
Canadian Online Pharmacy
Authorized Software Resellers aka Online Store
CDC emails on H1N1 vaccines and personal profile
If the WOT Programmers would enable a system for categorizing Site like already mentioned, it would be possible to lock the Tool for only these Sites. It would prevent abusing the Tool in a way we already know.
I appologize for my bad english, but i hope it is good enough to understand what i am trying to say.
--------------------------------------------------------------------------------
German/Deutsch
Ich denke, dass das MassRating Tool weiterhin für Gold und Platin Members nutzbar sein sollte, aber mit einer Einschränkung.
Das Tool sollte nur für die von g7w genannten Kategorien funktionieren, da hier durchaus sinnvoll:
Pharma Bonus
Canadian Online Pharmacy
Authorized Software Resellers aka Online Store
CDC emails on H1N1 vaccines and personal profile
Wenn die WOT Programmierer Seiten-Kategorien implementieren würden, wäre es möglich das Tool nur für diese Kategorien einzusetzen. So könnte kein Mißbrauch mehr in der bisherigen Größe stattfinden.
_________________________________________________________________
Going for Gold :D
-
Hm, but there are far more dangers on the internet, and they're constantly changing.
So if it was limited to certain sites, you could just use automatic blacklists and close the mass rating tool completely right?
Btw, first you say, "The Mass rating tool could be applied to Gold members as well as platinum members" and then, "Going for Gold :D". Wasn't that what we've been talking about here? ;-]
--
Per aspera ad astra
Mass Rating
I think warm_fuzzy_feeling is on the right track.Award it to people that have a good security based knowledge and forget about giving it to gold or platinum members.You could get to platinum and know ziltch but some raters are good as soon as they join.
Perhaps Sami could work out a logarithmic method to pick accurate raters.We need quality and not just quantity. :)
Nice idea
Nice Idea of you Silver Pirate. An algorithmic thats makes Reputation on users :D
A hypothetical question of me...
What if somebody creates his own MassRatingTool, with for example C++ or AutoIT.
Will this abuse be detected? If it will not be detected it doent matter if you lock MassRatingTool on Platinum Members. How long will it take till the first inofficial Tool will be in the Internet?
__________________________________________________________________
Going for Gold :D
Re: Nice idea
Will this abuse be detected?
Yes.
Thats
Thats great
__________________________________________________________________
Going for Gold :D
Good job
I think this idea was good, because platinum members have no real reason to abuse this power, as with some gold members would just use it to gain activity points for platinum status. Plus platinum members have a lot more experience and know how to use it properly. Good thinking WOT.
The "Top 100" board is a problem too.
Many Platinum WOT members may also abuse the Mass Rating Tool just for the sake of getting into the Top 100 list. Like what Dante v3.11 has mentioned, I think using the algorithm that is used for verifying a particular WOT users' trustworthiness is the way to go. Just give the Mass Rating Tool to those who are trusted (no matter what their rank is), then will they only rate each and every site properly and accurately.
Virtual Protection for Your Virtual Experience. Get Into the Vault, Now.