Hiring people to review reports

[Deepbluediver]

There seems to be an awful lot of hate for the reporting system today. It's got its problems, but realistic answers for improving it seem to be few and far between.

One of the most common suggestions is that a Moderator or Admin should review every report to confirm it's authenticity, because Valve makes tons of money and could totally afford it.
And I started wondering about that.


How many reports get filed? I have no idea. Some people use all their reports every week, and then use up the ones they get back, too. Some people (such as myself) hardly report at all. Maybe it's an average of 1 report per player per week. How many people play DotA? Lets say 6 million people; I think that's about roughly correct.

So 6 million people at 1 report each per week is 24 million per month.

How long does it take to review a report? Maybe some, like real intentional feeding, might be short. Others, such as ability abuse, might take longer, especially if the reviewer has to search for it. Lets assume that they can review 1 report every 2-3 minutes.
In an 8-hour shift, 1 reviewer could review maybe 200 reports. That's 1,000 reports per week, or 4,000 month.

So 24 million reports divided by 4,000 reviews, means we need 6,000 reviewers to keep up.

Also, lets assume that there's no problems with language and we can outsource all of this to someplace with cheap labor, like India, where the minimum wage is about 3 dollars per day, or $780 per year. So to hire 6,000 reviewers would be $4,680,000 per year.

Best case scenario- it costs almost 5 million dollars per year to hire minimum wage indian employees to review these reports. If you want someone who speaks the language the report is filed in or has a better education, then that's rapidly going to multiply. If they had to hire all American employees, (at a minimum wage of 8 dollars per hour) the amount would rapidly approach $100 million.


Just keep all that in mind when you want someone to hand-check every single report before it gets acted upon.

[noname]

With all due respect, (and I definitely agree with you as I have made the same argument with less depth), even crunching the numbers for people isn't going to stop the stupid "LP cause report" threads. The people who need to read this will not read it.

[256mb emoji cache]

I feel like it's significant that the people who are whining about the report system being broken act like they deserve it in their posts.

[Weird Grim]

hire players do that and pay them 0.50 per houre in the steam wallet do 8 houres and you got 4.00 then you can buy a game on sale or some items in the market

[DARKO]

Originally posted by Weird Grim:

hire players do that and pay them 0.50 per houre in the steam wallet do 8 houres and you got 4.00 then you can buy a game on sale or some items in the market

GREAT IDEA +1

[corbet]

No man power should be wasted on reviewing reports.

No free money should be given to desperate kids wanting to do this. Giving away steam money for a task like this that benefits no one, is dumb. people should work in real jobs, and use that real money on steam.

[DARKO]

Originally posted by Deepbluediver:

There seems to be an awful lot of hate for the reporting system today. It's got its problems, but realistic answers for improving it seem to be few and far between.

One of the most common suggestions is that a Moderator or Admin should review every report to confirm it's authenticity, because Valve makes tons of money and could totally afford it.
And I started wondering about that.


How many reports get filed? I have no idea. Some people use all their reports every week, and then use up the ones they get back, too. Some people (such as myself) hardly report at all. Maybe it's an average of 1 report per player per week. How many people play DotA? Lets say 6 million people; I think that's about roughly correct.

So 6 million people at 1 report each per week is 24 million per month.

How long does it take to review a report? Maybe some, like real intentional feeding, might be short. Others, such as ability abuse, might take longer, especially if the reviewer has to search for it. Lets assume that they can review 1 report every 2-3 minutes.
In an 8-hour shift, 1 reviewer could review maybe 200 reports. That's 1,000 reports per week, or 4,000 month.

So 24 million reports divided by 4,000 reviews, means we need 6,000 reviewers to keep up.

Also, lets assume that there's no problems with language and we can outsource all of this to someplace with cheap labor, like India, where the minimum wage is about 3 dollars per day, or $780 per year. So to hire 6,000 reviewers would be $4,680,000 per year.

Best case scenario- it costs almost 5 million dollars per year to hire minimum wage indian employees to review these reports. If you want someone who speaks the language the report is filed in or has a better education, then that's rapidly going to multiply. If they had to hire all American employees, (at a minimum wage of 8 dollars per hour) the amount would rapidly approach $100 million.


Just keep all that in mind when you want someone to hand-check every single report before it gets acted upon.

1./ u DONT need controll EVERY report so u can divide that number by 10 or more
You can controll random report.
2./ U dont need all full-time workers. U can pay players for do modderation stuff and just hire Supervisors who will random contorlls.

3./ VOLVO make ton of moeny so could totally afford it.

4./ There is only few main languages English , Spanish, russian, Chinese, Korean...
so dont be so sceptical

[[))=333]CorazónDeChurreta]

Originally posted by M. Jollinir, Hammer of the Johns:

I feel like it's significant that the people who are whining about the report system being broken act like they deserve it in their posts.

Did you reported once? Is just a three button window and only can be used in-game, while you should pay attention to the match.

And it cannot cover intentional AFK issues (moving around to avoid DC penalty) or skill abuse (already exists, but is not simple to prove). That means to give the chance to include a DETAILED report, as happens on old dota lounges. That also diminishes the chance to automate reports (current issue) and exists tools like dotabuff to see details of match (that uses some kind of Steam API).

And of course, improve penalties to something better than LPQ joke.

[DARKO]

And another improvement. You dont need controll every random report. You just need check last one report , that trigger LPQ punishment.

[256mb emoji cache]

Originally posted by darko:

And another improvement. You dont need controll every random report. You just need check last one report , that trigger LPQ punishment.

That doesn't make any sense.

[Deepbluediver]

Originally posted by Weird Grim:

hire players do that and pay them 0.50 per houre in the steam wallet do 8 houres and you got 4.00 then you can buy a game on sale or some items in the market

1) That's less than minimum wage in most western/developed countries, which makes it illegal.

2) I'm pretty sure that it's also illegal, for an entirely separate set of reasons, to pay people in credit. They used to call it company scrip: http://en.wikipedia.org/wiki/Company_scrip

3) The reports are being reviewed by the same people who issued them in the first place, the players. What's to stop a Russian from reporting someone for not being Russian and a Russian reviewing for agreeing that that is totally a reportable offence?
I fail to see why you think the players will be any smarter or less troll-ish the second time around.

4) There's no quality control whatsoever- if I get paid by the hour, then what's to stop me from clicking "Begin review" and then walking away from my computer to make toast for 40 minutes while the game plays. If I'm getting paid by the review, the best thing I can do is just click the "guilty" or "innocent" button as fast as I can, without watching a single minute of gameplay.

5) At that rate I need to work for 5 hours and review 100-150 reports to earn the equivalent of a single chest. I don't know about you, but in 5 hours there's a lot of other things I'd rather do with my time, such as work a real job and earn enough to buy several chests.
Unless, of course, you are OK will all your reports being reviewed by 13 year olds.

Originally posted by darko:

1./ u DONT need controll EVERY report so u can divide that number by 10 or more
You can controll random report.

How does that work? Caching only 10% of all the reports means that 90% will still go through, which includes 90% of the bad reports.

2./ U dont need all full-time workers. U can pay players for do modderation stuff and just hire Supervisors who will random contorlls.

That's problematic for all the reasons I outlined above, including that just because you pay each person less it doesn't make the total cost needed to review everything any less.

3./ VOLVO make ton of moeny so could totally afford it.

I believe my original post was all about that fact that no, they can't.
If you want me to take you seriously, post your own math for determining that conclusion.

4./ There is only few main languages English , Spanish, russian, Chinese, Korean...
so dont be so sceptical

So what happens to reports made in French? Or Brazilian? Or Filipino?
I'm skeptical because it's a huge undertaking that shows absolutely no promise that it's going to be any better than the current system.

[GoabNZ]

I would love that. Too many times you get idiots who watch you make one mistake, or just be unlucky and "OMG feeder! Reported!"

Why, just yesterday, I was playing LC, three of our team fed Skywrath Mage hard. Knowing that he is still super squishy and that I have a bit of farm, if I can get a duel I can stop him. Multiple times I get a chance, but lack of wards from our feeders (even Sniper, who fed the most and tried to build a Skull Basher of all things) meant I never had the vision I needed. One fight, I had smoke to gank Skywrath, but he managed to chuck up his ult as I began to duel him so that killed me. And then one player started shouting that it was intentional feeding, and ability abuse and everything. One mistake, one bad play - it happens - that is not feeding and the report needs to be discarded. Besides, they were feeding Skywrath, did they see me complaining? Thankfully, they fed only Skywrath and the rest of the team was terrible and fed our FV, so we won.

Even if we had to pay for our reports to fund the reviewers, I would gladly do it. You can have as many as you like, and if the player reported is deemed to be at fault, you get a report back. This way, reporteds aren't based upon the number of reports a player gets, but what a hired reviewer feels is bad playing. Players can abuse reports at their own cost, but it wont necessarily do anything.

[puddy tat]

Originally posted by darko:

Originally posted by Deepbluediver:

There seems to be an awful lot of hate for the reporting system today. It's got its problems, but realistic answers for improving it seem to be few and far between.

One of the most common suggestions is that a Moderator or Admin should review every report to confirm it's authenticity, because Valve makes tons of money and could totally afford it.
And I started wondering about that.


How many reports get filed? I have no idea. Some people use all their reports every week, and then use up the ones they get back, too. Some people (such as myself) hardly report at all. Maybe it's an average of 1 report per player per week. How many people play DotA? Lets say 6 million people; I think that's about roughly correct.

So 6 million people at 1 report each per week is 24 million per month.

How long does it take to review a report? Maybe some, like real intentional feeding, might be short. Others, such as ability abuse, might take longer, especially if the reviewer has to search for it. Lets assume that they can review 1 report every 2-3 minutes.
In an 8-hour shift, 1 reviewer could review maybe 200 reports. That's 1,000 reports per week, or 4,000 month.

So 24 million reports divided by 4,000 reviews, means we need 6,000 reviewers to keep up.

Also, lets assume that there's no problems with language and we can outsource all of this to someplace with cheap labor, like India, where the minimum wage is about 3 dollars per day, or $780 per year. So to hire 6,000 reviewers would be $4,680,000 per year.

Best case scenario- it costs almost 5 million dollars per year to hire minimum wage indian employees to review these reports. If you want someone who speaks the language the report is filed in or has a better education, then that's rapidly going to multiply. If they had to hire all American employees, (at a minimum wage of 8 dollars per hour) the amount would rapidly approach $100 million.


Just keep all that in mind when you want someone to hand-check every single report before it gets acted upon.

1./ u DONT need controll EVERY report so u can divide that number by 10 or more
You can controll random report.
2./ U dont need all full-time workers. U can pay players for do modderation stuff and just hire Supervisors who will random contorlls.
Speak english please
3./ VOLVO make ton of moeny so could totally afford it.

4./ There is only few main languages English , Spanish, russian, Chinese, Korean...
so dont be so sceptical

Speak english please.

[Samadhi]

There is no reason that a system similar to LoL's tribunal system cant work with dota2.
It makes sense to use the community as a means of self regulation, as its quite clear that the report system is a trolls playground.

[Deepbluediver]

Originally posted by RF Samaister:

There is no reason that a system similar to LoL's tribunal system cant work with dota2.
It makes sense to use the community as a means of self regulation, as its quite clear that the report system is a trolls playground.

Have you played League? More importantly, how well does the tribunal-system work overall?

Just by reading through their website, it seems like the biggest drawback would be that it can be slow to react. How many days could I spend ruining games before I accumulate an above-average number of reports and enough players review everything to render a verdict?
It's not hard to find threads like this: http://www.reddit.com/r/leagueoflegends/comments/23dkvx/for_the_love_of_god_riot_please_do_something_with/

[Samadhi]

Originally posted by Deepbluediver:

Originally posted by RF Samaister:

There is no reason that a system similar to LoL's tribunal system cant work with dota2.
It makes sense to use the community as a means of self regulation, as its quite clear that the report system is a trolls playground.

Have you played League? More importantly, how well does the tribunal-system work overall?

Just by reading through their website, it seems like the biggest drawback would be that it can be slow to react. How many days could I spend ruining games before I accumulate an above-average number of reports and enough players review everything to render a verdict?
It's not hard to find threads like this: http://www.reddit.com/r/leagueoflegends/comments/23dkvx/for_the_love_of_god_riot_please_do_something_with/

Compared to the absolute absurdity of this system, slow but accurate is preferable to the status quo, IMO.