You are here

Google has "human raters" in its search "algorithm"

Today Google publicly admitted for the first time that its purported "neutral" and "unbiased" search algorithm is not completely-automated or computer-algorithmic like Google has long and consistently represented to the public. 

  • In a stunning first-time disclosure in a Richard Waters FT article by "the Google engineer responsible for its ranking algorithm," Mr. Amit Singhal:   
    • "Google’s Mr Singhal calls this the problem of “brand recognition”: where companies whose standing is based on their success in one area use this to “venture out into another class of information which they may not be as rich at”. Google uses human raters to assess the quality of individual sites in order to counter this effect, he adds." [Bold added for emphasis.]

Wow. After a decade of passionate public representations that Google's vaunted search algorithm is "neutral' and unbiased, we now learn it has substantial regular human intervention to discriminate what site gets what ranking, who gets found and who does not, and who wins and who loses in the business of online content.

  • From Google's website:

    • "Ten things we know to be true:" "We never manipulate rankings to put our partners higher in our search results and no one can buy better PageRank. Our users trust our objectivity and no short-term gain could ever justify breaching that trust."
    • Code of Conduct: "...But "Don't be evil" is much more than that. Yes, it's about providing our users unbiased access to information..." [bold added]

Now that Google has let slip that it does have some type of official human editorial board adjusting/manipulating/editing search results, we better understand: 

  • What Mr. Amit Singhal meant when he described Google to the UK Telegraph as "the biggest kingmaker on this earth; and
  • Why Google SVP Jonathan Rosenberg said on the official Google Blog: "We won't (and shouldn't) try to stop the faceless scribes of drivel, but we can move them to the back row of the arena." 

Now that Google has publicly disclosed that Google employees do manually adjust/manipulate search rankings as part of the opaque "Quality Score" process that many companies have filed antitrust suits about ( TradeComet, MyTriggersFoundem, ejustice.FR, Ciao, Navx, etc.), antitrust authorities should want to know how these "human raters" maintain objectivity and what internal controls are in place for this human process to not be abused anti-competitively given Google's search advertising monopoly per the DOJ.   

Key questions for antitrust authorities:

  1. What are the formal guidelines and decision criteria that this equivalent of a Search Discrimination Board (SDB) follows to try and remain neutral and unbiased, and prevent anti-competitive behavior?
  2. Does the SDB have any specific written internal controls and procedures that document its many discrimination decisions? Is there a formal Google process to ensure the overall integrity of this human rater discrimination process?  
  3. Is the SDB transparent and do websites affected have any notification of how the SDB made its decision so there is due process to correct any factual mistakes?
  4. Is there any way an affected party can reach a human at Google to discuss the SDB's ranking of their content?
  5. Can anyone appeal decisions of the SDB? What is the process and when are decisions final?
  6. Does Google or any independent third party monitor or audit the SDB's deliberations to cross-check Google's public representations of unbiased search results?
  7. Can antitrust plaintiffs subpoena records of the SDB's deliberations?
  8. Are human raters involved in the ranking of political or religious websites? Are there any recusal procedures to ensure SDB human raters don't have the appearance of a bias or conflict of interest?
  9. Does the SDB attempt to capture the neutral and unbiased views, values, and concerns of the U.S? or the country the search is conducted in?
  10. Are the same SDB members deciding the rank of non-Google-owned content, the ones that decide the ranking of Google-owned content? How is that conflict of interest disclosed and managed? Are Google's top executives ever involved in how Google-owned content is ranked and if not why not?
  11. What is the criteria that the SDB uses to decide which news organization's reports rank higher among sites like NYT, FT, WSJ, FT, AP, Reuters, Fox, CNN, etc?
  12. If links are a factor in determining the rank of content, and Google's advertising revenue is derived from sites' search rankings, how does Google ensure the human raters of the SDB are not influenced to reward Google-owned content or Google partners' content that Google revenue shares with?  


In sum, this first-ever disclosure by Google that "human raters" manually discriminate in the "quality scores" that determine a website's supposed neutral and unbiased search ranking, exposes a rats nest of conflicts of interest that Google has in its "black box" business model. This is important because Google now is:

  • The world's de facto editorial filter by which Internet content gets found;
  • The only revenue collector for most of the world's websites; and
  • The dominant gatekeeper for any business seeking to reach Internet users and websites; ...and finally because...
    • Google is publicly representing itself as neutral and unbiased in order to maintain the trust of users, advertisers, publishers, and Governments around the world.    


For more information see: