Email or username:

Password:

Forgot your password?
Graham Sutherland / Polynomial

that "Google made their search results worse on purpose to get more ad impressions" article that's going around is based on an internal email chain from a legal filing, in which an employee mentions this potential approach for increasing engagement metrics... as an example of a bad approach and not something they'd be comfortable doing.

based on my reading, there's nothing in here that says they actually did it. I can't see anything in there that supports that claim.

justice.gov/d9/2023-11/417557.

40 comments
Graham Sutherland / Polynomial

the article links to the results of a survey on people's perception on the quality of Google results over time, showing a negative trend, and I think that's something we all share similar anecdotal feelings about. but then it goes on to make the direct claim that they did it on purpose, which is (as far as I can tell) a claim without any concrete basis.

Graham Sutherland / Polynomial

certainly unsatisfying that Google is not unambiguously the bad guy in this case, but since there's no actual evidence of malfeasance it's just straight up misinfo

Graham Sutherland / Polynomial

the fact that search quality seems to have dropped in the months after this email chain doesn't concretely support a causative link or intent. the push towards AI summaries likely impacted a lot of that. do I think it was pure incompetence? probably not. do I think it was 100% premeditated malice? also probably not. my gut says they expected AI summaries to increase engagement, and then they did but not for the right reasons, and then multiple incentives coincided so they kept it.

Graham Sutherland / Polynomial

maybe it'll turn out they really did do it on purpose. wouldn't put it past them. but for now we have no solid evidence that they did, even if that narrative would fit perfectly with public perception and make for great headlines.

Graham Sutherland / Polynomial

still though, fuck Google, for so many other reasons. just probably not this specific one.

Dave Anderson

@gsuberland I have no privileged information either way, but my thesis for why Google search got worse is much simpler, and tragic: high quality search is incompatible with corporate incentive structures.

Google's search quality back in the day were upheld by a _massive_ investment in search quality. An entire arm of the company operated more like a research lab than a company, trying all kinds of wild new tweaks to ranking, running experiments, developing new methodologies to quantify "how much gooder are these results for this query". It seemed to me like they spent a ton of time and money trying and abandoning things that didn't work out, to get the occasional payoff of moving the quality metrics by a few percent, or tenths of a percent.

When Google entered its bean counter era, I'm morally certain they looked at Search Quality and went "wow your org has terrible ROI, let's get rid of all this stupid open ended research and just rub some machine learning on the problem, that's much more efficient, we can scale that up just by turning this here knob and nobody has to think about it any more."

I have no proof for any of this, just a deep conviction that the way Search Quality operated is the reason the search results were so good for so long, and also would not survive contact with an efficiency-oriented executive.

@gsuberland I have no privileged information either way, but my thesis for why Google search got worse is much simpler, and tragic: high quality search is incompatible with corporate incentive structures.

Google's search quality back in the day were upheld by a _massive_ investment in search quality. An entire arm of the company operated more like a research lab than a company, trying all kinds of wild new tweaks to ranking, running experiments, developing new methodologies to quantify "how much...

Pierre Bourdon

@danderson @gsuberland the second half of the problem is that search quality is a problem becoming harder over time, so even if you maintain the same level of investment your quality goes down.

(Reasons include: expectations rising, amount of content on the internet rising, SEO being more and more adversarial, giant information silos like FB/Discord, paywalls, abuse profitability rising, and probably dozens more.)

Dave Anderson

@delroth @gsuberland Yup, that's another piece of my usual rant about this: to some extent, search got worse because the spammers won the arms race, not because google explicitly decided to make search worse. In an arms race, you can lose just by not pushing harder.

Again I have no evidence either way so we'll see what the investigations show... But given google's trajectory overall in the last decade or so, I find neglect to be a more compelling explanation than a vast web of conspiracy, or the actions of individual supervillains.

@delroth @gsuberland Yup, that's another piece of my usual rant about this: to some extent, search got worse because the spammers won the arms race, not because google explicitly decided to make search worse. In an arms race, you can lose just by not pushing harder.

Again I have no evidence either way so we'll see what the investigations show... But given google's trajectory overall in the last decade or so, I find neglect to be a more compelling explanation than a vast web of conspiracy, or the...

Az

@gsuberland is that the Where's Your Ed At article from April '24 (wheresyoured.at/the-men-who-ki)?

That one points to changes from '19 which was well before the GPT and AI summary stuff AFAIK.

Graham Sutherland / Polynomial

@az no, but that article has its own problems and also fails to present much in the way of real evidence.

Taureon

@gsuberland also we shouldnt forget that search quality gets worse because people over time figure out how to SEO their shitsites
that probably makes a sizeable amount of why

miunau

@gsuberland its frustrating, like with the soundcloud ToS bad faith reading thing. i had someone back up to a corner of "well you cant trust companies anyway" like guh

computerywar

@gsuberland Ed Zitron went into a lot of detail regarding this a bit ago if you want to read: wheresyoured.at/the-men-who-ki

Graham Sutherland / Polynomial

@computerywar right, that essentially says the same thing: the email chain is suggestive of there being a push towards lowering search quality to increase engagement metrics, and there was pushback. search quality generally dropped, and an insinuation is made that it was intentional and causative, but no actual evidence is provided to support the claim.

shironeko
@gsuberland I think it's quite obvious at least that SEO slop is winning against Google. Before LLM at least it's hard to come up with a reason why it would be the case without Google being complacent.
leo vriska :light2:

@gsuberland oh wait no that was about a different study. still (the judge claimed that google's numbers found that worsening search results would "only" have a 1% drop in revenue when according to him the actual conclusion was that ranking quality is really important if it can affect revenue as much as 1%)

Pierre Bourdon

@gsuberland also that survey is abysmal. It was done by a random credit card comparison company, they don't state a methodology other than "online lol" (on 2 different dates 6 months apart, all mixed together?), and the questions are basically all leading questions.

Graham Sutherland / Polynomial

@catsalad @sjb yeah, but again, circumstantial. none of this adds up to a smoking gun. do I think they're shady as shit? yup. but it's counterproductive to jump to a conclusion.

Sebastian Zdrojewski

@gsuberland @catsalad @sjb wasn't there a talk years ago about how bigG didn't know how their ranking system actually "behaved"? I remember this coming out a few years back too (I may be mistaken) which also pushed for SEO related discussion and doubts.

seroundtable.com/google-dont-u

0xDECAFBAD

@gsuberland @catsalad @sjb Occam's Razor.
And you can convict on circumstantial evidence, I assume the government did not include this in their case filing because it was an interesting research study.

Graham Sutherland / Polynomial

@slyborg @catsalad @sjb there are lots of other reasons why they might have included it. establishing a timeline of events, or demonstrating that a concern was raised and they were aware of it. legal cases have a lot of tedious paperwork that doesn't do anything except establish one specific fact.

Thomas Lobig🐓

@gsuberland acute observation, as always.
Just a little thought here. Based on an anecdotal story about a company that made strategic decisions based on an Excel sheet that a third party provided purely as an prototype, filled with numbers they had pulled out of their ass: It seems likely to me management got wind of the idea, ignored the context and made it happen. Large companies are weird organisms.

CiarĂĄn McNally

@gsuberland I was thinking the same but also fuck Google and their genocide tech

Graham Sutherland / Polynomial

@ciaranmak that goes without saying. just annoying that convenient but unsupported narratives get in the way of actually holding them to account.

CiarĂĄn McNally

@gsuberland The alternative is they made it worse for no reason to push an unclear AI adoption agenda.

Graham Sutherland / Polynomial

@ciaranmak if the two are related, my guess is they pushed the AI features expecting engagement growth because users like it, then they did get engagement growth but it was actually because the AI summaries suck and people keep having to dig further to find and verify information, and then they figured "well ok that didn't go exactly as planned, but since the metrics are up and we really want to sell the AI stuff..." - I can see that being hard to argue against when everyone's incentives align.

Sablebadger

@gsuberland regardless of whether the article is legit, it's hard to escape the fact that google searching, and search results HAVE gotten far worse. so it makes for a believable story.

Thanks for the deep dive into the source material though, it does help to know what facts vs. rumors.

andrea-smandrea

@gsuberland i recommend you listen to Cory Doctorow’s podcast “Understood: Who Broke The Internet?”, specifically the episode “Don’t Be Evil.” it basically covers everything you’re calling into question.

Graham Sutherland / Polynomial

@andrea_smandrea yes, I have read the article version of that, and it provides nothing to contradict what I wrote here. claims are made, no evidence is provided, and there are a few bits that look to be patiently wrong in hindsight.

Hey Gus

@gsuberland I swear they did and then rolled it back, but I don’t want to read all of that again.

My takeaway was a sharp reminder that google, like meta, are advertising companies that have various web services as well.

The email chain is sad. Like the slow death of a passion for technology getting fisted to death by greed. Like bowels perforated dead.

Graham Sutherland / Polynomial

@elebertus I think you're thinking of the thing where they deranked Wikipedia as an experiment for a few weeks as an experiment? and it hit their revenue by 1%, which people took to mean "they decided they could lower quality without losing revenue", but then it turns out 1% is considered a huge loss at that scale and the actual outcome was "we are definitely not doing that".

zauberlaus

@gsuberland well ok. except that their search sucks ;)

RegressionToTheMeme

@gsuberland Raghavan destroyed it intentionally, just like he tanked Yahoo search.
He's a clown who claims to be a computer scientist, but hangs with the marketing creeps.
He undermined and backstabbed his way to control of Google Search, and trashed it to improve its 'marketing metrics'.
He did everything he could to force people to spend longer on the website, and to obscure the amount of paid advertising the users were fed.
Yahoo search was the biggest name in the game until he got hold of it. He ran it into the ground so bad that they sold it for chump change and contracted the searching to Bing. 🤣

@gsuberland Raghavan destroyed it intentionally, just like he tanked Yahoo search.
He's a clown who claims to be a computer scientist, but hangs with the marketing creeps.
He undermined and backstabbed his way to control of Google Search, and trashed it to improve its 'marketing metrics'.
He did everything he could to force people to spend longer on...

jaseg

@gsuberland I think this way of thinking also tends to anthropomorphize the company too much. When large organizations do something truly bad or stupid, it often arises from bad incentive structures and everyone involved “just doing their job”. I think it’s important to acknowledge that shit like this can happen without any actual bad intent or conspiracy, just out of bad organizational practices and lack of care.

Go Up