51 points DeepLogin 3 hours ago 14 comments

Bender 2 hours ago | parent

still have the ability to disable the crawler via robots.txt flags

Never rely on robots.txt. Configure it, but do not rely on it. A bot will do what a bot can do. Their bots are trivial to spot so close the connection, bonus if done silently closing the socket and inhibiting first RST from web daemon and second RST from the OS.

cowlevel 1 hour ago | parent

Wouldn't feeding them false data be even better?

esseph 44 minutes ago | parent

We've already been using AI to target military strikes without human intervention.

Make of that what you will.

usrnm 43 minutes ago | parent

That could consume as much, if not more, resources as returning real data

zeryx 4 minutes ago | parent

Angel banning is better, 100% agreed on giving it false data that the scraper builders can't easily detect. Give it seemingly correct and useful but totally wrong answers

cyanydeez 2 hours ago | parent

Humanity experts warn oligarchs, "not good"

News at never.

jareklupinski 1 hour ago | parent

just rename the company to Meta Law or Zuckerberg & Associates already

shevy-java 1 hour ago | parent

The name Suckerberg I mean Zuckerberg is so strange. That's just raw german:

Zucker = Sugar Berg = Mountain (or a big hill, but typically it refers to a mountain)

So, a "mountain of sugar" (lots of sugar).

Suckerberg kind of fits better, since "sucker" is not german (berg still is of course; I think scandinavian countries also have berg, though that could be burg, aka castle). Plus, Suckerberg describes the evil behaviour of Facebook better. The USA really has a problem with its billionaires.

Scrapemist 1 hour ago | parent

What about Altman?

N_Lens 39 minutes ago | parent

Saltman

drooopy 20 minutes ago | parent

I read somewhere once that Sam Altman sounds like a name for a fictional villain made up by Hideo Kojima for Metal Gear Solid, and it will never not make sense.

fuzzy2 38 minutes ago | parent

Not strange. It's a Jewish name.

wongarsu 13 minutes ago | parent

[resource]+[mountain/stone/field/valley/tree] in German is a very common pattern for Jewish surnames. A quirk of how surnames were adopted in German-speaking regions in the 18th century

Sugar mountain at least makes sense, unlike e.g. wine stone

Heronsunny 49 minutes ago | parent

Blocking by UA is pointless now, meta-externalagent rotates and a chunk comes off non-Meta ASNs. We matched on ASN plus TLS JA4 and that actually stuck.