65 points sammy0910 1 hour ago 32 comments

ben_w 1 hour ago | parent

It's too late to comment on your older posts, but I'd just like to say: it's nice to see someone making oldschool web browsers work properly on the modern web.

https://www.sammystraus.com/#using-llms-to-make-the-emacs-we...

sammy0910 48 minutes ago | parent

:) ive been dreaming of a usable emacs browser for so long

vanschelven 59 minutes ago | parent

I _also_ posted to HN and made it to #1[0]; the effect this had on my CPU/network traffic on a 5/EUR month VPS over at Hetzner was in the single digit percentages.

As per the wiki page for the C10k problem[1] where it says "_was_ [emphasis mine] the problem of optimizing computer networking stacks to handle a large number of clients at the same time". This was solved over a decade ago...

Now get off my lawn!

(TBF I think the author acknowledges this)

[0] https://news.ycombinator.com/item?id=43725815

[1] https://en.wikipedia.org/wiki/C10k_problem

dboreham 41 minutes ago | parent

The c10k problem is a different problem. It relates to simultaneous open TCP connections at a server. It would take a very very large number of crawler bots to force open 10k connections, I suspect.

bix6 58 minutes ago | parent

> one specific user. I won't name names

Who is?!

sammy0910 45 minutes ago | parent

haha im not naming names

nekusar 37 minutes ago | parent

You shouldnt defend bad actors and abusers. Hiding them is tantamount to them attacking others indiscriminately. Cause youre not the first victim, and definitely not the last.

You see something, you say something, or you do something.

spiderfarmer 46 minutes ago | parent

Try posting something viral on Mastodon. The logs clearly show that it isn't the most efficient social network.

dyingkneepad 7 minutes ago | parent

Can you elaborate more, please? I'm curious.

oneeyedpigeon 39 minutes ago | parent

What confused me most about this is that the entire site is, apparently, a single page (there's no sitemap that I can use to verify this; there IS an RSS file which appears to confirm it) containing all posts. Links at the top (headed "Table of Contents") are all in-page links.

At first, I assumed this was a single blog post, especially with a "Table of Contents" at the top. I guess it's following some old-school blogging conventions, but AFAICR, even those sites retained individual pages for separate posts.

sammy0910 33 minutes ago | parent

it's org-html-export-to-html babyyy lmao

sammy0910 30 minutes ago | parent

I think I could improve the format a bit with something like Jekyll, but I haven't had a good chunk of time to sit down and work on it

kqr 3 minutes ago | parent

You can also link between Org documents and org-publish will export the documents with working web links, if you're lucky. That's how https://entropicthoughts.com works.

switz 37 minutes ago | parent

There is so much spam, abuse, and aggressive scraping that I've had to block:

  - AWS
  - Google Cloud
  - Azure
  - Alibaba Cloud
  - Oracle
and a handful of other ASNs on almost all of my properties. It would be one thing if it was simple scraping, but it's often genuine agentic abuse (just spamming for vulnerabilities* or scraping millions of pages with very little valuable or dynamic content). These clouds must get the abuse happening under their own roof under control.

I banned Alibaba's Cloud two years ago and it still hits one of my domains 1 million times per day, despite every route 4xx-ing during that time.

Even Apple has aggressively ramped up its scraping in the past few months, but thankfully they do respect robots.txt.

It's frustrating because some of my real users route through these clouds via their work VPNs. Also, there are a lot of less-than-reputable AI scrapers using global bot-nets via residential IPs to scrape, making it even harder to detect and ban them without affecting real people.

* like leaked .env routes, brute forcing admin panels, and so on

Sharlin 24 minutes ago | parent

Application-level banning seems very inefficient, these domains should be banned at the network (IP) level. Just drop any incoming packets from these domains/IPs, send absolutely nothing back that even confirms that you exist.

switz 13 minutes ago | parent

They are banned at the edge in Cloudflare, they never hit my application.

simmerup 12 minutes ago | parent

Isn’t scanning for vulnerabilities illegal? Traditionally security researchers had to be very careful about such things

dawnerd 3 minutes ago | parent

I’ve had to block similar on clients sites. Their search pages would start to get hammered with obvious agentic search spam. Searching for thing not even related to the site in all combinations of the queries. Some of the user queries contained potentially PII.

paaloeye 34 minutes ago | parent

Have you tried dealing with bots using CF Turnstile? Ref: https://www.cloudflare.com/products/turnstile/

sammy0910 27 minutes ago | parent

not yet, but I think something like this would be a next step

rrr_oh_man 11 minutes ago | parent

I did this for alyph.ai, but was recently flooded with so many browser-using, Turnstile-defeating bots from Taiwan that I had to completely turn off sign ups. It's not fun anymore.

Not even third party login worked (Google, Linkedin, ...) to curb this.

They were logging in, scraping the client API for vulnerabilities/possibilities and then spammed erotic dog poems to Claude Fable until the $5 welcome credit got drained.

Fun New World.

ChrisMarshallNY 30 minutes ago | parent

Heh. I posted a GitHub project Show HN, a couple of days ago, and it never even made it out of new, but it got me some bots.

That's a big reason that I never post any of my apps here.

Welcome to hell, kid...

janalsncm 29 minutes ago | parent

Ok, on the one hand to have one bot hitting your website to populate some kind of slop newsletter is pretty annoying.

On the other hand, “multiple times per minute” doesn’t seem that high? Even if it was 60x per min that’s fully within the limits of what a 2015 laptop in the closet can handle.

sammy0910 18 minutes ago | parent

yeah, it was a wake-up call that I needed to make certain parts of my application faster, and I've been working on it for the past few weeks

mococa 21 minutes ago | parent

Same with me, every-time I got 1st page I got a lot of robots

Havoc 20 minutes ago | parent

> I won't name names, but I looked them up, and they describe themselves as a service that listens for changes to a website, and then summarizes them into an AI slop news letter. And by 'listens to changes' I mean query our website many many times per-minute. Blocked.

That sort of behavior would justify a name and shame imo

Inconsiderate shitty services don’t deserve the politeness of protecting their name