YHK - yet another hacker news clone

DeepSeek search and fetch – $0.75 per 1000 successful requests

124 points fagnerbrack 1 hour ago 53 comments

Doohickey-d 1 hour ago | parent

How well does this (and similar services) deal with anti-bot protection? What happens if I attempt to retrieve a page protected by the cloudflare checkbox or by Anubis? An error, no useful content, or the actual content I want?

m00dy 1 hour ago | parent

you can bypass those easily except Google Captcha v3, you need to have your own proxy swarm to make it at scale.

weird-eye-issue 1 hour ago | parent

There is no single answer to this, it entirely depends on the exact service and the exact page

Most US services like Claude and ChatGPT send a specific user agent that makes it easy to block them and specifically does not try to get around any sort of bot protections

Meanwhile other services could use a variety of tactics such as simply setting a browser user agent and/or using random data center IPs or residential IPs, etc etc and then the blocking would just come down to whatever the individual page has configured which can actually be pretty hit or miss

m00dy 1 hour ago | parent

I've spent a lot of time building datamint.xyz, and dealing with user agents, datacenter IPs, and VPNs gets pretty tricky. What works right now is a fork of the latest Firefox with anti-fingerprinting patches, paired with a high-quality residential proxy network with clean IPs. Even the basic setup needs 11+ code-level patches, and there are still a few edge cases, but this setup should cover 99% of cases.

thisismyanon2 51 minutes ago | parent

Thank you for escalating the arms race that's currently destroying the internet. I can't look at the Apple user forums, my health insurance company's portal, pay my county taxes, browse Digikey and Mouser, view Github or whatever forge, etc. etc. without having to jump through Cloudflare or Anubis garbage designed to block your abusive behavior.

I'm just overflowing with joy that you're comfortable wasting my time and money.

stavros 13 minutes ago | parent

Even if it's not the GP, if there's profit to be made, somebody will do it. I was reminded of the power of economic motives the other day, when it turned out someone bought an entire trustworthy reseller just so they could compromise the Ledger supply chain by including a tiny eSIM module that OCRed the screen and transmitted the seed.

Incentives are crazy powerful.

latexr 34 minutes ago | parent

I have to agree with the (sarcastic) sibling comment. You’re essentially making everyone’s life worse in the name of personal profit.

Please stop. The last thing we need in the world is more greed and selfishness.

lejalv 10 minutes ago | parent

Since when have moral appeals worked against the central feature of capitalism?

Only strong regulation can help.

ReptileMan 8 minutes ago | parent

>Most US services like Claude and ChatGPT send a specific user agent that makes it easy to block them and specifically does not try to get around any sort of bot protections

The last part is extremely dubious with some recent reveals.

jack_pp 5 minutes ago | parent

recent reveals show that OAI and Anthropic give users the ability to go around bot protections or they internally use it?

tyre 1 hour ago | parent

> Search API finds web pages for a query and returns structured results. It is powered by DeepSeek's in-house search engine.

Looks like they've built (or are building) their own index and return cached results. What you do with that information is up to you. If you want to visit the live page, you'd need to find a way around bot detection.

One reason (not the only one!) people use Anubis is to prevent the endless flood of requests from AI companies. If Deepseek scrapes once every few hours, that helps the website owner with the traffic problem while allowing access.

If they don't want to allow access, my guess is that deepseek's search crawler respects robots.txt but I don't know.

dsco 1 hour ago | parent

DeepSeek Flash really came at a bad time before the Anthropic IPO. I use this thing daily and it’s neck and neck with SOTA models. The search is one of the things setting it apart.

redox99 1 hour ago | parent

It's really not neck and neck with Opus 5.5.

eru 49 minutes ago | parent

Really depends on your use case.

You are right that there's plenty of use cases where Opus 5.5 is better.

gvfsa 35 minutes ago | parent

If it depends on your use case then it’s not neck and neck…

lejalv 15 minutes ago | parent

Cost-adjusted,it might be ahead.

It's key to name the boundary conditions. Unlimited cost is a very HN assumption.

TomGarden 14 minutes ago | parent

Max Subscription seems to be the HN assumption most of the time

sudo_cowsay 1 hour ago | parent

i thought open models weren't SOTA. therefore not neck and neck... in your experience, do you think it depends on your type of work?

ra 1 hour ago | parent

They're about half a gen behind. Very impressive to get this kind of utility from a flash series model... the impact of RL seems to be behind much of the coding and agentic improvements lately.

blntechie 54 minutes ago | parent

In my experience, for coding, they are pretty much SOTA. Maybe not for other sciences and problem solving.

rjh29 1 hour ago | parent

It's absolutely not SOTA for vibe coding a whole app, but for guided work it is great.

throwaw12 39 minutes ago | parent

which model is SOTA for vibe coding a whole app? I always use models+agents in guided setup

the moment there is a model which can code whole app, we can fairly say software engineering is done.

platinumrad 36 minutes ago | parent

Is anyone actually oneshotting a whole app or even vibecoding one in multiple steps without it being "guided work"? I really wish announcements (like DeepSeek's recent one) would stop focusing on the ability of the model to spit out Minecraft clones on demand.

antoniojtorres 34 minutes ago | parent

Some people are trying to one shot entire companies

bahmboo 16 minutes ago | parent

And failing.

MrScruff 32 minutes ago | parent

OP didn’t mention one-shotting.

There are plenty of use cases for the ability to quickly through together and refine an application without reviewing the code though.

gcanyon 24 minutes ago | parent

I have one-shotted the two most recent relatively complex apps I've tried:

1. a puzzle game with complex interplay, pieces to place, design vs. play mode, storing multiple solutions with the puzzle, and more. I've since iterated something like 200 times as I worked out the design etc., but the first iteration was 99% correct and what I asked for

2. a learn-to-read application with built-in stories (that the LLM wrote) the ability to import epub, text, and HTML, that uses the built-in speech libraries as a default, but also supports an open-source TTS library that I gave in the first prompt, tracks progress and problem workds, has multiple games to reinforce learning, and has a parental mode for tracking progress and configuring the tool overall.

Frankly I was stunned that Claude one-shotted that second app, there was a lot in the prompt, and it did it all. I fully expected to have to iterate on the first draft multiple times. But it worked out of the box.

altmanaltman 1 hour ago | parent

It literally is not though... even DeepSeek doesn't claim that. It's just super cost-effective

vrganj 1 hour ago | parent

Or at a really good time, depending on who you ask.

Not everyone wants Anthropic to succeed.

jimbob45 56 minutes ago | parent

If Anthropic or OpenAI fails, the other wins the US market, no? I’m oversimplifying but…that’s a pretty realistic possibility in the face of people thinking sinking one US AI company would hurt the greater US venture into AI.

vrganj 54 minutes ago | parent

What if both fail?

tourmalinetaco 47 minutes ago | parent

If they did then Google would take over, and others would form from their competent engineers. Also, they don’t need to be profitable to succeed, they merely need to hype investors enough to ensure that their losses matter less than their growth projections.

vrganj 36 minutes ago | parent

What if all of them fail because the capitalist model of centralization followed by rent-seeking gets foiled by an open alternative?

That is explicitly the Chinese strategy.

Now, what if the largest capitalist economy went all-in on an AI that might no longer be sufficiently monetizable?

porridgeraisin 24 minutes ago | parent

> explicitly the Chinese strategy

Oh come on, this post sounds like one of those twitter bots. No one seriously expected them to just give away billions in R&D for free, and they arent. The chinese model companies are actively moving away from the open source setup. Even state export controls.

Hamuko 54 minutes ago | parent

No? Why would either of one of those have to win? Even if you limit yourself to US companies only, Google and Meta still exist.

cheema33 47 minutes ago | parent

> If Anthropic or OpenAI fails, the other wins the US market, no?

I am of the opinion that they both either succeed or they both fail. They are both losing money hand over fist and their model quality is pretty close to each other. There is not a whole lot that sets them apart.

nkmnz 42 minutes ago | parent

I agree on the analysis, but not on the conclusion. What makes you think that the conditions you describe are stable grounds for a duopoly where both could be successful at the same time?

dudisubekti 49 minutes ago | parent

> bad time

Regardless of the performance metrics... why bad time?

Deepseek is more like a pet project with huge state backing, and they don't seem to compete with Anthropic in terms of profitability.

meerita 26 minutes ago | parent

Models are comming every 4-6 months. Anthropic, OpenAI, all of them know this.

brianjking 19 minutes ago | parent

4-6 months? Try between 8-18 days for a foundation lab release.

Havoc 1 hour ago | parent

As reference point exa charges 4 bucks

thombles 1 hour ago | parent

For comparison that’s $5 with Brave Search API and $12 with Kagi API

ece 33 minutes ago | parent

$4 with Exa, $1.50 with SERP API

frenchtoast8 1 hour ago | parent

For comparison, Kagi charges $12 per 1000 searches (only talking about their API pricing, like what you’d use with an LLM; their $10/mo plan has unlimited searches but it’s a TOS violation to use those credentials with an LLM).

Havoc 47 minutes ago | parent

>it’s a TOS violation

Competitor uruky is more relaxed about api and is cheaper. Creator is active on hn

https://news.ycombinator.com/user?id=BrunoBernardino

suncemoje 51 minutes ago | parent

About a year ago I built my own search and fetch MCP tools for agents based on third-party scraping providers. It is about 15-20x more cost-effective than what OpenAI and Anthropic is offering with their integrated web search and fetch tools. Very impressive that DeepSeek has such a competitive offering today.

shevy-java 39 minutes ago | parent

These prices will skyrocket eventually. AI companies need more money.

RAM prices will also probably stay very high as long as AI companis act in such a parasitic manner.

gvfsa 34 minutes ago | parent

Can someone explain why fetch costs money. I thought it was your locally-running harness doing it. How can they charge you for that?

BlackRabbit1 31 minutes ago | parent

Most of your local requests will fail (with Python/Curl/whatever) if you are not using a remote-controlled Firefox/Chrome. And even then you will be bitten brutally by captchas, rendering them basically unusable for mass requests. So it's better to outsource your web search to an external service with a stable API.

The existing search machines that offer free APIs for local harnesses are super bad with low quality.

This is worth the invest as LLMs can be guided closer to existing information/code to adapt it.

stavros 24 minutes ago | parent

I was really excited to use this (well, I still am), but then I realized Ollama Cloud offer this too, though they don't really mention pricing. I think it's somehow rolled into their subscription.

larme 23 minutes ago | parent

I wonder if this is censored. If not then users in China mainland can use the api to workaround gfw (though there are much better ways)

Jskewel 6 minutes ago | parent

Is there any way to test this without an API key?

I am curious if my large website is in their search results. My company charges $1 per page crawled of our website, and the only traffic that worms through our firewall is using hacked residential ISPs. I really need to know who to send the (large!) bill to.