24 points ldng 2 hours ago 47 comments
As the saying goes, it happened before and will happen again : it now the turn of DuckDuckGo to enshitify my searches by bubblification and ignoring quotes.
Also, not interested in IA chatbot, I prefer multiple sources and to cross-check the information myself, thank you.
What alternatives are you using / recommending ?
spottedmarley 1 hour ago | parent
p.s ecosia does have their annoying obligatory ai chat bot that attempts to answer things which i could do without but i just mostly ignore it
cyanregiment 1 hour ago | parent
I once tried to index the web with local Mistral 7b. Here’s what I did:
1. Copy/pasted an English dictionary and removed the definitions, so I just had an Array of about 500k words.
2. Wrote a simple loop to iterate over the words, with the instruction to Mistral to try to guess 10 URLs related to that word. I would typically get 2-3 sites per word.
3. When visiting a site, also check that site for links to other URLs, and summarize it with hash tags, visiting the new sites as well and recursively doing that.
4. Keep a tally of how many sites are being linked to for the page’s “rank”.
In the end you should have a massive list of objects with valid URLs ranked by how popular they are (to each other, by link), and a set of searchable hash tags for each.
I ran this for days and I never got past B, and almost every site was adult content. Most of the sites came from other sites. But it does work! Janky page crawling and ranking.
Anyway more people should take on web search, I agree. LLMs offer shortcuts.
benj111 51 minutes ago | parent
As you note, you got into an adult content 'bubble'. Going to random sites doesn't mean that site is good in anyway. Wikipedia, or some other proxy for quality at least gives you a good baseline.
If you're aiming to index all of the web, you should have the same endpoint, but in the meantime my way, you are indexing 'good' sites
cyanregiment 35 minutes ago | parent
With either approach, we would both eventually end up with all the websites - that being a surprisingly small number (~1 billion of which we only ever access ~1000).
I got in an adult content bubble from starting with "A" ("Adult") and would have hit another one presumably at "X". But "B" had quite a few too, lol.
Most of the words were bubbles - where it spent a decent amount of time on each one and I wondered if it would ever move on or was stuck. There are just a lot of backlinks and tangents for every word you can think of.
Another fun thing about this experiment is you will find a lot more international websites that no other search engine will find - some weird personal sites and some surprisingly good forums.
But yeah, you could enter anywhere - Wikipedia, or some other random Array of words. It doesn't have to be a dictionary.
esseph 19 minutes ago | parent
worldsavior 1 hour ago | parent
philipov 1 hour ago | parent
sakarrakas 1 hour ago | parent
ldng 48 minutes ago | parent
bejd 36 minutes ago | parent
[0] https://web.archive.org/web/20140720050529/http://dontbubble...
eth0up 1 hour ago | parent
eth0up 40 minutes ago | parent
But I am not sure the comment is being downvoted for irrelevance or any other rational purpose.And ddg is not going to disable the IP bubbling for the lite version
Furthermore, I think we need a modified Godwin's law, to describe the inevitability of any discussion on HN regarding search engines to eventually turn into an ad for Kagi. Not saying it's a bad thing, but I think most people are not ready for a paid search engine.
But at least no one is saying "just use AI". That would be bad.
kstrauser 1 hour ago | parent
Arubis 1 hour ago | parent
colinhb 1 hour ago | parent
bayindirh 1 hour ago | parent
dinkblam 58 minutes ago | parent
drcongo 57 minutes ago | parent
mrtomservo 54 minutes ago | parent
Using a browser that defaults to another search engine is jarring. I see screenshots of what Google search results look like now, and it's like seeing the neighborhood you grew up in with shuttered storefronts and mouldering houses.
teroshan 51 minutes ago | parent
- Ability to block/lower/raise/pin domains. Some examples of commonly tuned domains are here https://kagi.com/stats?stat=insights
- Shared custom bangs across my sessions. I can add work-specific bangs (e.g. !bug -> Jira search) and have them available across my devices.
- Regexp redirects, to automatically redirect online documentation results of some libraries to the specific out-of-date library we use in a legacy project.
All of these are not as important as knowing there is no incentive for the search engine to enshittify the results to bump some sponsored product in the results, to raise ad-heavy websites to the front to get back some precious ad revenue due to a conflict of interests, and to lower the quality of results in the hopes of making me search a second time so I am served a second batch of sponsored links.
chinggis 42 minutes ago | parent
abtinf 41 minutes ago | parent
Some things that I wish were different about Kagi:
I wish Kagi Ultimate LLM credits could be shared with the whole family, rather than each person having to sign up for an ultimate level account. That costs an additional $162/year per person, which makes no sense for my younger kids (even though their Research Agent and image generation would be great for my kids). This is actually the reason I’ve been hesitant to setup a family plan at all.
I wish I could use Kagi LLM routing as a generic LLM provider outside of Kagi Assistant. (I don’t really care that they charge ~4x the markup of openrouter; I just want one place to pay the bill and that it includes search is nice).
Alternatively, I wish Kagi would support BYOK.
I wish Kagi API search would simply draw down from the same bucket as LLM credits. I really don’t understand why they are splitting it up like this. I just want to give my local LLM harness a search tool hooked up to my $270/year account.
I wish clicking on the top left Kagi logo on Kagi News would just go back to the Kagi home page.
I wish Gmail hadn’t routed Kagi emails to junk, because I missed the Kagi early adopter free tshirt.
I wish Kagi would implement an Apple TV video app that I could use in place of the abomination that is YouTube.
I wish there were guides in Kagi help or the feature pages themselves that included lots of community examples of how to actually use Kagi features. Like I want the Kagi Redirect Rules for more than just routing Reddit to old Reddit.
There is a lot more. But I think this list is a ringing endorsement of Kagi. Objections betray interest. I use it constantly and can’t wait for it to get better.
mtlynch 33 minutes ago | parent
I'm always surprised when I see people say, "I can't imagine paying for a search engine."
Imagine if you lived in a city with horrible air pollution, and you were coughing all the time and coming down with mysterious respiratory illnesses. And then you go to a friend's house in the same city who's happy and healthy, and it turns out they have an air filter they pay $10/mo for. And your reaction is, "I can't imagine paying $10/mo just to breathe air."
Pay for search! Get out of the ad-optimized Google black hole. This is a tool you use hundreds of times per month and it fundamentally influences the way you experience the web. It's something that should be a no-brainer to pay for even if it was only marginally better than Google, but I've found it significantly better than Google in most dimensions.
whitefang 1 hour ago | parent
santiagobasulto 1 hour ago | parent
It will definitively be a problem in the long term for monetization of the authors. But for me as a user, it's the best.
dfxm12 52 minutes ago | parent
slig 56 minutes ago | parent
Feels like Google Search from 2008.
cachius 49 minutes ago | parent
grepex 42 minutes ago | parent
Nicholas_C 30 minutes ago | parent
dmazin 56 minutes ago | parent
dml2135 54 minutes ago | parent
traceroute66 51 minutes ago | parent
This could easily yield very interesting (i.e. "not just another bing proxy") results once it starts gaining traction.
I use DuckDuckGo at the moment, only because Startpage started making themselves impossible to use when they started aggressively blocking access via VPNs. As other people have already said,see `noai.duckduckgo.com` and `lite.duckduckgo.com` which might address your specific concerns.
If you are a Mullvad user, there is also Mullvad Leta[2], which has been security audited[3].
I might shift more towards Ecosia/Qwant.
[1] https://blog.ecosia.org/eusp-germany-rollout/ [2] https://mullvad.net/en/blog/introducing-mullvad-leta-a-searc... [3] https://mullvad.net/en/blog/2023/5/16/security-audit-of-our-...
ldng 38 minutes ago | parent
bronlund 45 minutes ago | parent
SyneRyder 45 minutes ago | parent
If you just want a free search engine, Brave is probably the best index. Bonus points to Marginalia for finding the obscure things Brave doesn't surface.
But if you want to go down the rabbit hole, it's easy to build your own metasearch engine and search several engines at once - an AI model can make one for you in a single night with just a small PHP script, if you have your own website / webost. In my case, the combination of Brave, Mojeek & Marginalia covers all my search needs.
Deeper down the rabbit hole, you can build your own search index with SQLite, as an additional index for your metasearch. There are projects like Hister that will help make your own search index:
axegon_ 43 minutes ago | parent
As for Kagi - they've earned a life-long ban from me for political reasons.
tunapizza 43 minutes ago | parent
mlok 38 minutes ago | parent
supersparrow 37 minutes ago | parent
erelong 32 minutes ago | parent
other things that came up in a search not mentioned in post yet (that I have heard of before):
mojeek or yacy - are these any good?
list of searx instances (I've used some of these a while ago but not recently so idk of their quality currently): https://searx.space/
dartharva 24 minutes ago | parent
gtfiorentino 8 minutes ago | parent