I think 'more customization' which is a theme with a lot of these alternatives is a fundamental dead end. I don't know where this persistent myth comes from that people love choice and tinkering, because they clearly don't. There's a huge cost associated with having to make choices, and one feature of successful modern apps is that they're frictionless. That's why TikTok is so successful. There's no login, no user chosen social graph, everything's abstracted away.
And that's by the way why Google is still successful as well. Because it literally still is a simple box where you put a question in and it gives you answers without needing to do anything else. The only way to beat that is to make it even better while not making it more complicated which is very hard to do.
> a simple box where you put a question in and it gives you answers
Google is very good at this, but this is exactly what I _don't_ want in the "Next Google". I want a _search engine_ for the web, not an answers engine that tries to know what I want better than I do.
Search the web. Give me links to websites. This seems obvious to me, but everyone is trying to be like Google.
I'm convinced the death of the web and independent forum communities is largely the fault of Google's lousy search results no longer actually returning real web results.
The death of the web is because most people don't want what you want. They don't mind walled gardens, so long as they are easy to use and have the content and connections that they want to see.
The audience of HN is extremely skewed towards preferring systems that allow tinkering but that's not what the market wants.
This. People don't realize that the early web was elitist. Now, the entire population is online. And, as you said, most people simply don't care about the stuff we care about.
That's also why "Google's search results are soo bad." They're not. For the bulk of Google's visitors, they're good enough.
But to be honest the internet is still the internet. The web still exists. Any lamentation at the loss of the "old" internet is that you don't have more angry uncles spewing political rhetoric on your motorcycle forum.
Do you want the unwashed masses in your specialist forums? I certainly don't. Seems to me things are working pretty well. I do worry about the next generation but to be honest, the same was said about me.
Instead of complaining about the state of the web just be the change. Host your site, run your forums, live your life. Stop worrying about how other people should live their lives. Take some youngsters under your wing and show them how a keyboard works instead of a touch screen.
FYI: Wiki says: <<Eternal September or the September that never ended is Usenet slang for a period beginning around 1993[2] when Internet service providers began offering Usenet access to many new users. The flood of new users overwhelmed the existing culture for online forums and the ability to enforce existing norms. AOL followed with their Usenet gateway service in March 1994, leading to a constant stream of new users. Hence, from the early Usenet point of view, the influx of new users in September 1993 never ended.>>
> Take some youngsters under your wing and show them how a keyboard works instead of a touch screen.
This x1000.
The only way to ensure that future people can continue to maintain things is to teach those who are interested.
While I think we need to scale back the opaqueness of tech to the average user[0], there will always be some who just know more.
[0] By which I don't mean "more configuration options", because the average user doesn't want those.
I mean make it easy to access the source - and just as importantly make it easy to use changes to said source, so those who think, hm, I want this, can learn to fix it themselves, or find someone who can.
> Do you want the unwashed masses in your specialist forums?
Having the choice between "unwashed masses" and "curated to suit political agenda" I would firmly choose the former.
Before I get attacked on political agenda angle, please understand that it takes trained professional actively trying to remain objective to have impartial moderation. Many professional discussion moderators fail here and you just cannot expect neither unpaid volunteers (doing things out of passion which is not impartial in the first place) nor paid employees enforcing company policy (again very much partial) to remain objective.
The goal of moderation should be to allow for multiple angles of thought to emerge and not get drowned. Give me the tools to see those non-mainstream positions.
The unwashed masses are often filthy and malicious (on the internet).
Moderation, of the governance and oversight meaning, is unforunately very necessary to keeping any pool from becoming a cesspit. By some combination of technical, volunteer, and participant effort (membership fee, or service requirement), areas can be kept decent.
Without this, decent and would-be participants are run off by spam, scams, trolls and harrassment, political rants, religious rants, etc.
Hence what happened to Usenet. I was active on comp.lang.c comp.lang.c.moderated comp.sys.3b1 comp.sys.3b2 back in the mid 1990s, by 1998, it had become a total hot mess. The moderated groups imploded, or shutdown, when the moderators just gave up. In particular, comp.lang.c.moderated was a fountain of knowledge on C language implementation specifics, then a few really pedantic asshats joined, and their MO was purely slandering and shouting. They had agendas, and they had no interest in sharing knowledge. I think a couple of them thought they could affect the evolution of the language, from outside ANSI, to suit their own personal opinions. Nasty little dictators. I completely dropped off usenet around 1998. In retrospect, I hung on for to long, hoping for a return to rational behavior.
Wonderfully put. I would add that we need to see things for their scale and proportion. Yes, the simple walled web is bigger than ever. So is the hacker community, and this website (so far as I can tell), Reddit (for all its faults), etc.
Admninistration web sites are now forcing you to use google(blink/geeko) or apple(webkit) based browsers.
Asking nicely to restore interoperability with noscript/basic (x)html browsers does not seem to work, namely keeping the door open to alternatives requiring a reasonable effort of development, and not the army of devs of big tech.
> For the bulk of Google's visitors, they're good enough
I'm not sure that's true. Just to pick an example, basically any type of product search now leads to auto-generated spam/borderline-spam websites that scrape reviews from Amazon, which are themselves often fraudulent.
I guess you could say that's "good enough" but only in the sense that someone being scammed at some point consents to something that makes them a victim of the scam.
For many searches, you'll see "reddit" autocompleted to the end of the search because Google's organic results are not good enough and people are trying to weed out the spam somehow, albeit in an imperfect way that restricts results to one site.
I've been using Kagi search and I find the same stuff there. I can block them from my personal results as I encounter the garbage, but it's there by default.
And the people trying to get their garbage spam sites to rank high are wise to the common tricks people use to attempt to get better results. Just today I accidentally clicked on a result that from the title looked like it would lead me to reddit. I should have paid attention to the URL because instead I was sent through a bunch of redirects to some terrible spam site.
DuckDuckGo is horrendous for spam and malware distribution. I have stopped recommending it to non-technical people because they can't recognize the obviously evil/bad-pattern sites that are on the first page of many DDG searches I do
User manuals for stereo systems, specs for old hardware. Anything that can be generically referenced, maybe a bit harder to find. Someone can throw up a webpage that SEO's on "manual" or "user guide" and when I type in "Sony DTH-345 user guide" I'll see results like "manualsonline.xyz" that are clearly bogus.
I can't lab it up now, but that is the gist. And DDG has it while Google rarely has the same level of it.
That's my understanding. And while I appreciate that Bing is the "baseline", I would be interested in what it would take for them to do additional cleanup on results. Blacklisting domains should be a no-brainer. "Safe search" for suspicious sites, not adult theme.
Part of it is, that everyone already has heard about Google. Trying to get another search engine that people trust and have heard about is hard, not to mention it has to work decently.
Worse than that are the SEO results that don't give you what you want at all.
For products, one of the worst offenders is Target. Yes, Target. They show up near the top of about 60% of my searches, including for things they simply do not carry. But hey, they carry a similar product that you might be interested in! (not)
SEO results make Google money, but they also make the results suck.
Thats the advantage Facebook has, people trust each other if they know them or a recommendation comes from a friend of a friend, but you cant really see who is recommending something on Amazon, Reddit and other sites.
What exactly is Google supposed to do here? They're returning what's available. Beyond that it goes from search engine to active curation and recommendation - and they are actively moving in that direction by replacing results with "answers", which I find to be much worse.
Have you read any of Google's advice to site owners? It amounts to "provide your users with what they're looking for, and do it with a fast and easy to use site". If you think SEO is an easy hack to get any site to the position 1 for any search, you're just ignorant of the landscape. All the "SEO" crap you see about keyword density or whatever, is just an industry trying to sound different and like they've got the key to position 1.
> They could start penalizing sites that show different things to googlebot and a real browser again.
Google is ridiculously good at this. Would be keen to see examples of a site that does this in a meaningfully negative way that ranks well for searches that matter.
I don't fully agree with all of your other points but I can see why you'd mention them.
Nothing, really. It's the "what's available" that's the problem. Quality paid services can't gain market share because free is a anticompetitive. Consumers will almost always choose it if it's not complete garbage. This throws all the metrics off for search because quality paid services become a statistical anomaly.
Desiring choice isn’t elitist. Early web adopters were passionate and willing to put in more work. Nothing wrong with that, and nothing wrong with liking simple default settings either.
Wondering if, by “elitist”, GP meant more like “out of reach to many laypeople because of learning curve.” Is there a good single word for that? “Difficult” and “complex” aren’t quite right.
Anyway - you’re right, nothing at all wrong with wanting choice. I think the point being made here though is that “layperson gravity” / mass market appeals / lowest common denominator is going to mean that tuneable web search will be a niche product, forever. Even if we’d both like that niche.
Not trying to be awkward, but even back in the early days of Netscape, it wasn't like you had to recall any arcane commands to use most search engines.
Sure, they all had limitations then, such as not necessarily knowing similar word senses to create more nuanced searches, but that was a fairly level playing field.
Is it simply that it was initially a bit obscure and not everyone had found out about it? That's not really a barrier in terms of difficulty - as soon as people got into the "in crowd" they could use it just like the rest of them.
Cost was one. Hardware wasn't cheap and it would seem to have no practical use except to satisfy one's curiosity. A PC for the kids cost about half of our family's monthly gross income. A modem would be 10%. And the phone bill caused tourette's like symptoms more than once.
there are so many options, many complex, but many too are simple. One of my personal favorites is using github pages, you can host a plaintext html document for free in about 10 minutes (including downloading a text editor and github desktop)
When I started all modem calls required remembering 'atdt'
When netscape came out if you were lucky to have ppp access setting it up on windows 3.1 was difficult but once you were up as long as no one picked up the phone you were fine.
But even this was too difficult for the average person who lived in an aol world where the internet was limited to aol.
Anyone I watch older than 50 years old is constantly frustrated by google, they just don't know what's good, what's not. I think they use it because it's the "norm" and that is the only reason.
And as far as everyone else.. I have no idea to be quite honest. I have a fairly big handful of non-techie friends who think google sucks but is good enough so there is no real incentive to change.
With any habit people don't change it unless they have a huge reason to. No one stops eating double cheeseburgers until their doctor literally tells them they are going to die. I think that's what we're up against here.
Hey, now wait a second there, you young whipper-snapper. I'm sitting here reading HN while I install Debian on an XCP-ng VM, and I hear I can't understand whether Google results are good or not? There are plenty of us over 50 that actually lived the early internet and watched it first-hand transform into what it is today.
This is a great reply. Real question: Why hasn't the same happened to HN? To be fair, there are some arguments that play out over and over again: SF crime & homelessness, Bay Area housing, US taxes, central banks "printing money", office vs work-from-home, feeling forced to adopt liberal social views / stances at a tech company (Black Lives Matter, LGBTIQ+), etc.
And somehow, there is still enough good, new content to keep the conversations fresh. Some of the best are when someone shares a personal story, then there are tens or hundreds of follow-up posts -- generally supportive or inquisitive.
> Anyone I watch older than 50 years old is constantly frustrated by google, they just don't know what's good, what's not.
Yes, and now imagine these same people, most of whom never bothered with basic google search options (eg. excluding entire domains, mandatory keywords, ...) presented with a search engine with a plethora of knobs and levers to tune it to their liking.
It isn't elitism. It's competence. A system designed by competent users for competent users are going to be different from one made for general masses. There's nothing elitist about that.
> This. People don't realize that the early web was elitist. Now, the entire population is online.
That's an excellent demographic point.
A counter-argument is that the move from port 80 to port 443 is the elitist drift. Instead of innocently putting information out there and sharing in an egalitarian fashion, we now have certificates and gatekeepers and lions and tigers and bears, oh my!
In an age of free SSL/TLS certificates, and built-in support in pretty much anything, is this really true? Making your content available via port 443 really isn't impeding anything at all.
not us yahoo / Altavista / or whatever else was around at the time users. Google was so clearly better right out of the gate that I distinctly remember having conversations conveying the shock and awe at the quality of the results. Before Google everything was mediocre at best.
I’d say the for the bulk of Google’s visitors the results aren’t merely good enough, they’re insanely great.
Now, that said, the G monopoly is horrible and needs to die.
>Er, isn’t the whole point of Google tracking you and knowing your mothers blood type so they can give you better search results tailored to you ?
AFAICT, quality personalized search results isn't the goal for Google here.
I think the point of all that is generating revenue via advertising sales.
And while providing high quality search results might once upon a time have been a goal, both as a goal in itself and a tool to drive user adoption/engagement, that's no longer necessary as they have a (relatively) captive audience and a (relatively) captive customer base (advertisers). As such, quality search results are no longer all that important.
I think this is exactly why Google's search has become trash. They don't need it. Even when their goals changed from "making searching the internet better" to "Making money and collecting data on everyone" they still depended on search to see what people were doing both online and offline.
Now they have millions of cell phones collecting data and the GPS location of everyone offline, they run extremely popular DNS servers to see what websites people go to, most websites (including educational and government websites) include google's trackers so on most websites every page loaded will ping at least one of google servers. They've got people uploading their personal and work documents to their cloud. They are swimming in data collected from sources outside of web search. Between that and the lack of actual competition it's no surprise they aren't investing in making searching the internet better for people. In some ways they profit from search results being terrible. If anything beyond the first few results on the first page is filled with spam and irrelevant websites the top few results to any search become even more valuable.
so they can give you better search results tailored to you
That depends on whether or not you agree with Google's definition of "better". What is better for Google is a set of results that make you most likely to click on an advert. Failing that, they want you to click a link, see the site, and either click an ad there or quickly return to Google where you'll click on an ad. The worst outcome for Google is that you'll click a non-paid search result, like it, and stay on that website.
This is the fundamental problem with search engines - if they work well and give you exactly what you're searching for first time then they won't make any money. A lot of what Google does is subtly trying to give you results that look great but really aren't.
What the market has produced is not ipso facto what the people want. The market is simultaneously optimizing many things. Walled gardens are much better explained by companies benefitting from not having to allow their competitors access to their customers than by "being what the people want".
People don't mind walled gardens because for a while they've been good enough if not better than the previous status-quo. However, those walled gardens are decaying such that there might actually be demand for something better if it existed.
> However, those walled gardens are decaying such that there might actually be demand for something better if it existed.
Some walled gardens, like facebook, are decaying. Others, like tiktok, are vibrant, still fresh and new.
Walled gardens are a natural result of the pursuit of capital. If you burn VC money, but create no moat (it turns out the walls of a walled garden are also a moat, weird huh), then as soon as you introduce a clever algorithm that introduces 50% more ads to extract profit, your users will all leave.
As such, a business is incentivized to build these walls and moats.
How do we avoid this?
Well, we do have examples. Mastodon and other open source projects eschew walled gardens in favor of free software ideals. Web3 embraces a certain "decentralized" vibe which lends itself in this direction. Universities, and other public-ish institutes like DARPA, created the original internet and many of its technologies.
Unfortunately, open source projects will struggle to advertise or find users. They do not have the initial capital to get as much momentum as the VC-funded alternatives. Web3 seems surely doomed to end up also building walled gardens for the crypto-anarcho-vibe is only skin-deep, and many a regular business is now highly involved.
This leaves government entities. The government is the one group that is both well funded enough, and has motive to create protocols which prioritize the user's freedom over profit (after all, the users will pay taxes regardless of how high the walls are).
It seems to be out of fashion these days for the government to actually do anything though, so perhaps there is no more chance of that than of a free software project managing the same.
Yep. Imagining nontechnical users would endure paths of greater resistance to avoid shady privacy and openness practices is magical thinking. Privacy and openness are important, but having tools designed to maximally reduce the overhead of solving your problems is more important.
I know I get on people's nerves on HN by harping on user needs and FOSS usability and interface design and such, but I think we as a group need to start taking user needs much more seriously. Open-source alternatives will always be alternatives until without taking usability and the overall experience seriously.
Many, if not most FOSS software developers choose some commercial tools— count how many MacOS and Windows laptops you see at OSCON or FOSDEM. Consider how different the cost/benefit analysis would be for people without the most basic requisite knowledge to reason about software problems, let alone troubleshoot, or throw in a PR to address them directly. Commercial companies don't get these seamless experiences by magic— it involves research, design, development and testing to deliberately remove friction and stumbling blocks rather than assuming your use case is universal, or that docs are a suitable replacement for fixing usability problems. There's nothing stopping any open source project from doing any of it. Volunteering isn't unique to coding— people volunteer to cook and build houses and clean up trash, too. But developers run FOSS and developers need to deliberately incorporate those other perspectives into their projects to get the benefit.
I think you’re conflating “tinkering” with simply desiring a different feature set. In my case I actually want “less” from Google in terms of number of features. I don’t want to tinker either, but we naturally reach for toggles as a way to tell the system we want different (not necessarily more) features.
Most of my search results at this point look like a spam ridden inbox from the mid-2000s.
This is absolutely true; however there are riches in niches.
The % of internet users who want X probably is lower, but I’d gander the absolute number of people who would want something like this is much bigger than it was a decade ago.
This community forgets that because we’re told by investors, the media we read, etc. to go after the biggest markets possible. “Hunt for elephants, not field mice,” they say.
Don’t forget that a niche market on the web can still be massive and small fortunes can be made building products and services for them. You might not even need to have investors on your cap table to bring products to market to sell to those niches if you know what you’re doing, but shhhh… don’t tell them I told you that.
I think you’ve touched on an interesting point here with respect to connection. Content naturally lives inside a walled garden and is often created there whether by users or professional studios. Users expect and dont care about this. However _connection_ is naturally between things and is a source of user friction when things dont connect well together.
Imagine a commenter in a walled garden complaining about the walled garden's audience. "Facebook's audience is extremely skewed toward..., but that's not what...." How long before some Facebook reader asks, "Then why are you using it?"
If one dislikes systems that allow tinkering then why read and comment on HN. I do not understand.
Apparently "the market" dislikes systems that allow tinkering. Presumably "the market" refers to some people who said they dislike systems that allow tinkering. Either that or the commenter is inferring "the market" dislikes systems that allow tinkering even though no person has actually made a statement to that effect.
Maybe HN is not the ideal forum for discussions of "the market" (due to lack of interest and/or understanding), but maybe HN is a decent forum for discussing systems that allow tinkering.
The best example of this is having a Google Home device and just being like, "Hey Google, how many ounces are in a cup?" as you're standing in the kitchen with your hands covered in flour.
There are no more websites worth linking to anymore... If you filter out all SEO spam there is barely few webpages left...
Google is desperately trying to hide that fact. Most of the web 1.0 can nowadays fit into small town telephone directory... You do not need mulitibilion dolar business to run web directory...
Try search.marginalia.nu (especially https://search.marginalia.nu/explore/random but try some searches for git commands or history too, just remember it is a search engine, not a conversation partner so only include words that should be in the article you search for) and come back to me afterwards.
I thought like you that if even Google couldn't find anything it was not there, but after discovering marginalia I now know it is just Google that has become unusably bad.
For day to day searching I now use Kagi and for me it is easily worth 10 or maybe 20 dollars a month since it "just works" unlike Google and has a larger index than Marginalia.
For now though Marginalia gets the money since Kagi is still in beta.
Just searched for "linux users" in marginalia and google. Google's first answer seemed spot on. (users command usage); Marginalia provided me with in comparison _marginal_ results.
Maybe it is because google knows me better then i am aware of. I really don't notice google results getting worse, while i read so several times in HN comments...
Yeah it's not a search engine for answering questions, but for finding documents. You'll get along with it better if you see it as something like grep for the web. This is something I'm very intentionally trying to accomplish, as it's something I feel Google has gotten worse at.
This is just retrieving articles when I search, not actual websites. Interesting if you're looking for article related to search keywords I guess, but genuinely unusable for actually finding something specific.
Gave it another try with "ssh scp". Google's first result explains me how to use scp (ssh provides a hint about the context), which was what i would be looking for. Marganilla... not so much it seems
The second hit links to a man page of scp. It is a formal description of the syntax, not what I was looking for... I'd rather have a short intro and a few examples of typical usage instead, am I being pedantic? If i wanted a formal description i would google 'scp cli', 'scp options' or even 'man scp'. Also, on a tangent, to be honest -- I find searching simply for 'scp' not a very clever approach. How did Marginalia guess it would be the cli tool and not one of the other acronyms?
Yeah that's not the sort of search engine this is. If you search for SCP, it will show you documents where that term is relevant, using domain ranking as a tiebreaker. It's quite intentionally not trying to read your mind.
I think it showed the man page first because that domain is highly ranked.
Google still searches forums pretty decently - I think what you are describing are two separate phenomena
1. Yes Google search has gone to shit - even putting stuff in quotes now does not do an exact search (there is another checkbox you ALSO need to use for that). It tries to be too smart even when no user is logged in.
2. Giant mega forums like Reddit have really taken over. Instead of a dedicated forum people just go to subreddits. Personally I think it is good and bad and I still try to actively participate in both.
> I'm convinced the death of the web and independent forum communities is largely the fault of Google's lousy search results no longer actually returning real web results.
I think you have the cause and effect reversed. (Though I disagree with the meme that Google's results have gotten worse. For me, it's more useful than it's ever been. I often find the information I need without even needing to click on a result.)
> this is exactly what I _don't_ want in the "Next Google".
You don't want that, I don't want that, and I am sure many people here on HN and similar gathering places for powerusers agree with us.
But "the next google" doesn't have to replace google for us, it would have to replace google for the average consumer (AC). And the AC likes the "simple box". The AC doesn't want to fiddle with customization options. The AC is used to walled gardens, advertising, "Apps" that are just wrappers around webpages.
So any "next google" will have to compete exactly at the "simple box" game to get the attention of the masses.
>I'm convinced the death of the web and independent forum communities is largely the fault of Google's lousy search results no longer actually returning real web results.
You are wrong; Open Web died or almost died because of walled gardens like Facebook, Instagram and Twitter. Billions of people who hang out on Facebook, Instagram and Twitter would hang out instead on Open Web e.g. websites, blogs, forums etc. if walled gardens didn't exist.
The "we answer your question" thing could be useful, but:
1. It should be in addition to (not instead of) the web search results.
2. The answers can't be wrong. Google often gives such a low quality answer that I no longer want to use it for this, besides asking about the weather and sunset/sunrise.
"Word for a bad scientist" gives me results as if I had searched for "Mad scientists fun stories", and it wasn't even offered as a correction. Google can't even accept that I'm typing the word I want anymore, I must surely want something else.
A lot of search results pointing to independent forums are about people asking the same question I have only to be told to use Google to find the answer.
All you have to do to improve on Google at this point is to do less, make it less bad, i.e. a process of removals, not additions. Just do the same thing, but without all the shitty extra stuff. But then what's the business model? (Since that is in fact most of the shitty stuff. Oh sure there's still the SEO spam, and you're in that arms-race, like it or not, even if you're not a successful search engine, so you do the best you can with that.)
Speaking of doing less, I would love to see the web be more hierarchical or semantic (but not necessarily "the semantic web" as it's currently conceived). Google itself is what made the world reorganize itself. A world where that kind of search exists will reorganize itself around search, maybe not always for the better.
Concrete example of going from a hierarchical/semantic world to a search-based one: Instead of finding your socks in the sock area, which is inside your clothes area, which is inside your "do private things" area, let's say now every sock has a trackable chip in it similar to an AirTag and you just say "Alexa where's the nearest pair of socks?"
Pros:
No effort spent on putting your socks in the sock place. Just throw them anywhere.
Instant access to socks.
Cons:
Big Tech, with all its limitations and machinations, now mediates and controls the relationship between you and your socks.
You succumbed to the temptation to slack off, and now there are socks everywhere. The Roomba doesn't even work right.
Those 4 points can be solved by ad blockers or other browser extensions. The real problem is that the results below the ads are also ads, seo spam and clickbaity stuff.
For Kagi, at least, there's a very well integrated search customization method that they didn't bother to show here. For any search result, you can add a ranking adjustment for the site it came from. This is directly in the results, so it's very accessible, and quite easy. One of the choices is 'pin', which is fantastic for technical work: 'sqlite.org' is now boosted over everything else, for me, and it's exactly what I want. I could just as easily take it out, if it becomes a problem.
>I think 'more customization' which is a theme with a lot of these alternatives is a fundamental dead end. I don't know where this persistent myth comes from that people love choice and tinkering, because they clearly don't.
I agree, BUT I would like to share my experience moving from Google to Kagi. Google serves me primarily ads, whether it labels them ads or pretends the results aren't ads, they're mostly ads. I see the same horrible domains pop up frequently, and there's nothing I can do about it. Kagi gives me the ability to remove these domains from my search, and it is INCREDIBLE. How many times has Pinterest barged into your search results?
Now, one could argue that these are domains which no one wants in their search feed. So perhaps this could be solved by being less beholden to shareholders and an advertising model. However, what about domains which most people want, but I don't? For example, I was finding increasingly polarising political content coming from MSNBC. I didn't like their "human" angle on news, and I didn't appreciate their extremely polarising commentary. So I want to remove all MSNBC results from my search feed. I understand I am probably in the minority. Perhaps my political views don't align with the typical customer. Could a well designed search provider anticipate this need of mine without giving me some kind of method to tell the provider that I don't want to see MSNBC? I'm not convinced so.
So I think the answer here is both. Design a search engine well, and give users the ability to customise it.
More likely, it exploits human's cognitive weaknesses successfully with a simple way.
It learns how how people get their dopamine dose. And there is no going back. You need more and more, more extreme content. More polarization. All you need to do is to open app and get that dose. Is it the same for search engine?
And people make more crazy stuff to get views. How this ends? Not well, probably.
I don’t think it is fair when we frame products as being objectively and consciously nefarious in this way. Conjuring images of executives rubbing their hands together, giddy with enjoyment that the war is leading to more exciting content.
These are firms that are meeting a legitimate need- and that’s the need to feel connected. Tik-tok provides that, and very effectively. Plenty of people get genuine enjoyment out of their product, and meaningful connections do happen thanks to Tik-tok
Maybe it is not fair, but that is what happens and eventually it is acknowledged by executives, which leads for new design decisions based on that on TikTok and other platforms to get more money and users.
I don't think they fulfill some gap of the need of feeling of connected in a real way. More like a bandage. We have seen the development of Facebook, Snapchat, Instagram. Their audience is fading on countries who have used them longer time, what went wrong? How is TikTok so good that they try to adapt it on their platforms as well? No way to connect?
Short video clips which might or might not lead for a real conversation. They might offer escape from reality in your lunch break at work.
I understand the perspective of "feeling connected". It brings people together with similar mindsets on entertaining way. Or at least people who seems to enjoy similar things.
On the contrary, is it different than some oldschool cults or religions? Cults which are using psychology writings as base for feeling mutual understanding of themselves. Or religions which share same ideologies and use it as a solution for their problems?
Technology is advancing, is TikTok a modern solution for finding your role and place in the world when it does not make sense and you feel you are alone with your thoughts? Maybe it is, maybe it then fills some gap.
I agree that TikTok is providing entertainment (well, that is what dopamine usually is). It is easier to hook people on short videos which are done by global audience versus Netflix where there is a limited amount of material and they cost a lot to make, when audience on TikTok is mostly making them free and you just pick suitable ones with your algorithms for showing the other audience.
However, there are many problems in this. How it can be abused and how it creates people living on their own bubble, like people on some extreme Facebook groups. When a narrative includes only content that boosts your own thoughts, a reality can be lost. We need some research on this matter, but for some reason social media companies are doing their best to prevent that.
Someone people also really get addicted on the entertainment and cannot stop using it. Well, same thing can apply also for alcohol, but is addiction risk closer to opiates for example?
I don't really believe that executives are thinking for the best of the people, so optimizing platforms to hook users is a quite dangerous play.
>It learns how how people get their dopamine dose. And there is no going back. You need more and more, more extreme content. More polarization.
What kind of TikToks are you watching? Tiktok sees I like watching Japanese videos that compare American and Japanese culture so it knows to show me more of that. Trying to say that showing me videos I like is them just giving me a done of dopamine is such a weird way to phrase it. Should they be instead just constantly show me videos they think I would dislike?
>You need more and more, more extreme content. More polarization.
This seems a little sensational. Many people (myself and most of my friends included) don't see _any_ "extreme" content or polarization. A quick scroll through my feed is largely nothing but magnet fishing, frog tracking, cats, DIY projects, and geologists talking about rocks. It's enjoyable and arguably a dopamine dose on-demand, but not necessarily a road down more "extreme content and polarization".
I agree that it is a little sensational, as it happens mostly when you like only limited groups of things. It gets harder for algorithms to polarize if you "mess up" with the algorithm and like many different kind of things.
To clarify, I don't mean with "extreme" necessarily a bad things, just content which gives you "extreme" emotions.
This is the thing. I'm the power user in my circle. People comes to me for suggestions about pretty much anything involving tech. Sometimes just because they see me with different stuff.
So if I'm not the average user I may look for other options, as other people like me may do. If I find such options reasonable for the average user, that will be my recommendation to them.
Tiktok optimizes for momentary engagement and fast-paced social content. It turns out that's a niche that people want filled, and the rapid bouncing of ideas between users has created some very fun content which sometimes escapes containment so I can watch it. But not everything is going for the same goal, and honestly I wouldn't be surprised if more than a few players could really get in on that space. For other use cases, customization may well win out over a totally frictionless experience. A search engine is a tool, and benefits from more options much more than Tiktok does.
Tech power users (and power users in general I would say) might have more money to throw at the problem. "Nicheness" isn't necessarily a bad thing if your niche is profitable. I heard somewhere that power-grid-scale transformers are have insanely long lead times so the industry most be pretty niche (when's the last time you needed one of those?) and yet I think we can all agree that the equipment is valuable and I bet those manufacturers are making bank.
>I don't know where this persistent myth comes from that people love choice and tinkering, because they clearly don't.
People want to accomplish stuff, this means they need tools to do stuff and if a tool can be customized to do the stuff faster or better people want the customization. At my job we have paying users that requests features that indeed are work related(not moving shit around). I know GNOME-minded people will disagree and they prefer to bend their work to fit a guru-s vision.,
Now my turn to ask, why do people like you think there is a generic and basic solution that works at the same time for the casual user and for the user that has a lot of tasks to accomplish? Is there some theorem that shows this, like "The GNOME theoreme of product design, keep removing features until the shit convergence to the local minimum where you find the minimum product and the minimum set of users possible.
Because most users would rather not have to customize anything. It's funny that your example is gnome, as most users are not tinkerers and would never use Linux on the desktop unless it were made so simple that (again) they would not have to customize anything.
Obviously there are exceptions, and some tools are so advanced that it's necessary to be able to customize them. No argument from me there. But for most tools, and most users, there's just no hunger for customization. Almost nobody wants to have to manage an array of options to do a web search.
I agree, there are users that don't use customization for X but they use it for Y, so you get the idiotic philosophy that removes customization from X and Y.
So you get GNOME fanboys that love that 10 features they personally don't use are removed but when the ones they use is removed their brain finally realize that not all people use the exact same options, the exact same workflows etc.
What is even more shitty is when soem feature is removed like the System Tray and first they pretend they do not understand what you mean when you say this feature is very important, then after someone wastes his time to explain a n=th time again how working people use the System Tray features at work or at home the GNOME dude finally admits the issue and offers some workarounds that are not equivalent but are "possible to do but with lot more work".
On short, some people do work on the computer, some people use search engines for work to find relevant stuff, this people do not ask features like "please use the exact same padding everywhere because I am OCD" or "please make those buttons/corners or edges smoother so I don't cut my tongue when I lick my screen", this people want the customization to do a task.
I am not sure why simple people have a SEGFAULT if they randomly end up in the Advanced section of a settings section, what do they expect when they open Advanced? Google main page has a small link for Advanced search see
https://www.google.com/advanced_search?hl=ro&authuser=0 how many GNOME users got hurt by this link existing?
The biggest problem with GNOME, and with Mozilla, and with almost everyone who's commenting on their choices, is that all of them are shuffling deck chairs around on the Titanic.
The Titanic didn't sink because of the arrangement of deck chairs, and Mozilla didn't sink because of any features they did or didn't provide (and GNOME didn't fail to achieve significant market share in the first place because of features either). The actual problem doesn't have anything to do with the stuff on the deck at all.
It's the one-two combination of vendor-lock in and bottomless marketing budgets. Since most of the value of the Windows platform and the Web is the immense amount of stuff that's built on top of it, there's a huge lock-in effect that prevents you from even reaching parity, much less exceeding it. And in order to overcome the marketing budget and pure inertia, you need to be ten times better, not just on par.
If GNOME becomes as usable as Windows, it won't have anything to do with what they actually do in the desktop environment itself one way or the other, whether it's continuing on the road they take now, or reverting everything back to the way GNOME 2 was, it's totally irrelevant. GNOME becoming usable will be entirely because of Valve investing in Wine, combined with a whole bunch of other apps moving to Electron and shipping Linux versions because heck why not?
Unfortunately, while they are probably already on par with Windows, they aren't ten times better than the Mac:
* The Mac has a bottomless marketing budget. Good luck competing with that, GNOME.
* They've shown a lot more restraint than Microsoft has, probably because macOS is considered a niche product to round out their catalog rather than being their one and only operating system like Windows is for Microsoft. They have even reversed course on a few anti-features, like adding back USB-A ports to the Macbook Pro even though it made the laptop slightly thicker. And unlike Windows RT, they didn't lock down the ARM Macs.
* Those tectonic shifts I mentioned that made Linux usable? They also make the Mac usable, because Wine is open source and Electron is basically its own operating system. Anything truly good that GNOME does, Apple can copy it just like Chrome copied all the really good stuff from Firefox.
The issue in open source is with projects with not a strong leader ship , then you get some wanna be designer copying Apple because they read some book and now he thinks that shit needs to look and feel different. Then you have developers that want to work on new cool stuff and not maintain existing code, so every few years you get a full reset but because of inexperience or incompetence the new version is buggy for a few years, it gets fixed but then the developers are bored and want to rewrite it using some new ideas/tech.
What would work IMO is someone with money paying the developers and the designer but force them do do customer support, you don't play with the new shit until most tickets are resolved and customers waiting for response are satisfied. maybe a paid support would help too.
Did you reply to the wrong comment? Because nothing you said has anything to do with any of what I wrote. It's not even a counterargument. It just reiterates the original point, which I don't entirely disagree with, but don't think has anything to do with GNOME's lack of market success.
Market success has almost nothing to do with product quality. Well, okay, it does, but only in the sense that you need to not actually be a total fraud. You can get away with dismal quality as long as your marketing is good [1]. In formal terms, software development is a loser's game [2].
This implies that the churn you're complaining about has nothing to do with market success. You might not like it, but that doesn't mean the failure of Linux on the desktop is actually caused by it.
MacOS is the perfect example: it's both easy to use and the preferred choice by many professionals.
"Hold on", you say, "professionals want options, like user-expandable RAM etc". No, that's the misconception about the concept of a "professional". Unless you are a hardware engineer, tinkering with your notebook's internals is the absolute opposite of professionalism, its either a completely misguided waste of time and money, or a perfectly fine hobby.
Real professionals get work done. Customizing their workspace is something they feel ashamed to do, because it's procrastination at best.
What? Professionals customize their workspace to make it more efficient all the time.
Just that exactly is the workspace changes a lot, the hardware internals are hardly ever a consideration for any professional, it's easier to just buy something that works well from the start (and hardware costs peanuts).
> I don't know where this persistent myth comes from that people love choice and tinkering, because they clearly don't.
It's often said that people don't want more choices, they want to be confident in the choices they make. It's the strength in simple product lineups like good, better, best. Googles lone search box is good example. Apple's product line also tends to be a good example at times.
Most people don't want to tinker if they can avoid it, but many will come to appreciate the power of the advanced tab if and when they need it. These startups should take the "people are lazy" line of thinking to heart and make customization profiles easy to share, whether by direct link posted to Slack or a public customization "store" a la Chrome Web Store.
Google is anything but a simple box that gives you answers now. It hasn't been that for a very long time. I wish it was.
It is a box that gives you a screenfull of ads, some spam copycat sites I wish I could remove, and a lot of clutter. As many people on here have said before, it only gets away with it because it owns the web browser through Chrome, Android and its Apple deal for Safari.
>it only gets away with it because it owns the web browser through Chrome, Android and its Apple deal for Safari.
From someone who tried the switch to duckduckgo and uses ddg as the default engine: I can't remember the last query I typed without adding "!g". Google doesn't get my queries because the service is shoved down my throat, it gets them because the alternatives I tried are worse wrt the total scope of my queries.
Just tried to search "t-sne" was pleasantly surprised the first result [1] was what I found on DDG and a good result. Compared to Google which couldn't even find it on the first page. Most of Google's result are not exactly useful, and the only useful article is not my favorite in terms of quality. Order being.
1. Wiki article
2. Github repo
3. API documentation
4. Introduction with Python Code(Not my fav quality article)
5. Guide in R
6. Research article
I can't quantify what is better. But Google didn't give me what I wanted but DDG and you.com did. So congrats. My only issue is seeing Medium articles above the web results. While Medium might have the answer in this scenario the top web result was correct but de-prioritized which was incorrect from my prespective.
I looked for "Rewe" to see wether I get informations about local supermarkets, got a panel-view that is expensive to parse (eyes have to move in multiple directions, content is not clearly focussed), and was rewarded by getting no information about the local markets. Panel view only, forever, seriously, ONLY makes sense if you already know all the elements that will be displayed. Two giant rows of icons are terrible UX. Please make your default view easily-digestible. I enter a query, get a set of results. That set needs to be represented in a way that the brain actually wants to operate on. Nobody wants to operate on a set by digesting a table of unknown contents. The correct UX is a list.
I didn't bother checking customization-options, the ux was hostile and the results did not give me what I want. Pass, sorry.
Dude why do you have to fill these threads about all the spam on search with yet more spam promoting your crappy Bing-based search engine. Just let the conversation happen without all the self promotion ok? Jeez Louise this is annoying!
I still love one of the design stories of Steve Jobs in Isaacson's biography. I don't remember the exact phrasing, but essentially someone at Apple was showing Jobs the new DVD burner software the company wanted to distribute. It likely had a ton of options to it (like other dvd writer software of the time). Jobs' response was roughly "Hmm, ok. Here's what we're going to do. There's going to be a box, and you drag the files you want on the dvd into the box. Then we're going to have a button that says "Burn DVD" and when you press it, it burns the DVD"
I've been using Kagi for a little while. The customisation isn't really necessary - the defaults work fine - but it's quite nice to have when you want it. I've banned Pinterest from the search results.
> I think 'more customization' which is a theme with a lot of these alternatives is a fundamental dead end.
It doesn't need much--a single button with "Remove this site from future results".
That would solve 99% of the problem because it would destroy SEO optimization since whole swathes of people would have different criteria due to the blocked websites. It would also tell Google which websites suck if they didn't already know.
Alas, it would also destroy their ad revenue because once a site got overly scummy people would start delisting it.
> Because it literally still is a simple box where you put a question in and it gives you answers without needing to do anything else.
And that's great if it works! The problem is that once it fails (and, at least in my use-case, it does so quite often), working with it becomes an absolute pain.
I, too, would prefer an omniscient box perfectly answering my questions. But it clearly doesn't exist. And a box with screws to adjust so that I can eventually find what I'm looking for is the second best thing.
There should be a "make me lucky" button - to tell you what you ought to be searching for, since it's got enough data to know you better than you know yourself :)
>I don't know where this persistent myth comes from that people love choice and tinkering, because they clearly don't.
I think it really depends on the product.
If it's something like Photoshop or Vim which are used by professionals for multiple hours a day and productivity is to be maximised, customisation is not just a nice-to-have but a necessity. Power users appreciate it too.
You're absolutely right in 90% of the cases, though I'm not sure which box web search would fall into.
I think 'more customization' which is a theme with a lot of these alternatives is a fundamental dead end.
I tend to agree. It's an attempt to deal with bad design by putting in switches, options, and knobs to tweak. That's because 1) it's easier than focus groups, A/B testing, and taking video of users using the thing, 2) design takes some artistic talent, and 3) it lets the programmer blame the users for the problem.
This is a vice of open source people. It's why Linux on the desktop has never taken off. "Just edit /etc/conf/foo/bar/prefs.txt" is not a design. Nor is "On the visual side, you can modify everything about the way things look, even being able to write your own custom CSS."
I'm critical of Google's search, but this is not the way to fix it.
Consider just few more search options, alongside "News", such as "Scholar" and "Noncommercial", for when you're overwhelmed by crap. And "Popular", for when you want the crap. Don't add another interacting dimension of tweaking.
What counts as customization? I might agree that most people don't care to change font sizes or colors or even themes but they might want to be able to tell Google to never return results from a particular site. Is hiding results from a site customization? Is it the sort of customization that would overwhelm a user if they saw it as an option?
There's a book I love to send people: https://www.amazon.com/dp/B001REFRZG/ref=dp-kindle-redirect (old school. no kindle. just hardcover. :) ) "Something Really New" How innovation works. First question in the book is: you need to come up with a new idea for a faucet company. Customer research says: Users want lots of variety in their faucets. Everyone then immediately comes up with the same exact ideas: faucets that are easily customizable. Faucets that have skins. Etc.
When really innovative stuff is just about removing steps. If a process has 10 steps, remove as many as you can, and now you have something truly innovative on your hands.
I feel like Google did exactly that. Pre Google steps: search for XYZ, wait, check first link, second link, spam, spam, check next page. Post Google: search for XYZ, get XYZ.
Of course blog and SEO spam is such a problem on Google now that it looks more similar to your Pre Google steps.
I don't know how to easily fix that though. Simply "crawling the entire internet" is still not a simple problem, let alone doing something more useful with the result than google can. Ahrefs is an interesting business but not what people mean when they say the next google. "Machine learning" but I think google is all over this already (and has been for years).
Google does a bad job at getting user feedback about results while reddit does better so people search reddit, maybe a hybrid is an opportunity.
eh... except google is now basically just the pre-google product: Search for XYZ, wait, skip promoted link, skip second promoted link, third promoted link is actually the direct competitor to what I fucking searched, click result that was on top ten years ago, but is now almost below the fold.
Or worse - Search for exact term: get a page full of "Missing X - must include X" links hidden in tiny text below a result, click "Must include X" get the SAME FUCKING RESULTS again, click tools, click the dropdown, select verbatim, finally see decent results
This is not a binary "the people who don't want" / "the people who do".
The old Google was able to do an exact search using Boolean expressions. Man, I miss that all the time....
It's not rocket science to provide a user interface for this. A combination of GUI configuration, keywords, and specific URLs for these cases would go a long way here without bothering the regular user in the slightest. Yes, it's probably slower than in earlier times, pure search volumina and its handling by AI weighting etc. considered. But there is also a "don't want" on the part of Google and the other players.
Exactly this. Scrolling through the customisation feature list, I just see "more work", "more work", "more work" - and it's not super clear how doing all this extra work is going to help me. And I'm in the target audience for a "technical" search engine; this is a total non-starter for the 99.999% of the world we call "normal people".
The useful ideas in here is the "just my data" search, linking into external providers across siloes. Of course, Microsoft, Google & Apple already have this as long as all of your data is held in their ecosystem.
There's a certain type of person who loves buttons and knobs. They're also the same sort of person who might decide to make a new search engine. But yeah, more knobs is definitely not something the market is asking for.
Google has these options, but these are solved by AI. So with that you come in a catch22: people want personalised content, but rather not have their data given away.
If you open YouTube without login, you get all kind of rubbish so you want to login. Probably this is by design: you want what google wants: results based on your data. Both happy.
Now if there was a privacy friendly way of doing this, I am all for using that. I just don’t see how, and I don’t see who wants not to gain a profit if you would have that data.
So the next google probably is another google.
If Google has these options, why can't I search for any image and have it not return any Pinterest results? If I search with -site:pinterest.com, I get Pinterest's million alternate tlds, if I just search for -pinterest, Google decides, in their infinite wisdom, that I didn't actually mean that and ignores it.
I haven't used Google's search for a long time. The sticky ones for me now are gmail and YouTube. gmail because it is a lock-in, once you have the email address, I don't think you can ever change. Since I am a Chinese now I have to think about if Google stops serving me in the future, but I am deeply invested in my gmail account :( The other one is YouTube, where it can work without login.
Google's simple box is not attractive when others can provide the same quality in search results.
I agree, none of these are the next google. Except perhaps YouWrite which taken to the limit is asking an AI for the answer rather than searching the internet.
I agree; choices and tinkering add complexity to a search engine, and search engines need to be simple.
That aside, a search engine needs to be really focused on privacy. Customising it implies that the user needs to log in and remain logged in. I'd much rather no do that with a search engine, and I don't want its behaviour to react to my identity either.
This! This is why I love Apple products too. It abstracts away many of the things that 99% of the people (general public, not HN community) don't need or use.
It's better to focus on the core basics and perfecting it than to add many features and try to support and maintain them especially if they are features that only 1% of the users use.
I agree with this, however it could be used as a platform for others to provide their own config and you can piggyback off the work of others.
It's kind of like ad-blockers where you don't have to maintain a list of domains to block. Others do that for you. And then people could create hosted version of their simple box with all the infra taken care of.
Couldn't agree more. For the average person outside of our bubble, google "just works". They don't give a fuck about customization or the occasional ads. There are still plenty of people who click on the sponsored links without even knowing, or caring, that they are ads.
Spot on. Back in the days of web portals, millions was spent on making them customizable. I can't find a link, but I believe back in the early 2000s reading that less than 5% of people actually customized their portal.
It never fails to amaze me how many people, apparently triggered by these omens, come out and say this as if there was never such a thing as a default configuration. "But you MUST configure!" Um, no.
Right. Their premise is “There’s no average human”. IMHO, it is ridiculous.
The majority of people are just average every day normal ones, and they don't give a flying fuck about customizations.
At https://you.com we believe in choice but not force it. It will just work out of the box, but as folks shop or get really into something like coding - we have heard from many users that they like or dislike certain sources or apps.
Like w3school - it's in every search engine but some folks hate it so they can downvote it.
I personally benefitted a lot from the ability to like the reddit app once and then see more real reddit results?
I would go one step further and question the premises that a) the next google must necessarily be dominant/monopolistic and b) that said dominance must necessarily ride on top of free user economics.
What TikTok shows to me is that users want specific things when they go to a site: for TikTok, they want fresh entertainment, plain and simple. I go to HN when I want tech discussion. I go to reddit when I want aggregation of niche topics. I go to costco.com when I'm looking to shop.
Google is frankly a horrible experience for ecommerce, it simply cannot compete w/ the likes of Amazon or any retail store website, really. Being a search portal, it's fundamentally incompatible with the concept of evolving through permanence of a hivemind; every new search is like reseting a would-be community to zero all over again, so you cannot gradually build up a collective commons on Google like you can on HN. Youtube has recently gotten pretty bad with cycling fresh content on the front page. For stack overflow sort of stuff, any other search engine does more or less the same. For trivia, Siri/Alexa are fine substitutes. The Google search properties have become mediocre on average; they're not particularly great at any particular thing.
Are many of these domains walled gardens? Yes, think of things like Doordash/Uber Eats grocery catalogs, a ton of brick and mortar shops are choosing to integrate with these delivery apps, and these catalogs are completely invisible to Google. News SERPs on Google are often garbage since they just link to paywalled content half the time, might as well just get a subscription from the actual news outlets. Etc.
IMHO, the next Google is already here, and it's everybody else wisening up to the simple business fact that they need to own the top of their funnels.
i think more customization can help you get power users to spread the word, as long as it doesn't get in the way of someone who just wants to do/read/understand as little as possible to get results
I just went on a mini vacation to Vegas, and was thinking how nice it would be to just call up someone and ask for some simple advice. I did a bunch of online research before I booked things, but man was it painful. There are so many copy-cat blogs who just throw together a bunch of basic information with no real research done in order to get those clicks/adwords.
For example: best hotel pool in vegas. Seems simple enough. Circus Circus actually has a waterslide, but if you dive deeper (read a hundred reviews manually) you find out parts are often shut down, and the hotel itself is quite trashy and smells bad and has bad service (explains the really cheap room rate). But do you find that information on blogs? No way, they just include all the hotels with pools and copy in the verbiage directly from the hotel websites or other blogs.
There's probably an actual traveler blog out there that tells you all this and has great information, but it's hidden by all the SEO optimized trash blogs.
And this example can be applied to so many things we do all the time. Try to find a product you want on Amazon without spending half a day sorting through reviews and trash blogs.
So, personally, I think the future will be actual human service. I'd pay a few bucks to call up a service to answer these questions definitively for me.
I find reddit works remarkably well for this. I've gotten great actionable recommendations within an hour or two for:
- good coffeeshops
- tacos
- Korean grocery stores
- places that have traditional style al pastor tacos
- finding a specific coffee brand at a local grocery store
- finding EDM songs similar to a particular song (dullscythe)
- hot chicken
- canolis
and a bunch more similar things. I would think if you posted to the /r/vegas subreddit asking about the coolest hotel pool in vegas, you'd get a bunch of up to date info.
I have found that this only works if you take recommendations with the understanding that you're almost always getting "the reddit answer". Every sub has theirs, and their reasoning might be because they agree with some entirely unrelated thing, and have bonded over that to the point that other recommendations between each other are heavily weighted and rapidly creates a "go to" response.
Example: check r/sandiego for [insert type of taco] suggestions. There should be thousands of options if you're actually from San Diego but you'll get 5 or 6 at most that stand out a lot, and 1 or 2 that get the most up votes or responses.
Now that may in fact be the "best" (depending on your and their definition) but you might also find that it's just the restaurant that has the most interesting social media presence, or the one you go to in order to signal that you're "one of us". Maybe that's exactly what you want, maybe it's what you want to avoid, but either way it's important to know the mood and trends of the subreddit as well as general reddit culture (and don't tell me it doesn't exist).
I think this is a really good observation, and "bonded over" is a good term. At the extremes, people will even recommend the thing and disparage other options when they haven't tried either one themselves. They're just trying to be part of the group.
Edit: A couple more real examples off the top of my head:
- In some circles, everyone recommends getting a Shure SM7B + a Cloudlifter for recording voice. It's not a terrible option but not really better than many other mics (many of which are higher gain, removing the need for the separate preamp), and even if selecting an SM7B, depending on your audio interface you may not need the Cloudlifter. It's just a preamp, not magic.
- On Reddit's /r/NewZealand, everyone loves Whittaker's chocolate and hates Cadbury. Whittaker's is locally owned and honestly is better than Cadbury, but from reading the discourse you'd think Cadbury was completely disgusting. Any sort of moderate opinion (e.g. liking both) is not allowed.
So your observation is spot-on, that a subreddit community will produce unbalanced opinions due to groupthink. Likely true of any active community forum that would give recommendations such as this.
However, if I was looking for recommendations about a good taco place when visiting San Diego, I would bet that the groupthink recommendations of an active San Diego subreddit would be vastly superior to what you'd get on Google.
Unfortunately, this points to gamification of reddit recommendations as being a major target for the same folks wrecking web search results with their SEO blogspam garbage.
>... "the reddit answer". Every sub has theirs, and their reasoning might be because they agree with some entirely unrelated thing
One way to stop this is sort by controversial. It allows you to see what the hivemind votes down at the top.
I also find city subreddits can be much more polarizing and needing controversial sort vs interest subreddits. The only common thing city subreddits share is geography so you get wildly different opinions which controversial sort helps with. The subreddits dedicated to an interest are much more susceptible to the reddit answer.
The other option is search for an /r/askvegas or similar -- if a city subreddit has a dedicated subreddit for questions you can usually get better answers on the asking subreddit, but they are closer to the reddit answer than a general city subreddit will give you.
You need to be careful about reading the subreddit rules when you do this though. In my for-locals subreddit tourists asking questions like this were a plague, even when there was a very clear and prominent rule forbidding any kind of tourism content.
Also important to at least try searching at first. So many people thought they had a super original question but it'd really been asked and answered to death already.
Coincidentally, the top post today on /r/SanDiego was a guy asking about the best burger in town and the mods ripped it down, saying "Nice try, zonie."
This tends to work for me as well but signal vs noise ratio has worsened over time. Despite Reddit's questionable decisions in the past years it still has useful information for all sorts of things.
Reddit's own search has become less reliable so I search via DDG like `<query> 'site:reddit.com'`, falling back to `!g <query> 'site:reddit.com'` if the former doesn't get me anywhere.
Yea I always just put in the question I want into DDG and then just add reddit in the end. It's so much better than any blog or anything because at least it gives the semblance of real users giving their opinion.
I like you.com search better, as their reddit app yields less fussy results than the reddit search itself and i normally don't have to append reddit at the end of my query.
That's because Reddit is one of the few places left on the web that is open (aka: not a walled-garden like Facebook) and has people talking to people without financial incentive. HN is another.
Reddit is changing fast though. It's becoming more walled-off every day as they push to become the next Pinterest, and it's increasingly plagued with spam and bad actors.
Google brought this upon us by lifting SEO spam websites to the front page while pushing down helpful websites written by humans without incentive, like Ask MetaFilter.
that's why we have a reddit app inside https://you.com - you can set your preference for it once and boom, you'll get results from reddit whenever relevant.
And reddit works because it's not an algorithm that decides which post is good, it's people with upvotes. And that in turn can be used by search engines.
Maybe that's what would fix Google. Up&down-votes for web results. Would probably be the easiest way to penalize bad sites and promote good niche pages. But after they removed downvoting in Youtube, I wouldn't get my hopes up.
When I visited my mom in Romania, I was amazed that hair dressers filled *exactly this purpose*. It almost felt like you were "Googling" for 30 minutes and getting a haircut as a side-effect.
Customer: How is the newly opened Spa?
Hair dresser: Other customers said the water was cold.
this is so true! when i first moved to the USA, a librarian in austin texas was so incredibly helpful to me in sorting out/understanding the "i've just moved to the USA" process. i knew not even a handful of people and she was so selfless and generous in her efforts to help me navigate pre-web then-INS and TX state bureaucracy as a 0day new person here.
How is finding the best X at something from I assume hundreds if not thousands of choices, a simple question ? In particular, would asking any human give you the actual answer ?
At best we’d be in the same situation as looking at guide books or Michelin guide kind of rankings, created with human oversight but static in nature. More probably we’d get the most reputable hotel with no actual evaluation if it is effectively better than the myriads of others.
And that’s without digging into what you value in pool experiences vs what any evaluation system took as criteria to rank them.
I think this is an example of how we are trying very hard to simplify problems that are inherently complex. And sometimes it kinda seem to work, so people start forgetting it still is complex and rebuilding it from scratch means reapplying magic on it again.
How could it possibly be only a few bucks to answer questions like this though? A human would have to be doing the research and writing for you. If that human is trying to answer a query a minute to earn a reasonable wage then your results are going to be rushed and poor. If the human takes enough time to find a good answer then the search will need to cost enough to make it worth the time of a skilled researcher.
I could see a "virtual assistant" version of this where you pay fifty dollars and send a request like "Reservations in Vegas for these dates at a hotel with good pool."
Fifty million dollars would change my life. I would take a guaranteed fifty million dollars payout over one in ten chance to get a billion dollars.
What are the chances that I cumbents will stand by idly as the "next Google" just takes over? Best case I can think of is the incumbent imitates the upstart and becomes better.
The future is trust networks. Where you trust a number of friends, who trust other people, etc. and you can use a matrix of trust to retrieve review scores, etc.
Imagine trusting HN, visiting Amazon and getting all the reviews from other HN users ...
I agree with everything you said about the spam blog problem.
And yet Travelocity (and others) would have told you how terrible Circus Circus is in five minutes or less.
22,000 reviews, 3.2 stars out of 5, among the worst in Vegas (among major hotels) with a gigantic number of reviews. Its room rate alone helps you to begin immediately forming a good conclusion about its quality. This isn't a subtle thing.
Oh, but that's not realiable, one might say. Yes it is. It does a great job of approximating the quality of the hotel in question, and it's an exceptionally easy and fast means to narrow with. It isn't a perfect approach (is it really a 3.1 or 3.3 star quality?!?) and doesn't need to be, it just needs to let you know that Circus Circus is garbage, and it does exactly that.
You can also get some decent mileage out of Booking.com and checking reviews for keywords. That won't help you quickly find the best hotel pool, but it will help you find out what people think of a specific hotel's pool (e.g., Circus Circus).
I can empathize exactly. Trying to do a bit of research on an upcoming trip seems like something that should be fun or at least not feel like an overly burdensome exercise. But as you point out it's a giant time drain filled with frustration. Everything from the booking sites that create a high pressure experience("3 people are looking at this room right now!") to the influencer chum pages that seem to be more about marketing their brand than providing practical information. You can lose two hours trying to research a trip and still not got anywhere. A service where you could talk to local people would be great. That's a service would be worth paying for, like a travel advisor.
I can imagine the first two steps of their process working for a human-assisted search machine. Specifically, the "finding sources of information" and "filtering information" steps. But, I'd imagine the human workers would still need more complex / configurable search tools than Google (which is probably what the workers in the study used to find their sources).
If you don't mind staying on Fremont Street (I prefer it to the Strip), the Golden Nugget has a pretty nice pool setup (complete with waterslide) and is a "nice" place to stay.
Amex has a really good concierge that handles this. And it's free with most cards. Absolutely underrated, especially when you're traveling internationally
Exact same experience except with Iceland. Google "top attractions in Iceland" and you get a bunch of garbage posts telling you about the Blue Lagoon. I ended up reading an entire book on iceland tourism and perusing many blogs and reaching out to many friends, and the Blue Lagoon is definitely not one of the top spots.
Feels like you asked the wrong question. You asked about the best pool, not the best hotel with a good pool. But even do, I've read many sites that tell a little bit the hotel when I search for the best pool. Not one of them listed circus circus
This maybe works for Vegas, but what about places without a large demographic on reddit, or where the locals wouldn't be writing about these things in a language you speak?
Another important feature of Kagi that I'm paying close attention to is: they are currently privately bootstrapped as far as funding goes.
To me the fact that Kagi is not currently VC funded is huge for me as far as adoption. Every customer facing VC funded startup I've worked at inevitably starts to institute increasingly anti-user practices while grinning and talking about "customer first!"
I know it's a huge ask, but if Kagi remains privately funded/non-VC I'll happily pay the moment I can.
I've only been using the service for a short while now but have been enjoying it a lot. The ability to blacklist domains has already dramatically improved my search.
It's really good in some ways, and very lacking (for me) in others. The search is fine. Good enough that I would switch. However, if I was out and about and quickly needed directions to Walmart, my normal flow with Google is
Pull out phone -> Safari -> type "Walmart" -> click on the map -> Maps app opens and starts guiding me
When I switched to Kagi, the flow went like
Pull out phone -> Safari -> type "Walmart" -> top result is Walmart.com... no address to nearest Walmart to be found -> Close Safari -> Open Maps app -> search for Walmart
And it got so annoying to have those extra steps. I know I can change my workflow and get used to it. But it wasn't just directions. It was other basic searches. Like if I needed a phone number for a local business. It drove me crazy. I really hope Kagi gets better at those sort of things. I want them to succeed. But it was just too much friction for me.
To be perfectly honest, I cannot even remember making that change, but also couldn't find any info on whether Firefox or some extension may have done this. I will leave it disabled, however, because its purpose seems to be analytics, and opting out of (at least some of) them this way seems like a good idea.
Since disabling it this way removes the `sendBeacon` function from existence, you should just be prepared to properly handle its absence.
Is it always about analytics? Or is it ocassionaly also a useful API for other tasks?
I know little about front end JS dev, but the docs say it’s just a way to send an asynchronous request (no response) with a guarantee that the the request will be sent out, even if the page starts unloading. That’s apparently not a guarantee with the two other to send requests - XmlHttpRequest and fetch api.
I agree this disrupts my muscle memory, but if we're honest, that particular workflow isn't necessarily optimal. It's just what Google has trained us to do. If you're seeking directions to Walmart, the most efficient (and one might argue intuitive) method is to open your maps app directly, and type in "Walmart".
It's free while they're in beta. There's a "waitlist" but put your email on it and you'll get an invite within a week. Give it a try.
I've been using it for a couple weeks now on my work laptop and for programming-related searches its been great so far. And the usual annoyances that show up at the top of google and DDG don't show up on Kagi (geeks4geeks, etc).
The cost per search query is incredibly small. You could probably have <1% premium users. If your search engine is used by a billion people a day, 10 million paying users are still enough to pay a few thousand people developing your product (depending on your location).
I believe products can work that way, offer premium service and features for the very few that need it and a basic service to anyone else. In the end, the free tier is cheaper than what you'd have to spend on marketing otherwise.
I am a Kagi beta user, and I intend to pay when they start charging. I have wanted a paid ad/tracker-free Google for a decade now. I want to be the customer, not the product
I guess it depends on whether you consider "The Next Google" to imply that it becomes a the dominant company in the space, used by every "average user", or if it's enough to be a niche solution for highly technical users who prefer it to Google.
Also important for anyone actually thinking of taking Google on, very few of the features listed are things Google can't easily do, too. Attacking their strengths is crazy. You better have something both crazy good and hard to replicate by someone with more money than god.
Whatever replaces Google will be doing something that Google can't without causing them other problems. The first thing that comes to mind is make them choose traffic vs. advertisers (I don't know, if I had an idea of how to, I would not be writing this), but they're big enough that other wedges could start chipping away at their margins.
Actually, you are spot on. One simple feature of the Neeva app is that it shows inline search results as you type into the URL bar. This is because we aren't trying to show you ads, so we don't need you to visit the search results page (where Google and others show you those ads). We just show you the results straight away in the suggest experience. Now, this isn't going to show you everything you care about and you can still click to see the search results page. It is just handy to be able to quickly get to where you are trying to go and especially if it is likely to match what you are looking for (e.g., a wikipedia link). This is something Google cannot bring itself to do because it would be cost way too much in terms of lost ads revenue. There are other examples like this where Google and other ad-supported search engines just can't innovate, can't change the search experience. The current way of searching is too lucrative and there is too much business inertia around it. That's why Neeva is interesting and why I left Google to join and help :)
But that is exactly how Google searches worked on desktop platforms for more than half a decade (Instant Search), not some kind of a new idea. Given how long they kept that feature on, it seems pretty obvious that it can't have been the kind of revenue killer you suggest. If you can serve and display search results for a given possibly partial query, you can obviously serve ads too.
I was talking about mobile. As for desktop, Instant Search was serving up full page results instantly, which included ads. That's a different thing altogether, and of course, in the case of Instant Search there was plenty of room for both sponsored results as well as real results. On mobile there isn't.
I think this comment is a bit strange in the present, considering search engines like duckduckgo, which is basically Bing promoted with a "we don't track" advertising campaign (also hashbangs are pretty cool). DDG is not at google numbers, I know, but you don't need google numbers to make money.
I don't think privacy is a very special angle to advertise from either, promising to remove amazon-affiliate blog-spam from results for example, would be a major feature in this space as far as I'm concerned. Being able to edit searches, and potentially gain some intuition for how the search space is set up, might be a much more significant feature, depending on how people take to it. It might flop but atm I'm excited to check it out
I've long wondered about a search engine that doesn't index a page that has AdSense or similar code. There'd be a lot of collateral damage, but it would knock out a lot of annoying made-for-adsense sites at least. Basic thinking, but along the lines of what you suggested.
> - who pays for the service (ads? users pay? Average user will never use a paid service if a free one is available)
- how to resist attacks against the algorithm (Google has been fighting spam for decades)
The solution for o both of these might actually be a paid service. If you have a paid service, there is a possibility of it being profitable with much fewer users. As an example, let’s say you have 1,000,000 users at $10/month, that is a $10,000,000/month which might be enough to run the service and provide a comfortable profit.
With regards to the spam issue, the fact that you have a small user base would be to your advantage. Because there are so many Google users, it is in websites’ economic interests to spend money to try to game the algorithms. With much fewer users, your paid search users may not be worth it for the sites to spend money trying to game your algorithms.
It will still pay to put ads in to paying customer's feeds. It's more or less inevitable if the customers tolerate it. If you think that's impossible I'd point to how streaming services are now serving up ever more adds.
The spam issue can trivially be addressed by implementing actual penalties for rule-breakers. If it takes a long time to acquire a good reputation & ranking on the search engine, you're unlikely to risk it by doing something nasty in fear of your domain, keyword or brand name being banned for a long time.
Adding on to this, customization is nice but customization is not why DuckDuckGo isn't as good as Google. The reason nothing is as good as Google is because Google indexes way more content than every other service that I'm aware of
Lately Google has gotten so bad that I've even occasionally brought up Bing (which I've often referred to as the Zune of search engines) and gotten better results. It's looking more and more to me like Google has stopped trying to improve things and are simply milking their dominance for all it's worth.
Really? Or are you the one I should have refrained from feeding.
But if you must know:
First you need to collect a lot of content from the internet. From many different sites. With very different types of code structure. Broken html. More often than not behind some SPA JS code. Behind robots.txt files and bot protection efforts.
So the first problem to solve would be building a crawler at scale. That is able to crawl anything your users might want to visit but don't know of yet.
Then storage and retrieval. You need to store and update all this content your crawler collected. You need to enrich it with meta data and organize it for efficient retrieval. So that you can surface it to your users when they use your search engine. Indexing, structure, build g connections between content pieces. A lot of interesting things to think about.
Then there is the front end. Make it easy to search, to refine. Surface relevant content for search queries.
OH maybe I forgot, but you probably need to do a bit of engineering to make your system understand the users' search intent.
This is relatively straightforward for a limited search and document space up to a few million entries in your DB. A few million documents should be doable with off the shelf parts.
Bigger than that. I would applaud you if done with orders of magnitude lower than Google. Anyone would.
All of this is a long series of solvable problems. I should know, I've dabbled in solving most of them. This is why I suggest actually taking a stab at it before you dismiss it as impossible.
There are some problems that aren't as big as they seem. Parts of an SPA can't be reliably linked to anyway even if you find interesting text there, so you can just leave them out of the index.
Likewise, there isn't as great of a need to keep a fresh index as it may seem. The odds of a document changing is proportional to how frequently it changes. This is a bit of a paradox, where even if you crawl really aggressively, the most frequently changing documents will still always be out of date. Most documents are relatively stable over time. You can actually use how often you see changes to a document or website to modulate how often you crawl it.
The bad HTML is quite manageable. You really just need to flatten the document to get at the visible text. Even with really broken formatting, that's manageable.
The storage demands are also not as bad as you might think (most documents are tiny, sub 10 Kb), there are ways to lessen the blow on top of that. Both text and indexes can compress extremely well. Since you're paying for disk access by the block, you might as well cram more stuff into a block.
Most of the crawling concerns, in general, can be gotten around by starting off with Common Crawl (even if I do my own crawling, which also is finnicky but manageable).
> This is relatively straightforward for a limited search and document space up to a few million entries in your DB. A few million documents should be doable with off the shelf parts.
Right, so shouldn't the question be how to find the documents that are even candidates for being search results? Most documents are not ever going to be relevant to any query ever. Get rid of that noise and your hardware goes a lot longer.
I'm running a search engine on consumer hardware out of my living room that can index 100 million documents. Go a bit higher budget than a consumer PC, and you've got 5 billion. That goes a long way.
> Get rid of that noise and your hardware goes a lot longer.
What qualifies? What defines signal, what noise? I agree, that a lot (probably nearly all) pages will receive very, very little traffic/search requests. But are these therefore not relevant?
> I'm running a search engine on consumer hardware out of my living room that can index 100 million documents.
That's extremely cool. I would love to know more. To me an impressive feat already.
I think I was editing the comment while you were replying. Sorry about that. I was just adding to it though, didn't really rug pull on your response so I think it's fine.
> What qualifies? What defines signal, what noise? I agree, that a lot (probably nearly all) pages will receive very, very little traffic/search requests. But are these therefore not relevant?
Now this is a proper difficult problem with (probably) fairly subjective answers. I do however think it's something that warrants serious investigation. It's probably a decent candidate for a machine learning model combined with some manual tweaking for sites similar to wikipedia or github that have absurd amounts of parallel historical content.
Developing heuristics for this is a bit of a hobby horse of mine. It feels tantalizingly almost doable with just a little bit more resources and time than I have.
> That's extremely cool. I would love to know more. To me an impressive feat already.
Yeah it's at <https://search.marginalia.nu/>. I've built all the software myself from scratch in Java[1], and I'm doing my own crawling and indexing. The machine it's on is a Ryzen 3900X with 128 Gb RAM. Most of the index is on a single 1 Tb consumer grade SSD.
I do use a MariaDB database for some metadata, but I think it will have to go as its hardware demands is becoming a serious bottleneck.
[1] Despite using Java, I should say regarding the index. This is approaches sunk cost at this point. Building a search engine index is not something Java is at all suitable for, its limited low-level I/O capabilities is incredibly handicapping.
> I [...] didn't really rug pull on your response so I think it's fine.
No you didn't. All good. And I learned a lot from the extended answer. So I am thankful for the explanation.
> Developing heuristics for this is a bit of a hobby horse of mine. It feels tantalizingly almost doable with just a little bit more resources and time than I have.
I can totally understand the feeling. There are quite a few things that I'd like to go deeper into either at work or in private. But alas time.
> Now this is a proper difficult problem with (probably) fairly subjective answers.
I agree. And I don't have answers ready. A lot boils down to preference. Personally, for example I prefer written content over video. Except in a few areas were I like (some) explanatory videos. To me it comes down to the question of how easy I can skim the content when I am looking for an answer.
On the other hand - for deep immersion into a topic I use multiple media formats.
In terms of web search I sadly nowadays need to sift through a lot of seo-fied content that is there either to build a (personal) brand or to attract clicks for advertising revenue/affiliate revenue.
So in principle I agree with you on the noise problem. Still I also believe that there are real great gems to be found in the long tail. When I still feel like I came late to the party, but when I started out in the web in '97 there were so many lovely, quirky sites. So many places that people had put a lot of time, energy and thought into. And sites so packed full of information that I came away not only with more knowledge, but in awe that somebody would give this knowledge away for free.
There also were quite a number of horrible sites (my first ones probably included). So there was a noise vs. signal problem back then. Maybe not to the extent today, though.
> The machine it's on is a Ryzen 3900X with 128 Gb RAM. Most of the index is on a single 1 Tb consumer grade SSD.
Call me impressed. Sounds absolutely cool.
So even with a raid setup for redundancy this is doable.
May I ask how you decide to add me content? Do you follow links? Do you use other search engines' results as a starting point?
I could probably shoot many more questions, but don't want to be a nuisance.
> May I ask how you decide to add me content? Do you follow links? Do you use other search engines' results as a starting point?
I initially did basically a DFS-walk originating at a few websites I liked, with some filtering criteria that deprioritized websites that didn't look too interesting. Now that I have a fairly comprehensive mapping of the space I want to index, I use a few factors like frequent outbound links from highly ranking domains to inform which new sites to index.
> I could probably shoot many more questions, but don't want to be a nuisance.
Why? I don't change my reply based on the author. I reply to a statement to the best of my knowledge regardless of the author behind it.
And I learned already a lot in this thread after the explanations unfolded.
The initial statement sounded exactly like the armchair "experts" one so often encounters. Actually this was for a long time the first time that there is a person with substantial experience in the problem space behind such a statement.
> The initial statement sounded exactly like the armchair "experts" one so often encounters.
Maybe - but [marginalia_nu](https://news.ycombinator.com/user?id=marginalia_nu) isn't an armchair expert - they've actually implemented theor own publically available search engine - which is linked in their profile.
I didn't say they are. Only that the initial comment sounded like that. And in the thread above we discussed a bit about their achievements. I really liked it and learned a lot.
The reverse index is 180 Gb, on an SSD. I do think using SSDs are a major part of why this is possible on consumer hardware. I'd need a lot of spinning rust to get the sub-100ms response times I can get it to when the index is warmed up.
Should be said I do wear through this SSD at a pretty alarming rate. I'm at 193 TBW on this disk since I started using it as an index less than a year ago.
I do have a bunch of mechanical drives I use for archiving and as intermediate working areas as well, but the index itself is on an SSD.
Thanks - I'd be keen to try this at some point, if anything just for personal usage. I've got more than enough hardware CPU & RAM-wise, if all it takes is getting a few TBs worth of solid-state storage it seems like a no-brainer.
Raising the cost of spam would be a good first step.
At the moment, spamming Google seems to be trivial with no long-term penalties if you get caught doing something nasty.
A simple rule (manually enforced on a case-by-case basis) that would ban your brand/domain for a year if you get caught breaking the rules would get Pinterest into compliance from day 1 for example.
Using ads/analytics/affiliate links as a negative ranking signal would make a lot of blogspam/listicles/clickbait disappear if their only funding method immediately makes them rank much lower below where they are no longer profitable.
This would be easily exploitable by a competitor. For example, search engines (used to) rank back links - that is other domains pointing to your domain. Some bad actors took advantage of this by creating rings of sites that voted each other up. Google responded by punishing the behavior. Then, competitors started taking advantage of this punishment by creating a network of sites that backlinked to a competitor, so they would get punished instead.
This isn’t a hypothetical example - Google actually includes in their webmaster tools a “disavow links” capability so sites can avoid getting punished for bad actors trying to make them look bad. But you can imagine if the penalties were even more severe other folks may get caught up in an unforgiving dragnet with no judge or jury and no way to appeal.
My main point is that people will find ways to game the system, and usually sharp edges (“harsh punishments”) on any system will be taken advantage of by actors, and unfairly penalize others.
Agreed, I'm not saying this is the end-game or that it will be perfect. But a simple rule (that's actually enforced) saying that you are forbidden to serve a different experience to the Google bot vs a normal visitor would take care of Pinterest for example, and they're not even doing that despite it being a major complaint especially in tech-circles where Googlers no doubt lurk.
Average users may not pay, but specialized users may pay and pay more than enough to subsidize some sort of free tier.
Not to mention, if free search engines keep devolving into an endless sea of spam, people may have no choice but to start paying. There's plenty of things out there people pay for not necessarily by choice but because there's nothing else out there that would accomplish the task at hand.
Know how you’ll find the next Google? Same way those of us over 45 found this one.
A respected friend or colleague will tell you that the next Google just works better. That’s what made Google search win: it just worked better. It’s now I found out about it and probably how you did if you’re pushing 50.
We didn’t care about pagerank or know what it is. That came after we used it and wanted to know why it worked better - or we wanted to manipulate he results.
We didn’t read a blog post telling us what’s good.
We were using another search engine and Google arrived on the scene and it was just WAY better. I was using alltheweb. Friends were using other engines. In weeks, everyone smart and productive was using Google.
Google/Alphabet may be a large company now, but never forget how they started. They were just so good we couldn’t ignore them.
I can still visualize a memory from my high school library, when my friend showed me Google search for the first time. I think I had been using Dogpile.
I was a Altavista die-hard. Google was a great improvement. Honestly I think this is just asking the wrong question. In the world of walled gardens and industrial SEO I just don’t think a search engine will be the “next Google”
That's going to be conversational/NLP search. Curious to see how Andi plays out.
The simplest explanation is the scene from from Wall-E with the captain of the ship discussing with the A.I. It will be fluently integrated in our everyday lives.
Your answer reminded me of the Ford quote: "If I had asked people what they wanted, they would have said faster horses."
I don't mean that in a derogatory sense, I just strongly believe that in 20 years we'll look back at our Googling years and wonder how we managed to find anything at all. Who know what the next pattern of information retrieval will be, but I personally think it'll be an even bigger jump than the jump from reference desks and encyclopedias to Google.
I don’t think Ford ever said that, and I don’t think that quote is a good argument for not listening to users. Cars existed pre-Henry-Ford. They were just expensive. Many people could have correctly identified the need for a more affordable car. Ford’s innovation was actually doing it.
Also the Ford company’s dogged resistance to listening to customer wants (“Any colour you want so long as it is black”) hurt Ford as soon as they had to compete on anything but price, with GM finding success with “a car for every purse and purpose”. From 1921 to 1927 Ford went from controlling 65% of the American car market to just 15%.
All that to say: “innovator vision that flies contrary to common sentiment about what people want” is often overrated and can be a trap. That’s not to say that the customer is always right… but they’re not always wrong either.
Exactly. Give me relevant links to my term, with no SEO spam. Don't correct my term, especially when I'm searching verbatim (which I almost always want). It'd also be nice to be able to downvote certain results.
Making spam unprofitable would be a good first step. Rather than trying to detect whether something is spam or not, just target how spam sites are funded: ads, analytics, affiliate links, etc and use those as a negative ranking signal.
You'll still get spam if that's all that matches your query, but now all it takes is for someone to make a page matching the query without the aforementioned items to outrank the spam results. You wouldn't need to append "site:reddit.com" to your queries because the (mostly) non-commercial Reddit results would automatically outrank all the blogspam and listicles.
If ads were downranked it would make a lot of spam/clickbait/listicles unprofitable overnight as they'd rank low enough that the costs of creating & maintaining the spam site/content would outgrow the returns from ad impressions.
> Making spam unprofitable would be a good first step. Rather than trying to detect whether something is spam or not, just target how spam sites are funded: ads, analytics, affiliate links, etc and use those as a negative ranking signal.
This would be a really interesting experiment: A search engine that ranks websites by the amount of Ads and other spam that they contain.
The problem is that for every $1 someone is willing to spend to not have spammy results, a spammer is willing to pay $10 for you to see it anyway. And the more "trustworthy" a platform grows, then the going price for manipulating it will keep rising until they hire the right MBA who decides to squander the company's reputation for a quick buck. Seems to be a recurring theme at least.
This is one of the reasons I love working for You.com.
SEO is killing Google and it needs to be addressed before it kills the internet. Having the opportunity to build an app that solves a search, instead of perpetuating a spam system, has been really rewarding... and we're just getting started.
It's crazy how quickly I got quality results for "java ed25519 bouncy castle" on both You and Kagi. I literally spent all day on Monday using Goog and DDG trying to find implementation examples of Ed25519 and AES-256-GCM in Java, trying every variation of keyword, verbatim quotes, and site scope I could think of, and ended up using GH gist search instead to find what I was looking for. The results on Goog/DDG were literally all SEO spam sites copying content from SO or the Bouncy Castle docs that I had already read.
I've been trying out Kagi as my default search engine, but will give You a try next.
I think it would probably be worthless. SEO means money spent to the search engine company, and why improve UX when you could buy your seventh super yacht?
>It'd also be nice to be able to downvote certain results.
When I'm researching stuff, I go thru a whole page of google result, and 'open in new tab'...90% of the time it's crap, and I really wish I could signal google 'this was useless'.
You want the information that you'll eventually get from visiting links, not the links themselves.
In fact often you don't even want the information, you just want to solve a problem. I don't know about you, but I don't like learning all about air conditioners and spending time finding the product available in my area with the highest quality/price ratio that fits into my budget. I just want the best air conditioner for me.
Never mind the creators of those sites. Do they really want the Reader's Digest version of their pages scraped and presented to a user without getting the click?
To go a step further - you probably don't even care about having the best air conditioner. You probably just want to reliably feel cool when it's hot at a reasonable cost.
I would prefer it returned "knowledge" instead of links but I hear you. Still, I think the fundamental flaw of all of this including Google is everyone looks for info in documents, while what you really need is a queryable knowledge graph where documents are linked to. Google made a half assed attempt at it but never took off, I'm looking forward to what that space could look like in the future.
Any Google killer needs to have search indexing technology and infrastructure as a core competency to be truly successful. Kagi has done a great job of solving some of the UX and privacy problems endemic in online search these days, but at the end of the day they could be snuffed out at the whim of the big search providers (Google, Bing) if they decide to kick them off until they can get their own indexing solution off the ground. If an alternative search engine reached Bing's level of popularity, this would undoubtedly happen.
The same goes for DuckDuckGo and others. All of the above use the Bing search API for the majority of their web results which for most use cases is not economically sustainable.
I do think there is a large swath of users who will pay a subscription for a truly great search engine offering, but indexing has to be at the core of this offering, at least for me. If users realized this is just Bing results with a few enhancements and additional result sources mixed in, they might not be as willing to pay for what they could technically get for free elsewhere, albeit with significantly less privacy.
That said I wish Kagi all the luck in the world. As the original dev who planned out and built the initial backend implementation in Crystal and put together their early engineering team, I can at least say they are building on rock-solid, very fast and privacy-oriented foundations, and this is a truly web-scale product in terms of the infrastructure design.
Interesting, I actually think that the indexing and the infrastructure part of building a search engine is much easier now than it was in the early 2000s. It's the ranking that's the hard part. PageRank helped put Google on the map and they've battled the hordes of low-ethics SEO practitioners ever since. I think it's generally agreed that PageRank wouldn't cut it today.
I tried Kagi as the default search engine for a few days and realized just how many little things Google does that I prefer, but never noticed until now. Not strictly related to search results, but things like converting units (Kagi has this, but it didn't always work), getting a time in a timezone (same), shopping results, the QA answers, etc. It's a ton of work to implement all that. For now I switched back, but I'll keep an eye out.
> converting units (Kagi has this, but it didn't always work), getting a time in a timezone (same), shopping results, the QA answers, etc.
Honestly I’d be happy with a search engine that does search well, and use dedicated tools for the other widgets.
I think browsers could do a better work at using the omnibar as a command line tool to cover these cases. For instance I’d really wish it would be easier to map incantations like “/mtg intl” and have a predefined action executed. Bookmark keywords are already half the way there.
> Interesting, I actually think that the indexing and the infrastructure part of building a search engine is much easier now than it was in the early 2000s
All the more reason for any serious competitor to try to do it themselves, yet no one does.
My problem with google is that it often tries to figure out my search intent and then return results accordingly. the attempt might be commendable but for many types of searches they fail miserably. when they misread my search intent, the results are completely awful.
and to make matters worse, 95% or more of the web isn't even being shown to users and so I keep getting the same stupid search results: which is fine for many cases but not all cases. sometimes I need more than just another post by NBC.
I will say, google search is very good at technical searches, that's for sure. But, for example, they totally suck if you're looking information or doing research on financial matters or economics.
>I will say, google search is very good at technical searches, that's for sure.
Unless your technical search, for example, contains jargon with a similar spelling to or other sense synonymous with a common term that Google will helpfully also consider to match your query. I think it's actually rather mediocre for technical searches now that they've gone so deep into stemming and such and lessened the ability to use operators to refine queries. And, of course, the lack of support for symbols makes many technical queries extremely difficult.
I have a solution for the google problem, but don't have time to pursue this myself. So anyone interested, steal my idea! (Or bash it in the replies, it's HN after all :D)
Instead of pagerank we need peoplerank. Let's say we build a new elgooG search engine that you can customize by indicating which people you trust. For example I say to trust Paul Graham about the topics entrepreneur and startup. Then, authors like Paul can register a key on elgooG and sign their online articles with that key.
When I search topics related to startups, elgooG knows I trust Paul and will try to find articles from him or anyone that he trusts, or even 3rd grade connections.
It is very difficult to fool such a search engine with SEO optimization, since the user actually indicates which people to trust.
I'm pretyy confident it would be a good solution, but I'm afraid it's very hard to start up.
If you steal my idea, please consider giving me a 1% stake (or less) in your company ;D.
I sort of like where the idea is going, but I don't like that it just depends on trust. People would become search engine "influencers" that you follow, and would get paid to bless certain articles, as along as they exude an air of trustworthiness. So in a sense, instead of Google being the ad server, the people you follow are.
Maybe a combination of trust and reputation would work better, but you might still get groups of people vouching for each other's recommendations. I guess it's not an easy problem.
This algorithm would have an extremely bad discoverability an would be extremely susceptible to platformization. How would you find Melcher writing about car engine maintenance on Bent Piston forum when you yourself rank KingsEgg on Vroom forums as the authority on engine maintenance?
Which article do rank higher: one by Melcher that is highly ranked by large number of "random" individuals or the one recommended by KingsEgg?
> Glad to disagree, DuckDuckGo is good enough if not great.
For anything that isn’t the least bit obscure or technical. (Same with just about every search engine)
IIRC, someone on hn said that search engines are only good for things you already know are there, not for finding new information or different perspectives.
Google can be the next Google if they just stopped being evil for a second:
1. Let me ban domains like pinterest, quora, stackoverflow clones, stock image sites, etc without requiring a chrome extension.
2. Do what I ask it to do. Don't be too smart. Bring back the plus sign, minus sign, double quotes, tilde which have been deprecated over these years and stop polluting the results with what it thinks I want.
3. A new feature where I can search inside the top 100 search results. Where I can narrow down the search results using additional filters like I do on amazon searching for products. So i can say "5000mah -clickbank" in the top-100 search results to weed out spam and narrow my search accurately.
4. How about an image search that... actually returns the URL of the raw image?
Here's a dead simple use case that is just unnecessarily frustrating. As a user, I want a high res picture of a buffalo. So:
a. I do an image search for "buffalo" and the results page contains thumbnails of buffalos.
b. I click the first result, it's tagged with "nationalgeographic.com" so it's probably gonna be good. Instead of the image, I get another page, but with a slightly bigger picture than a thumbnail. When you over over it, it says "3,072 x 3,072" but the image itself is clearly not that resolution.
c. So I click that image, and it opens a new goddamn tab of nationalgeographic.com's web site, with another picture, still not the promised 3,072 x 3,072! WTF!? When I try scrolling down to look for the raw image somewhere I'm hit with an E-mail signup-wall. Good grief!
d. Little did I know, if at step b. I instead right-clicked on the image and selected "Open Image in a new tab", I'd have gotten the image I was looking for. Thanks, Google, for hiding the 99.9% use case that people want to do.
The actual user experience should have been:
a. I do an image search for "buffalo" and the results page contains thumbnails of buffalos next to clickable .jpg links. The End.
Why couldn't they litigate it? As far as I know it wasn't a court decision but a settlement, which means no legal precedent has been set and it wasn't determined whether their original behavior was actually illegal.
It's amazing to me that Pinterest hasn't been sued for this, despite having much more blatant examples of copyright infringement that don't link back to the source.
You can thank Getty images(1) for that :/ I personally think Getty and most stock image results are pure spam and google should have just ditched them altogether and kept the original search. Now google image search is also shit and full of stock photo spam:
Well, in the age of "creators" I'm sure there are countless of people who would be willing to post pictures of buffalo "for free" if they believed they had a chance of having their work actually make it to the top of the search results
For step d, why is that Google's fault? Your browser already has a means to view an individual image, why should Google reimplement something that a right-click can just already do?
I know your comment is tongue-in-cheek but I happily pay for Youtube premium and it's truly an amazing experience compared to the ad-ridden non-usable free Youtube. If Google created a similar Google premium where it had such features and no pesky ads I'd pay for it in a heartbeat.
yeah, but then your just encouraging them to make the free version a bad as is acceptable, like YouTube, and possibility create classes like on commercial airlines. you want the world divided into classes?
We already have classes, and some people do pay premium for better service and quality happily.
A world without that choice, where everyone's experience is equally bad ... how sad that would be. Just ask your average east-European soviet-era survivor.
I'm about to do this as well. Must be amazing experience. First when I stareted youtube on my FireStick it was one commercial 5 seconds. Then 2 comms 5 sec. Then 2 each 1:30 seconds, but I can "skip ads" after 5 sec. Now its 2, sometimes 3, each 2:50 (I seen ad for some Christian church had 28 minutes!) and sometimes no "skip ads" button. The YT+ cost $18 per month... alot, until you realize huge freedom of not wasting 15 to 30 minutes a day when you want to do some research and watch some science-focused videos.
An easy way to personally blacklist (and a bare minimum on/off switch if you need to toggle it) would make me so damn happy.
Why should I be forced to look at a page full of purple links when searching for some specific programming topic? It feels like there's a bunch of potentially-fake developer blogs that copy/paste from Azure documentation, without adding anything novel.
I always thought Facebook getting into search could be big since they have really good engagement data based on what people post and like but I was just listening to TWIST (https://youtu.be/KfibbbLeT3c?t=4014) and they made a great point about the potential of Apple Spotlight Search being a way for Apple to expand into search and advertising. If Apple ends up turning spotlight into a proper search engine across web and apps it could really take a decent portion of the market from Google and FB, especially since all of the premium users are already in the Apple ecosystem.
The dream search engine would always be a click/swipe away and have access to all of your personal data/documents and the web, if Apple wants it CMD+SPACE could really be that.
The perfect search engine would be powered by humans with a search engine. Google is fantastic but there is no human understanding of the needs and wants that can be conveyed through series of keywords.
For example when I type "best humidifiers" I want a suggestion that is ideally based on accurate anecdotes of people who've experienced a brand in a variety of context and edge cases.
What google gives me is a cnet article listing Top 10 in 20XX. This is why I add reddit to the end of the search query now.
There was a startup called ChaCha that was a real live human answering every query in real time. They would chat with you and find you exactly what you were looking for. In the beginning it was amazing, but then they couldn’t figure out how to scale and quality went down.
> For this new generation, privacy is necessary, and invasive ads are not an option.
This is a red flag to me. I would like if this were true for a next gen anything, but in my experience truly next gen experiences - the kind that spread like wildfire and displace incumbents - have no reason to make promises about privacy or ads. Why would they, if they have a product that consumers want to use?
Search is the wrong way to look at it. It needs to answer questions, like an oracle.
Anyway, Google is getting less relevant because the technology is getting better than "good enough", and any additional tech that Google adds is not really all that useful. It's like PCs. You don't need a faster one because your old one can do word processing just fine.
This depends on user preference and query. Some users want answers to a question, other users genuinely want to find websites.
If I search for "what's the weather now" I probably want to get an answer "like an oracle" as you say. But if I search for "riaa vs napster" I don't want an "oracle answer", I want a list of several websites so that I can learn more about the case. I'd like a search result for court documents, but also commentary from news websites and blogs, and maybe wikipedia. I want to open those tabs and come out the other way with more information, but there was no "oracle answer" to be provided.
An oracle is how Google sees itself. However, their quest to be an oracle has come at the expense of losing their edge at searching the web. So now there is an opening for another service to be better at search than Google.
Nitpicking here.. but the title and article are presumptuous are out-of-touch in 2022. Google is primarily an advertising company. Gmail, YouTube, Android, Mobile Apps, Google Home, Maps, devices, Search App all synergistically funnel users towards advertisers. G-Cloud and G-Pay may be among the few exceptions to that but also constitute Google. Point being: you cannot be the Next Google by building a better search engine, just like you can't be the Next Microsoft by building an email system better than Outlook or a desktop OS better than Windows.
Some days I wonder what would stop me writing my own web crawler and building my own damn search engine from scratch. I can't ever come up with a hard reason it wouldnt work in 2022, other than the dynamic javascript blogspam that may be more challenging to trawl for meaningful content these days. Perhaps this is a win-win: The shit I can't crawl I wouldnt want to read anyways.
Compete with google? Who gives a shit. I just want to be able to hit a full text index that points back to URLs. I don't need instant PhD-tier answers for life questions.
(Disclaimer: I work for a big company which makes a browser, an OS for smartphones and a search engine. This is my own opinion, and does not reflect my employer, and so on, and so on…)
> Some days I wonder what would stop me writing my own web crawler and building my own damn search engine from scratch.
A few things:
* Storing the index. For example if I search for "Sushi", google says there is 1.3 Billion results. That's already 19.3 GiB just to store the relationship between "sushi" and its results. (If you use UUIDs) And you need to do that for most words in the english dictionnary. And I'm assuming you're not even storing what you crawl for debugging/re-indexing.
* Website admins might throttle you. For example, discourse — a website which runs most discussion for Rust, Ocaml, and many other projects — blocked the bingbot at some point[1] due to its crawling aggressivity. Cloudflare, which is the front-facing caching service for almost half of the internet now, is notoriously anti-crawling. (It will quickly display captchas)
* grep over the web will yield you bad results. (most of the time) You want a search engine which at least groups words semantically. If I search for horses, I'm usually also interested in mares and ponies.
Let's be honest here, If we remove ads, and blocked all SEO spam domains, Google results are good! I've never seen any new search engines including the new ones, even close to the quality of Google results.
In my view Google could possibly be the next Google. They're leading in AI right now and if they can get the magic we've seen in some of their papers into products like Search or Google Assistant it would be major moment.
People aren't complaining about Google because of a lack of AI, but because of product decisions they're intentionally making, presumably that increase their profits, but many of us would be overjoyed if they simply rolled back to their 2012 algorithm.
Pathways was the first time when Google got to a level where they could collapse multiple searches into one. They needed working chain-of-thought prompting to make deep learning useful for researching solutions for problems.
Why is this always a given? Yes, privacy is good, but honestly, I find what I'm looking for in the first few links about 99% of the time with Google, because of the lack of privacy. They have 20 years of search history on me, including maps searches. They know where I live and where I go and what I like and what I buy, and they can read all my email.
And I get better search results because of it.
If I search for [haircut], I get the cheap places near me, because Google knows I'm cheap when it comes to haircuts, because they've seen where I've gone before. To get that on an anonymous search engine, I'd have to search for [cheap haircut near $home_address], so now I've had to type extra words and I've given up my privacy anyway, just with extra effort.
I'm not the biggest fan of Google knowing that much about me, but I also know it gets me great results, and that's a tradeoff I'm willing to make (and lots of other people are too).
The project also adds some explanation on the role of each search engine, also the page has been auto translated into several languages.
Ideally this directory/thesaurus should help you with finding an appropriated domain specific search engine for a given task, without the need of snooping on the part of the search engine.
[1] of course they also need the snooping for targeted advertising, but that's a different story!
None of the examples explained here impress me (with maybe the exception of Goggles by brave search).
- The first example is not much more than Google with a styling chrome extension
- The second doesnt bring anything more than Google when you are searching about webcontent. Some ideas on filters are interesting but this is not user friendly
- You is powered by AI, so it is basicaly giving not what you want except when you ask for the general case (the case Google works pretty well for)
- Andi is a chatbot. Which is basically Google + only the first answer
Brave Search Goggles, (what most people call collections, what Pinterest calls a board, what Twitter call List) is for me what is going to be the real innovation of search Engine. They are a way to bring collaboration to enrich content, but nobody has cracked the UX to make it work yet
Did you try Andi search? It returns full search results alongside the answer and has full preview content for many of the websites, and the content is from the websites not Google or Bing's snippets. It's completely different to the way you described it.
My biggest problem with it is that the alpha version still has errors and is weak for searches like local businesses. Also, big problem is I'm in Australia and everything is in imperial units. Having said that, however, I've been using it since it became available a few weeks ago and it has already improved a surprising amount. And the question answering if you ask specific-enough questions can blow your mind.
It feels like the engine understand well the query, but for test searches it didnt show better results than Google, it actually gave me almost the same answers. So if they use their own index thats pretty good. But not the Next Google
You might be missing the point. Try a difficult question instead of a regular web search. Something like this. "Why did Twitter add Elon Musk to its board of directors, and what impact did that have on twitter's share price?"
Then click on "View in Reader" on the results. This is not like Google dude.
People on the Discord are trying crazy questions like that and reporting what it does good or bad on. It screws some up but it is cool. It finds stuff Google doesn't.
I tested "What impact adding Elon Musk to Twitter board of directors had on twitter's share price" and I agree with you the result is much better that when you search the same thing on Google.
To be honest that's impressive for complex queries
while it is the case that we're doing curation internally at the moment, we fully expect to open that up in some way as we scale; that will likely be after our next release to (1) fix our really awful mobile UX and (2) to deploy some version of our time filter, which is a core part of our thesis about easily tracking topics over time, e.g., this thread with some March Madness examples, https://twitter.com/HeyBreezeThat/status/1511397583209570311
None of these seem like the next Google. Too complicated.
The earlier Google, when Google was at its best in both performance and results, was incredibly simple. It indexed the entire web. It returned keyword matched results based on relevancy. There was no customization required. No editorialized results. No Big Brothering the user against thoughtcrime or wrongthink. No promotion of The Current Thing. Just information, whether good or bad, true or false, as a search engine should be.
Something that can replicate that will have a good chance of success, but I think the longstanding challenge to both Google and competitors is how to rank material in an increasingly siloed and ideological web filled with agendas, astroturfing, SEO optimization, and spam.
>But right after that, we’re greeted by an “Interesting Finds” section, which has a fun blog post by Derek Sivers, an article filled with stories of Steve Jobs in Japan, and some other cool things you can’t find on Google.
I’ve tried with Neeva, and to be honest, even put aside the potential privacy problems, it just doesn’t work. The quality of search results is notably poorer than Google. I used it for a dozen of PyTorch & Numpy related queries, and a dozen more on convex optimization maths. I often find that it can’t find the right answer for me. It’s really a pity because I quite like it’s feature to search information from connected accounts, in my case I gave it permission to my Notion page used as archive storage. That went pretty well tho.
It was pretty good a few months ago, but I've noticed more and more SEO spam in the results.
They also started using dark patterns in the UI. For instance, some search results will bring up an obnoxious popup asking to enable location services. It blocks off a good chunk of the visible screen
This is on a desktop machine without GPS. And it sometimes pops up for search terms that wouldn't have a location associated with it.
And you can't disable it.
I'll probably take a look at Kagi when I get an invite.
The problem with so many customization options is that it creates friction to users. We need something that just works out of the box. Just like Google did when it launched.
People don‘t understand that Google is in a game of user data: you can only win this game if you have the largest market share, plus a plethora of additional data sources like ads. Microsoft invested 10s of billions into bing to get to the level it is at. But more clicks and user behavior data means you would win against bing.
The only company that could theoretically build a web search engine due to the amount of data they have is Meta.
The problem with going up against Google isn't competing with features, or even relevance. No one cares about features beyond basic relevance, and relevance isn't enough to get people to switch.
You have to provide an excuse, or perhaps an experience, that gets people to come back. Just one niche, is all you need.
You're not competing with Google the search engine, or even Google company. You're competing against google the verb.
The problem with trying to replace Google is that you have to..
1. Do (nearly) everything Google does as well as them AND
2. fix Google's weak spots (usually in subjective queries) AND
3. offer something new and exciting.
It's hard to do (1) well given Google's data moat and how good that makes them at head queries. It's rare that people stick with one of these alternatives because they struggle to be as good in the majority of simple queries.
We built a few such search engines before choosing a different route — we made something that augments Google (or Amazon, or whatever other search engine you use) and improves it in areas that it's weak in.
It's one approach, but we're trying to be more non zero-sum about it, and consider that the search engine doesn't have to be replaced 1:1 with a better search engine.
I dont think that's true. To replace Google, you need genuine sources (Githubs and Wikipedias) and non seo content farm sources. Just these 2 would be enough to replace Google.
One thing that sucks is that when I tried googling for NeevaScope, Google instead shows results for NervoScope, whatever the heck that is. I'm sure this was unintentional but it still feels anti-competitive. Anyways, here's NeevaScope:
> DuckDuckGo and Bing are not true alternatives – they’re just worse versions of Google.
Bing - maybe. DDG - it used to be much more of an alternative, now a little less, with their decision to actively censor their content. Still, they - supposedly - don't collect and store information about you. That's the sense in which they're an alternative. And in Google, there is massive, manipulation of results with all sort of commercial biases. In DDG we don't know that that is the case.
> For this new generation, privacy is necessary, and invasive ads are not an option.
Hmm. I notice they didn't didn't say search results manipulation is not an option.
> Why should everyone have the same search experience?
As far as search results are concerned - so that you don't get to play games with what's more and less visible on the Internet? For the sake of fairness?
> We all have our own preferences about how things should look and work.
Oh, so you want to _flaunt_ how you'll manipulate people's search results to fit your interests? I'm sorry, I mean "the default preferences"?
I'd like my search engine to be not too intelligent. Sometimes, humans suppose to know what they are meaning.
So
- optional verbatim and prioritized terms (like Google once did it)
- optional logical operators (and, or)
- a separate input box for context (either, specific terms like "product", "shopping", "legal", or some assorted terms to indicate a field of meaning and/or associated concepts, time span, etc)
It's the latter, where the search engine may shine and users may evoke its sophisticated intelligence, but, please, don't reinterpret the principal search terms to concepts that maybe just related. I do understand that this may not be for everyone, nor may it be the sole interface, but there should be a niche for this.
Example, wanting to find something about the early press coverage of System/360:
I just think that a separation of intends may be suitable. One is the actual search term (optionally verbatim and using logical conjunctions), the other provides loose contextual information and/or categories specifying the source set. These are by no means the same.
(Compare the example given above. I don’t think that you can do a search like this with Google today. This should also help with issues like newer content masking established, older one.)
Edit: As it is, search engines return a quite diffuse field of information, which may not what you want, when you're looking for specific information. Some of this is related to the confusion of search terms and categorial and/or contextual information, while its also due to the eagerness to return any information at all (and as current as possible). Sometimes, this may be even worse than returning nothing. Also, this may lead to an eventual erosion of information in the long term.
For centuries, human knowledge (at least, in Western cultures) has been organized in terms of content and categories. In a classic library, a research like the one mentioned above would have been easy. As it is, this kind of structured concept doesn't work with search engines anymore, or, at least, not too well.
> DuckDuckGo and Bing are not true alternatives – they’re just worse versions of Google.
I disagree. I use Bing for about half of my searching because Google simply struggles with things like exact phrase matching in many cases. Where Google falls off, Bing tends to succeed. Though Bing also sometimes lags behind, where Google tends to do better, hence my split usage.
My primary want from a search engine is to not remove search terms I explicitly typed in and to point me to search results that don't have the end goal of wanting me to buy something, unless that's what searched for. It boggles my mind that search engines today assume I didn't actually mean the search terms I typed in.
Long story short of Google's rise and "fall" in quality:
Altavista was dominant internet search engine in the 1990s but search quality was bad compared to what could've be done.
Google figured out machine learning was the key to success e.g. PageRank.
Google got incorporated in 1998.
Good leadership(Larry, Sergey and Schmidt) and vivid vision propelled Google to 90% of market share in search.
World Wide Web was drastically smaller two decades and one decade ago and Google was still able to "handle" it with already established machine learning algorithms.
Fast forward to today World Wide Web is zettabytes big and not even Google can rank it "reasonably" and efficiently. Google doesn't care that much about search quality anymore because they have 90% market share in search, billions flowing in ad revenue plus 99 other projects they need to manage and develop.
Not an internationally good name. The io domain is ethicly questionable (although they are of course not alone using it).
DKB is a German bank. Shouldn't be unknown in expat circles because it used to be close to the only one accepting customers living abroad and actively marketing towards them. Not sure whether they still do so, but at least they haven't made any attempts to get rid of me (knocking on wood).
Offtopic background: International banking (for small customers) is a nightmare because of compliance issues even within the EU which is supposed to he a single market. Of course as soon as you have the US in the picture it gets completely hopeless (because of crazy rules for their own citizens and them acting world police, i.e. requesting others than their own citizens and businesses to comply).
I've been using you.com for a couple of months now and I think it has a lot of potential for one reason: choice. You can choose to be in private mode where they don't track you or you can choose to see some sites higher than others. Feels more customizable and the layout feels new
If there is any next Google (or Google Search competitor) it would be, in my mind, something like Wolfram Alpha. I can search for things like today + 160 days and get much more rich results (as a simple example). Google does have some features like this already such as weather/climate, calculator, translate, etc. This area however is much weaker than I would like it to be. Wolfram can tell me how long it will take me to get a sunburn in my current location, and all kinds of stuff like that. Virtual assistants like Siri and "OK Google" might be heading in this direction, but why can't I ask "Ok Google, I will take a photo, then can you compress it and send it as an email attachment to Dave from work." Or something along those lines...
I disagree. If google served me with what I search for within the top 5 results in every kind of context, I'd be willing to pay for it $30 / month, problem solved.
No one needs filters, backround colors and all that useless fluff - that doesn't really hit the nail on the head. I just want to get a useful information quickly. Google is arguably unbeatable when it comes to dev related queries, but a complete trash with pretty much anything else. Now, the problem is with all of these appalling websites filled with ads, pop ups, hidden agendas of selling me something etc. That's on google because their business model depends on serving these results.
A new kind of search engine does not run on ad revenue but on value add for the user in exchange for their subscription fee.
> You can change everything, from surface level appearances, to the final ranking of results.
People don’t want this. My first boss told me he would bet on regular TV if only streaming was left. He didn’t account for recommendation algorithms, so it’s easy to prove him wrong. But the takeaway here is that people prefer things working for them out of the box. That’s the secret to Apple’s or Google’s success: It just works.
Just works can mean that you’ll get “some” search results or a TV show that matches your preferences.
Almost nobody would like to put effort into customizing their experience.
Google provides “good” search results for a majority of users. From a business perspective it would be a wrong decision to enable customization for a neglectible fraction of users if the vast majority just want to “consume”.
Next Google isn’t going to be index based search engine. Imagine a massive distributed neutral model which has digested billions of pages. The query isn’t going to be keyword-ish like “kids cough medicine” but rather question with all details like “what is least problematic over the counter good cough medicine for a 9 year old with asthma”. The results aren’t going to be bunch of links but rather aggregation, summary and organized composed information from various sources with citations. Imagine if a human had read all the pages and you asked him the same question.
I would also predict search to be much more compute intensive and therefore more expensive. The ad model is likely not going to be sufficient to pay for compute resources.
SANCTION is an experimental search engine powered by the popular experimental search engine Million Short – an experiment within an experiment. (This is alpha and buggy!) With SANCTION you can choose to remove results from any Country you wish, for whatever reason.
Result removal is based on both the ccTLD and geographic location of where the page is hosted (a 1st degree connection). We have also provided the ability to remove results based on the ccTLD and location the scripts a particular page includes via script tags (a 2nd degree connection).
"The next Google can’t just be an input box that spits out links"
I think one strong contributing factor to Google's success is its simplicity. All the listed competitors add a lot of complexity imho. While all the customization buttons, knobs, lenses, meta crawling, code generation features etc might add some value for the advanced and technically skilled user, it provides rather little value for the average user who just wants to look up a cooking recipe.
So maybe when searching for "the next Google" the interesting question is not "what search features can be added", but "how can search even be simpler than using Google".
Actually, just taking the ads out of the experience can make for a simpler and better search experience. I think the google founders knew this too (https://www.reddit.com/r/degoogle/comments/rzr2n3/the_founde...). They just couldn't hold back the avalanche of revenue that search ads yields.
I work for Neeva, and this is a big part of why I left Google to join Neeva. There has to be a better experience, and it doesn't start from another business that works just like Google. It has to be a different kind of business. Neeva does not make money from showing you ads, so it can provide a different search experience... a simpler search experience, like the original google even, but it can go further...
With the Neeva app for example as you start typing in the URL bar, it will take your input as search suggestions (just as any other browser + search engine would) but instead of just showing you completed search suggestion, Neeva will show you the results from running those searches inline. The idea being that maybe those results will be helpful to you and make it so you don't even need to go to the search results page. You can just take the result right there from the URL bar suggestions drop down. Saves you time. Simpler.
Stuff like that. There's a swim lane of innovation and ideas on how searching and browsing can be better that is just really hard for Google to build, even though many of these ideas are thought of inside the walls of Google. They just can't ship them if they are stuck being beholden to their search ads model.
Another great example... ever wonder why Google isn't working to make it so Chrome doesn't have a million tabs at the top of your browser? It gets to the point where it is hard to get back to what you were doing. Me, I just end up closing the tabs, declaring tab bankruptcy. Google is okay with that because it means I have to search again. The Chrome team wants to fix this but it is hard to do so as it would result in people searching less often!
Again, just means there is opportunity for a simpler better experience to be had and Google won't be the ones creating it.
To be fair to Google, they do have the I’m feeling lucky button, which will take you directly to Amazon (example from the article) rather than showing you links.
Exactly. Notice how "I'm feeling lucky" is only on the home page and not part of the search experience when using Chrome or any modern browser where you search from the URL bar? Wonder if that is intention? Not a wonder at all.
The "I'm feeling lucky" button would never be added to Google if it didn't already exist. It was grandfathered in.
I think they each capture different markets. Google is simple because for most people it is “good enough”. Google isn’t going to spend a ton of time developing new features for the (large) minority that find it inadequate.
Google also has a few good things going for them:
- it’s the standard when it comes to search and is setup by default on just about every platform
- People use the brand name itself when talking about looking stuff up (no one says “just duck it”)
- They control the ad market, the standards for what a “optimized” site is, and the web standards themselves.
The future search engine will look for content that is produced by humans . We don't need more info for the sake of having info. we need info that we can assume has been produced by an actual sentient being.
Since it's difficult to determine whether a human produced a particular piece of content, a good alternative would be to determine whether there's any incentive for a bot to produce such content, and target that instead. Ads, analytics, affiliate links, etc.
Sure, you could still make a bot that auto-generates blogspam without the aforementioned things and rank as high as before, but what's the point if you removed the very things that get you paid?
There's plenty of things out there that are trivially spammable but outside of isolated cases (where the objective is to cause damage/annoyance rather than profit) nobody is spamming them because there's no profit to be made by doing so.
I'm longing for categorized YouTube site... When doing home project I can navigate with tabs: home -> security -> outside cam installation -> softfit mounted -> how to strip softfit / how to run cable / how to drill and patch cables hole etc.
Could be Wikipedia-style cultivated if enough people know and care.
> No, most people have never given a single thought to this.
Probably because they're stuck in a local maximum with no idea how much better it can be. They may not even be aware that there are sites out there that have quality non-commercial content if all they've been given for the last decade is spam to the point where spam has been normalized.
putting aside the (idiotic) negative framing of third-world countries, that are called developing countries nowadays anyway, they also represent 3/4 of the World's population.
Most of these features have been tried, and abandoned, by Google itself. It's possible Google stopped doing them because they hurt its ad business, but it's also possible (and likely) they confused users or were difficult to maintain and not actually used. (What's a "non-commercial website"? And what happens when it suddenly becomes commercial? And when there are millions of them? etc.)
I predict that in 10 years Google Search will still be dominant, and we'll still be complaining about it.
If memory serves me correctly Google once had a desktop widget with search like neeva. I could search email, files, Web, all at once from task bar. Neat. Maybe I dreamt that.
Kagi requires for an account to be registered and will charge a monthly rate when it's out of beta, so I don't really think that it'll be a threat to Google.
> We don't want to kill Google :) Google serves a purpose in the world and it did help enable the modern society to exist, with all its marvels and flaws. Heck, it even enables Kagi to exist!
>
> Think of Kagi as a small, premium brand, providing a very different, tailor-made, search experience.
Happy to answer any questions you all might have about Neeva. I left Google to join Neeva about a year ago. Got inspired by the opportunity to make a better product. AMA :)
> We need new thinking to create something much better than what came before. In the last few years, different groups of people came to the same conclusion, and started working on the next generation of [foo]. For this new generation, [bar] is necessary, and [baz] are not an option. But that’s where the commonalities end. Beyond that, they’ve all [zoz] in very different directions.
This pitch voice makes me want to gouge my fucking eyes out.
The next Google should have less AI and be more deterministic. At least it should have a mode where it searches exactly what I specify instead of searching what it thinks I want. Also give me options to filter out certain domains. Again, let me tell the search engine what I want instead of the engine telling me what I may want.
I feel a lot of modern software is becoming very authoritarian. “We know better than you what you need”.
Google used to let you block domains a long time ago. I'm not sure why that was removed, but I have a feeling it had something to do with giving the expectation that you can block ads from a specific domain.
I wonder how feasible would it be to index everything locally? I mean how far are we from having the cpu and storage space to allow a search within our computers?
Do we really need to store everything ? Maybe a local "ai magic" could select the specific information we regularly search for and sync from an open public index.
In the case we need results it could sync and/or request those bits we are not indexing locally.
20GB, which is tiny these days. Fits on a $10 USB key.
I think with some of that AI magic we'll soon be in a situation where most of the traditional (non-proprietary) human knowledge you ever need (text, images, 3d models of machines, anatomy, etc.) will be immediately accessible for free via an open-source knowledge engine.
With a commodity network connection, that knowledge engine can keep itself apprised of the post-traditional/proprietary knowledge by doing all the drudgerous work of reading through the world of Ads to get to today's news, etc..
Mate that with more AI magic to make it easy to conversationally interact with the engine and you get an agent who can represent you online and provide a distilled interface of online for you. Your digital twin or avatar.
I hope the open source community can come up with a solution for this.
Maybe there are already projects with this goal, if not someone should start it.
It will improve our collective knowledge and at times like this were disinformation is really high we need technology/tools to reduce it
Privacy no need to explain why this is important
Resilience if google/any other search engine goes down it usually mean chaos
You have total control of what is shown to you so no one can manipulate
what is shown to you(at least not that easily) assuming it is open sourced
of course
I guess there are cons to this approach too but it looks/sounds really good :)
-$5 or less a month
-simple web interface
-accurate results
-privacy oriented
-NO ADS
That's all I want from a search engine. I don't care about shopping or maps or any of the other 'features' all of these alternative are trying to implement. I want a gnu style search engine. I want something that "does one thing, and does it well," and I want that one thing to be 'search.'
I too like many others feel that Google results have gone really bad in the last year which may be partially due to web pollution but the main issue is that I feel like google's search totally ignores my requests, a lot of times it even ignores exact matches using quotes.
If only we could get some sql-like powers in that simple input box. Maybe I'm not their target audience...who knows.
Thanks for the article, I've learnt new search engines despite spending a couple of years recently in web scale search. I think you may consider https://usearch.com/ as another dimension in web scale search, where query log is learnt from the data, making it quite unique.
I personally think this new search engine sucks. There's 1000 good reasons to create a new search engine to fight Google but they don't get it, they don't have vision and their features just sucks.
"[...]We do not log or associate searches with an account.[...]" why would I trust that and why would a create an account in the first place ?
I don't use it as my primary search engine as I often want images and news related to the search. But for any search that is going to get flooded with SEO type stuff it can get rid of a lot of the crud very easily.
Neeva's similarity search is very useful. It takes a page and creates a query from it (oversimplified, actually a document based k-nearest-neighbors) then executes the query. This feature used to be in Google but they removed it, not used enough I guess. Neeva has very limited ways to use this.
> The next Google can’t just be an input box that spits out links.
I have a fundamental disagreement with this. The best search engine is still an input box that spits out links based on what was typed. This allows the user to express what they're looking for, their query, in most natural terms.
I think the conundrum is people want a computer to read their minds to make things easier for them, but when it does, it creeps them out. The solution, IMHO, is less technical and more psychological. If I was trying to replace FAANG, that is the thing I would consider how to address.
I don't think displaying data in different fashions is enough to come even remotely close to dethroning Google. It's their data, how much of it they have, and the algorithm that sits in between the search input box and the delivering you results that makes them the best.
I look at statmuse.com and wikipedia: sites that provide DSS (domain specific search) for information as the anti-Googles. No one will replace Google, but a site that focuses on a narrow domain is extremely useful and is the one thing that pulls me away from Google.
Google won because they started early and had a right algorithm. They then had the scale to grow with the internet. It would be very hard to start a new general-purpose search engine now (capital, monopoly, tech)
Nobody seems to have The Next Big Idea for a better search engine yet.
I think the bigger reason why search isn’t getting much better is that incremental technical improvements are being offset by Google trading their dominance for greater shareholder value. A safe business can take away or buy-and-bury features that are better for the customer but worse for the business’s top line. Things like the Power Search API, or search that integrates with other platforms that hurt Alphabet owned businesses.
Traditionally big innovation breaks some socio-political constraint on market participation. I don’t see how AI will do that yet.
NLP generative models that you can interrogate with natural language prompts, and which generate coherent responses together with citations for sources, is definitely the future of search.
I don't know anything about AI or the relationship between market participation and innovation. All I know is that LLMs are a useful tool that is already displacing search in many parts of my life.
I see. These sound like two different products, Google Search vs Google Scholar. I can imagine AI + natural language processing is dramatically changing search performance of dense-detailed text.
I mostly replaced my use of Google Scholar with Elicit. You ask questions, and Elicit gives answers, with citations. Elicit is powered by GPT-3 and it is amazing. Go try it.
> Everybody has different preferences of how they want a search engine to look and feel.
I will never ever ever ever spend time going into the settings of a search engine to customize my search experience. Not when Google does a good enough job for most of my tasks.
I think the replacement of Google can't come sooner. Google invades our privacy and keeps users hostages for more money. Thanks to HN I am aware now how bad Google is. I always recommend people to switch to Amazon, Apple or MS. Google is EVIL.
You lost me at "Duckduckgo is a worse version of google"
Privacy aside, I find the results much more helpful on average these days, and weeding past sponsored links is enough of a hassle within google search results to consider it a UX QOL downgrade
Developing search engines and personal assistants are the way to go. Too much information these days to digest , what a better job for a computer that helps us with the literacy aspects of it.
The search engine I want is the one that actually searches for the text I input into it, instead of "trying to find the most relevant" (aka "whatever is cached at the moment)
The next Google should be powered by AI/ML algorithms to go around SEO tricks. If lots of people pick a result from page 16 that result should be on the front page.
I find ddg is an improvement on google because it’s more like google 10 years ago plus a quick !g tosses it back to google for the occasional topic google does better.
At the rate we're going, google will eventually be forced to open its index to competitors just like how telecom companies were forced to allow competition.
Do you have any example queries? The only spam I normally see are the sites scraping Stack Overflow but they are never above the actual Stack Overflow page.
Google had it search nailed for a long time. I don't expect Kagi to replace it if it's asking questions like "what killer feature do you want from a search engine" (paraphrased)
I don't want any features. I want a search bar, a search button, and results that relate to my query based on more than simply matching a keyword to a query word, but based on less than whatever Google factors in these days and certainly not based on whether someone has paid to have their domain featured for a query. I have never clicked a paid promoted search link on Google, or any other service, and I never will. If you have to pay for your content to be displayed as matching a query, it probably shouldn't be. Content should speak for itself.
The one thing I 100% never want to see is a selection of results that are at all influenced by what lots of other people have clicked on. I want to see results that pertain to my query, regardless of whether they have attracted the interest of others who have searched for something the same/similar.
I also don't want to see results that are influenced by my previous searches, by my location, or by data gathered about my Google account unless I specifically want those to be factored into my search.
Features like ways to refine search like inurl: or intitle:? Yes, cool, have them available.
Also, I expect a search engine to load fast. I don't expect a ton of JavaScript to be required for an input box and a button, and I don't want doodles of the day and other junk. They're fun, sure, but at least let me turn that off because I'm a psychopath.
I don't think we'll see a true Google competitor. Would I pay for a search engine that was just as good but kept it simple and returned better results than the trash Google can come up with? Yeah, probably, but I think all that will happen is that Google itself will come up with Google Premium Search, run it for two years and then shut it down.
I don't think for one second think that search is easy, it most definitely isn't, and Google has a head start on its competitors -- but I do think that Google was more effective 10 years ago than it is now. I cannot however say whether that's down to Google overcomplicating their search algorithms or whether there's just x% more garbage out there, where x is some very large number.
Have tried Kagi for a while as a beta user. It generally works fine, but I am just reminded every time when I try to do a search via Private Browsing - and Kagi, a subscription service, will apparently ask me to login - that each of my search history will be able to be linked to an individual.
Also a beta user. I wonder if the final version you’ll be able to stay pseudoanonymous - pay with crypto, no email, etc (kinda like Mullvad).
Not defending them and also dislike this (but can’t think of a way involving payments which doesn’t do it). I do find their privacy policy quite clear and transparent - https://kagi.com/privacy
And also let’s not kid ourselves - I would guess this is still likely more privacy-forward than a Private Session on Google. Would be surprised if Google isn’t using every potential signal to fingeprint you and link back that session back to some other login.
nerds like nerd knobs... not sure customization has a mass market appeal. I also fundamentally disagree with the sentiment that the general public cares about privacy... though nerds do.
I want to like this, but it also feels like another way we'll all be siloed in our echo chambers. I mean I guess this ship has already sailed with Google microtargeting it's search engine users through ever-increasing surveillance capitalism tavtics, but I was kind of hoping that some day search engines would be at least one way to establish commonality among us. Alas, I know this ain't happening. And, as I'm writing this, not even sure I want that.
IMO the next Google needs to use AI to classify results as fact/fiction and supposition/exposition etc. It will identify bias and even classify it (religious, political, cultural etc). With the abundant sources of information (and misinformation) ever growing, the job of a search engine will be to inform you about the information you are viewing and alert you to key factors that may be coloring the information.
> The next Google can’t just be an input box that spits out links.
Hard disagree. I spent years of my life working on a "new way to search" (Graph Search at Facebook, ~2012) and the biggest takeaways for me from all of that time is:
1) people don't want to learn a new way to search, and they don't want to learn new tools, filters, lenses, whatever you want to call it. it must be a keyword-search based input because Google has trained billions to search (and to think) this way for 20+ years now. imo the next google shouldn't fight this behavior, but rather adopt it as part of the core strategy
2) ownership of the "search intent entrypoint" is key. people generally don't care what they are using to search, as long as it's easy and fast to use and actually works. google understands this and makes it almost impossible to bypass searching with google if you're on an Android phone. All of these alternative search engines have a huge barrier to entry not only on the technology and data side, but mainly on the entrypoint side. They can't reasonably expect hundreds of millions of the people to go through and manually update their default search engine on all of their devices. What about when I go to use Google Assistant? Google maps? All entrypoints that Google has used to lock up search intent. It's brilliant. I believe there are moves still available for a future Google assailant to overcome these barriers, but I haven't seen many that have directly touched on this point yet. new technical shifts may open up new entrypoints here (VR/AR?)
3) the search engine that has the lowest time between [query] -> [answer] for a user wins. any search engine that adds additional cruft through forcing more clicks, usage of advanced tooling (filters, lenses, etc), visualizations, etc will lose out because they are getting farther and farther from the core problem a user has - "find me x". this is the thing that is driving the hero/knowledge units on Google, further cementing their leadership
4) the one brightspot - verticalized search intent is a huge area of untapped potential in the search arena. Google is great at general search (and honestly also at a huge number of verticals that it's developed over time), but there are many verticals that google honestly sucks at. I do believe there is a potential entrypoint here for a "new google" if they can take on a vertical-by-vertical expansion strategy. Owning the intent entrypoint would still be as important as ever, but this play would look a lot like the early Yelp vertical expansion strategy from the early 2000s. It's anybody's guess what the actual right starting vertical and entrypoint match is though
tl;dr Google won't go down easy, and a Google killer may actually look incredibly similar to Google in it's end-state
I might be biased but I see myself depending on trusted sources - no SEO links, no spam! I want yelp for restaurants, stackoverflow for coding questions, reddit for opinions or allrecipes for recipes.
And that's by the way why Google is still successful as well. Because it literally still is a simple box where you put a question in and it gives you answers without needing to do anything else. The only way to beat that is to make it even better while not making it more complicated which is very hard to do.