Hacker Newsnew | past | comments | ask | show | jobs | submit | toddmorey's commentslogin

It feels like the US for years has operated under the assumption that homeostasis for the global economy would always be “designed in California, assembled in China.”

Like there was something in the American DNA that was lacking in China and innovation would always need to happen here.

But China it seems doesn’t need the US to produce great cars, devices, robotics, or AI. We absolutely need China to help us build all of the above.


The one major area they are still behind in is CPU tech, but they are hungry and thus moving quick.

Looking at Loongsons processors for instance. About 15 years ago they coudl barely compete with a Pentium 2. Now they are about 4-5 years behind Intel/AMD. Further behind on some more specific work loads (SSL decoding for example) Not great but that is a decent jump. The jumps between generations are pretty decent.

LA446 was a decent enough processor core but had an awful memory controller that held it back as soon as it needed to reach outside of cache. As such it was SLOW.

But they learned the lesson and now the LA664 almost entirely fixed that issue. I think a big part of performance issues is that they are working domestic 5 to 7nm processes, so a good 5-7 years behind.

They are launching the LA864 later this year and are touting some decent performance gains. That is just marketing so far but something to keep an eye on.

Considering that these chips are using their own ISA, own designs, domestic manufacturing and they aren't terrible is a big thing.

I suspect in the next 5 years they have the chance of completely closing the gap. But it can also go the other way that they end up stalling as smaller nodes get much more difficult to attain.


> Now they are about 4-5 years behind Intel/AMD. [..] the LA664 almost entirely fixed that issue. I think a big part of performance issues is that they are working domestic 5 to 7nm processes, so a good 5-7 years behind.

I'm not finding many benchmarks but looking at this https://chipsandcheese.com/p/loongson-3a6000-a-star-among-ch...

it looks like it's right around Zen 1 class performance. Which I hate to tell you is 10 years old already...


How much does corporate espionage help them?

Probably the same amount it helped the US in the late 1800s/1900s, a substantial amount.

Who knows but any 'answer' anyone could give is pure speculation.

You could be right! But I do see this claim come up every time Chinese tech comes up. It might be a valid concern but it might also just be folks attempts to try and undermine the technology gains of the nation.

The ISA they have developed with based off years of with with MIPS and RISC V, so it isn't entirely new but they are definitely pushing it forwards. I have no idea if any of their developments could be back ported down the RISC V.


There are 1.4 billion Chinese. When they got their education system up and running again by the late 1970s it was simply a matter of statistics.

Might be more far to say: they needed the US until they caught up. The massive straight up IP theft helps a lot here. Though theft might be too strong since a lot of companies knew what they were getting in to

> The massive straight up IP theft helps a lot here

I think this is vastly underestimating what "catching up" means. All my life, people have been saying "China copies". Now they are objectively better at many things (including robotics), and... well it seems that we cannot "just copy".

I saw western companies trying to "copy" superior Chinese technology, talking to brilliant engineers explaining how much they were learning by actually trying to copy.

The lesson I got from that is that China did not "copy"; they learned. And it took time, and now they are better. Now the western world has to learn from them, I guess.


Growing up moving around both conservative and liberal parts of the US, from middle school to college, I distinctly remember several US history classes where I was taught the exact same narrative about Samuel Slater. About how he was an American hero and the Father of the American Industrial Revolution because he memorized a bunch of industrial patent blueprints and brought them over to the US.

It got told as: the evil English made it illegal to even import blueprints for factory machinery, to keep the colonies in resource-extractive poverty, so they'd have to send raw materials overseas to get processed, then import the finished goods. (My other history teacher, the Anno / Dawn of Discovery video game series, also cemented this bit about resource extraction in my head at a young age.) But then thanks to heroic ingenuity and cunning, I was told, the US was able to outwit the colonizers and process its own raw materials, eventually gaining full economic, military, and political supremacy.

Sounds familiar.


It's ip theft when the Chinese do it but when it's the American copying on Chinese it's called learning.

Producing great products is a game at which every player wins, because sellers must find willing buyers. It only fails if one participant panics and jumps out of the window, or if a significant number of people are not participating (this is always the case when wealth inequality is involved).

China is out producing us at new scientists and engineers.

> The lesson I got from that is that China did not "copy"; they learned. And it took time, and now they are better. Now the western world has to learn from them, I guess.

And Apple played a huge role in teaching them. We should all thank Tim Cook and team for almost single handedly bootstrapping China 2.0, the China that runs circles around the west in terms of production and development.

Peter Zeihann really got it wrong in his latter books.


For those who don't like my comment I'd implore you to read [0].

[0] https://www.simonandschuster.com/books/Apple-in-China/Patric...


Ok, not my favorite narrative, but assume asymmetric application of intellectual property rights was a big factor. Wouldn't the US exploiting asymmetric labor wages, rights, and conditions be the even bigger story? It still feels like a short-sighted own goal. The US abandoned its ability to manufacture. Maybe dark factories and robotics can bring it back, but manufacturing supply chains are just so much more advanced in Asia than in the US.

> Wouldn't the US exploiting asymmetric labor wages, rights, and conditions be the even bigger story?

Yes, but "the US" is reductive. The exploitation wasn't done by the towns having their tentpole industries shipped overseas, it was done by the people shipping them overseas and pocketing the profit. US capital owners made a deal with the Chinese Communist Party that was good for both of them and bad for the US.


That's really well said.

The promise was always to get cheaper goods and services in the US, so long as the Chinese firms never competed. Guess what, they compete now.


And good for the people of China presumably.

At some point we can’t keep blaming IP theft for obvious innovation and investments being made by China.

We also can’t blame subsidy. All countries subsidize their industries.

This video on the auto industry covers a different industry but has a lot of the same rhymes as far as China’s strategy:

https://youtube.com/watch?v=UhhZu0ZHdw4

The gist of it is that China does the following:

1. Treats low margin industries like mining and utilities as areas to focus investment and come up with incremental improvements, making those available to all companies. The West, by contrast, allows private companies to handle those industries, who logically don’t bother investing in them since their investors consider those basic industries to be low-value segments of the production chain. But now we see those advantages in China where investments have been made (e.g., the best battery chemistries and mining/refining, the cheapest power (when was the last time your local utility company focused on reducing pricing?)).

2. Because all companies in China have access to the same excellent infrastructure, they must compete furiously on quality/features/price of their products.

3. China allows foreign competition so long as they operate in China (see: Tesla) further insisting that their domestic products be globally competitive and that foreign products sold in their country benefit their local ecosystem.


Lol it was not ip theft it was American and European companies building factories in China themselves teaching them how to manufacture use their cheap labour. Well they learned and as they were the dong the manufacturing got better at it. I believe the current aerospace industry which the US leads in is also result of IP theft from the British then out innovating them.

> I believe the current aerospace industry which the US leads in is also result of IP theft from the British then out innovating them.

Jet engines, proximity fuzes, radar, how to make a nuclear weapon, etc. are all examples of British / Commonwealth technology "gifted" or "traded" to the USofA during the WWII years in exchange for production.

So, not IP theft .. but absolutely foreign ideas taken in by the US and built upon.


HN hates non competition clauses in contracts unless it involves Chinese workers.

But I think we underestimate the Chinese diaspora. They had been running factories, shops and banks from Singapore to Suriname for generations and answered the call from the PRC to share that knowledge base.


The US committed massive IP theft in the 19th century when we industrialized.

Sure but I think what people are actually concerned with today is China copying a product and dumping cheaply back in the country it was taken from. That scale and speed is not what was happening in the 19th century.

I personally have little issue with countries doing that for domestic use (I hate using term "IP theft"), but to re-export so quickly you can't run a viable business in your own country is not fine.


As did the big AI providers.

I would appreciate some reading pointers about this.

> Samuel Slater ... known as the "Father of the American Industrial Revolution", a phrase coined by Andrew Jackson, and the "Father of the American Factory System". In the United Kingdom, he was called "Slater the Traitor" and "Sam the Slate" because he brought British textile technology to the United States, modifying it for American use.

> He learned of the American interest in developing similar machines, and he was also aware of British law against exporting the designs. He memorized as much as he could, and departed for New York City in 1789. Some people of Belper called him "Slater the Traitor", as they considered his move a betrayal of the town where many earned their living at Strutt's mills

https://en.wikipedia.org/wiki/Samuel_Slater#Early_life_and_e...



What? How is someone born in 1947 relevant to ip theft in the 19th century?

Because USAs military literally stole his IP? He had patents for GPS systems that US military took (by making his very expensive US lawyer making a silly ”mistake” and oops he lost against US companies and they suggested ”let’s forget about the money if you just hand us over that patent that the US military wants”

'usa ip theft 19th century' in your fav search engine

Well, I did, and to save others the time, the most relevant resource I found appears to be the book "Smuggler Nation: How Illicit Trade Made America” (2013) by Peter Andreas

IP theft may only be part of the story though. it’s a question of priorities. US optimizes for profit which can place limits reinvestment. China seems to optimize for ubiquity and dominance, and has the capital to throw at those goals. when you’re beholden to the shareholder/ceo/investor, you make concessions to stay within their will. when you’re beholden to the state, you do the same.

Talking about IP theft with a straight face in context of AI. lol. Not that kind of IP theft, that doesn’t count.

this is a great read on the whole dynamic incidentally https://americanaffairsjournal.org/2021/08/the-value-of-noth...

Wait until you hear about the history of US industrialization. This trope of 'they stole our ideas' needs to fade away, it's a coping mechanism based on the assumption of inherent superiority of American society rather than the natural wax and wane of civilizations due to varying structural factors.

This so much. You can also read up about when Germany sent industrial spies to Great Britain. And the first documented case of industrial spionage was against... China.

It plays this way: you're behind, you ignore IP rules. You're ahead: you create them to defend your newly-gained status.

Also please no moralizing here on IP when the entire OpenAI/Anthropic playbook has been "massive straight up IP theft". The irony.


>"IP theft"

Can we stop this crying baby already. Every country has stolen from the other. Did you really expect countries to settle on sewing closes and ship all profits to foreign companies for eternity? The IP is just an artificial concept that participants follow for so long as it benefits all parties.


Propaganda. We americans ate that shit up.

There's nothing special about anything we design in the US other than time and money commitment to create it. China did have some espionage of course going on, but the vast majority of shit isn't some secret. And with the US shitting on China with restrictions, we increasingly caused them to invest time and money into things they otherwise would have passively accepted as coming from the west. ASML sees the writing on the wall for themselves in particular.


It's both.

The US has generally resorted to propaganda rather than addressing the self-inflicted structural conditions responsible for the erosion of our dominance. China also conducted a broad, sustained, large-scale campaign of IP theft across almost every industry.

Obviously there is no natural law preventing China from innovating (We have treated political liberalism as a prerequisite to innovation in a way that was always partly self-congratulatory), but it's also obviously true that the speed of the gap closure is due in significant part to theft.

That doesn't change the fact that they are now a legitimate competitor who has gotten a lot of things right (and among these, some things that we get very wrong) and probably actually leads in some areas.


I like this take a lot and agree with it. The US for too long has been asleep at the wheel on many areas, power generation one of them. China with no doubt has conducted very deep and sustained espionage campaigns and even with LLMs there is enough evidence that most of the initial gains was training off of western models. Again no complaints here but I think it’s important to acknowledge both which can be true at the same time.

>"Again no complaints here but I think it’s important to acknowledge both which can be true at the same time."

and this acknowledgement will pay your bills


Huh?

> Like there was something in the American DNA that was lacking in China

In most Americans' eyes, unfortunately, there was. It was just known by the name "American Exceptionalism". Yes, it's nonsense, but unfortunately it is nonsense that has historically been used by most empires throughout history, and believed just as fervently by said empires' populi since it's one of the central elements of imperialism as a whole.


The US models are still better though, let's not get carried away. Ours are better, theirs are cheaper. That's how it's always been.

Downvoters are being silly. If you want to make a case for American Exceptionalism being a hoax, that's fine. But don't use deepseek 4 pro (which is at 100 ELO or so below top models) to make that case. You have stronger arguments elsewhere.

Capishe?


Downvoters aren't being silly, it's about you not considering the context of the discussion when writing your first reply. Deepseek wasn't mentioned once in this thread before your second post and AI was mentioned once in a list of different industries. Those should have been a clue to why your first post was downvoted. Basically, you wrote a non sequitur post and are surprised that it is being downvoted.

Um, take a break between bong hits to read the thread title pal: "DeepSeek makes the V4 Pro price discount permanent". I don't know what you thought we were discussing, puff the magic dragons perhaps? Lol.

Seriously though, take a hike if you can't be bothered to read things.


DS4 is open weights so it could even be run free in quantized forms, is 10x cheaper than Opus and performs basically as well in most real world tasks. No one cares about benchmarks. In practical terms, it’s obviously a better option in most cases.

You’re defining “better” is “absolute best at any cost” instead of the more balanced price/performance considerations consumers actually take, so you can declare America #1 again. In a practical sense DS4 is so much cheaper at similar quality that it’s better in most cases. If i can throw 10x the tokens at the same problem at slightly lower quality, i can probably do a better job.


ELO is an absolute rating. You could make a claim about some unknown GM being "better" than Magnus Carlsen because his appearance fee is cheaper, but obviously nobody would take you seriously.

There is a best model, and then there is what you can afford. Call that the "better value" or something if you must, but calling it the "better" model is clearly spreading a falsehood.


imagine if being liberal meant pretending weaker Chinese models are better than US models as a form of virtue signaling?

We failed you somehow, kids. I'm not sure where, but we failed you.


>Like there was something in the American DNA that was lacking in China and innovation would always need to happen here.

There is (was): attracting the best minds around the world to a free and stable society. Trump voters threw it all away because they couldn't stand non-whites coming to America and doing better than old stock Americans.


> attracting the best minds around the world to a free and stable society.

China is comprised of ~91.5% ethnically Chinese citizens. [0]

> Tump voters threw it all away because they couldn't stand non-whites coming to America and doing better than old stock Americans.

The U.S. is more diverse than it's ever been [1], and under Trump we're still below the deportations of Obama's terms.

Sounds like open-borders immigration was never necessary in the first place, given that we're being beat by a country with a similar demographic skew that we had like 80 years ago. Coincidentally, when we arguably had our best economic opportunities for citizens. Who'da thunk.

Clearly, the only solution to our fading relevance is opening the border again and importing 500 million more ""doctors and engineers"" all the while China is investing in their *actual* doctors and engineers, and has extremely strict immigration policies [2].

[0] https://en.wikipedia.org/wiki/List_of_ethnic_groups_in_China

[1] https://en.wikipedia.org/wiki/Historical_racial_and_ethnic_d...

[2] https://en.wikipedia.org/wiki/China#Population_policies


You're conflating Mexican border hoppers with skilled immigrants.

I'm absolutely opposed to illegal immigration and have a more extreme position on how to deal with it than most Americans.

What I'm irked by are Trump's attacks on legal immigration and the general worsening of the environment. ICE's kidnappings, the 100k H-1B fee, and the recent Green Card thing have deeply eroded America's attractiveness to legal immigrants.

I think when MAGA came after H-1Bs, it became pretty clear that it's not about law and order, it's just a race thing.

And if you want to go gloves off, I'll just say it: the main problem in America is that its 3 major ethnic groups are infected by anti-intellectualism and slothfulness, whereas the Chinese and various other cultures are not. The direct benefit from skilled immigration is so that we can increase the ratio of people who actually value education and hard work vs the failing old stock Americans whose broccoli-headed kids dream of becoming YouTube influencers instead of astronauts.


H1-Bs are the most egregious example, because they're 100% used as a way to undercut/replace American talent. The irony is that the typical border hopper is working jobs Americans don't want, for wages Americans wouldn't take, and they keep a low profile to avoid getting deported.

The desire to be influencers isn't as boneheaded as you think, in a future where AI is solving the hardest technical challenges, the ability to get attention and create community is the last frontier. Influencers and salesmen will be eating good when scientists and engineers are derelict.


> The U.S. is more diverse than it's ever been [1], and under Trump we're still below the deportations of Obama's terms.

Ethnic diversity is neither really here nor there in terms of the measurable needs that immigration fulfills. Immigration keeps economic and population growth rates trending up. Having high skilled immigration to bolster science and research is nice, but it's still mainly about the growth.

Yea, Obama deported lots of people, but even then we still had net positive migration. Now under Trump, we have net negative migration for the first time in decades. The very public terror campaign waged by the Trump admin was in part to deter immigration in the first place.

> Sounds like open-borders immigration was never necessary in the first place, given that we're being beat by a country with a similar demographic skew that we had like 80 years ago.

1) Economic growth is possible with stagnating/declining population levels if you overcome those deficits with commensurate increases in productivity per capita. Otherwise, you're cooked.

2) The US is actually far more productive per capita than China - in fact, the US is one of the best in the world, as far as that goes.

With those points in mind, we can begin to see why China has an easier time growing economically with little immigration. The US has a much harder time doing the same. We need more population, since it's just harder to squeeze more productivity out of our already very productive workforce.

Once China achieves similar productivity levels, they will need to rely more on growing the population.

We were actually on track to catch up to China's population levels in a few of decades (thanks to immigration). So unless China successfully pivoted to mass immigration or expansionism, the US was likely to remain dominant - easily so - for the foreseeable future.

That's why the MAGA anti-immigration push is so tragically stupid and suicidal (if it persists). They're killing America's golden goose.

As an aside: I wish the "open borders" canard would die. We've never had open-borders immigration in recent history. Definitely not since 9/11. Not even under Biden. Border laws were enforced. Biden has the same apprehension rate at the border as both Trump and Obama.


That's such a gross misrepresentation of reality.

First of all, the only group of immigrants targeted by the admin are those critical of certain middle eastern regime.

Republican racists mainly care about the immigrants that do not take their middle-class jobs anyways.

Anti-Indian hate is restricted to a minority of software engineers and anti-Chinese hate is virtually non-existent.

I do believe it is idiotic to have your universities full of Chinese, your manufacturing in China and, at the same time, treat China as a geopolitical enemy.


people might not wanna admit it because it feels politically incorrect - but that belief is massively due the idea of "western (white) supremacy".

cz if you're smart & pragmatic - then you will know innovation can come from anywhere - but western elites choose to continually bury their heads in the sand.


As john oliver said on conan many years ago: "an inflatable barbecue!".

China can certainly design an inflatable barbecue. China can certainly biuld an inlfatable barbecue. But will the chinese people ever want and buy an inflatable barbecue? ... never. That is why the US will remain the premier consumer economy.


The US is the richest consumer market in the world.

And yet BYD is likely to outsell Ford worldwide this year (despite being banned in the US)

https://en.wikipedia.org/wiki/List_of_automotive_manufacture...


I agree with the hard part being managing state, especially environments and ports. I've never used lsof so much in my life.

Question on Remote Workspace: Can the remote machine port forward so I can use a browser to see / test current state of the app on the remote machine?


On the docket! Right now the main thing we have enabled is the file system + terminals + ai agents through remote workspaces, but yes dev environments is definitely on the agenda :)

It’s my daily browser. Small glitches occasionally and can lag chrome releases, but the best absolute non-adware browser with powerful features.

I have it configured to be ultra minimal with the look of Arc that I loved.


Have always said community solar programs are the way to do residential solar. In those programs, you pay for and own off-site solar panels, then the energy they produce is credited to your electric bill.

They end up installed at commercial locations ideal for solar: often on covered parking, in fields, or on industrial roofs. Easier to repair, they can do larger panels, no issues with your roof line or roof condition.


My electricity comes from a cooperative. You have to own at least 1 share in order to become a customer (250eur/share). They sell power at very competitive rates, are powered entirely by renewables owned by their customers, and they actively encourage energy savings. Most years they pay a dividend of around 5% of the share price, which essentially lowers my actual electricity price some more.

10/10 would recommend.


Now imagine if we create a separate company that invests in infrastructure and sells energy to willing buyers who don't have to deal with this. We could call it an energy company.

The difference is in motive. You want to have a cheap energy bill, the energy company wants to make a profit that grows every quarter. Those two goals are eternally at odds.

We could make it so profit wasn’t the prime motivator, instead it’s main motive would be to provide a public good. We would call it a public utility.

Now imagine this hypothetical "energy company" has the community over a barrel with no viable alternatives, and uses their monopoly to jack up power rates and screw the community.

I'm a member of one which has worked well: https://www.edinburghsolar.coop/

And a member of a wind farm project (Ripple Energy) which went bankrupt. So like all small investment schemes, I guess you need to keep a close eye on their financials.


That sounds a bit pointless to me. The benefit of putting solar on your house is that your house uses electricity, and transmitting electricity is not actually free.

If you just care about the overall transition to solar (which you should!) then you can pay for green tariffs and invest in existing solar energy companies or ETFs.


The panels on my roof go straight to the energy company anyways. You need to add batteries to use it yourself.

Only the surplus goes straight to the energy company. If you're consuming any energy and producing it from solar you'll use your own solar first, even without batteries.

Which is maybe 10-20% for most people without batteries. Better than nothing, but not great.

Yes! My son who has autism would eat anything we put in front of him until age 3, when his weight, appetite and health suddenly and alarmingly crashed. Ever since that episode, he's had a much more restrictive diet and food preferences. Night and day.

They never successfully identified what happened. Just diagnosed it generally as failure to thrive.


There’s some research on sudden onset autism being treated with antifungals; so at least sometimes a sudden change may be the result of something very specific in the gut.


There is not reasonable evidence supporting the idea that autism can be treated with antifungals.

Case reports are unreliable due to placebo effect.

The antifungal myth has been tested by too many well-meaning parents with no results.


Telling people to just ignore microbes and the microbiome,

to stop pulling levers when they are not enjoying their time,

is medical injustice.

How long ago was it you were in threads calling the treatment method in TFA an unfounded crackpot myth?


Azole antifungals destabilize biofilms,

often allowing the body to get a good run at any low-level chronic infections which have nested and protected themselves,

able to leave the biofilmed region and wreak havoc - even if only intermittently.

Very interesting, this impact of antifungals on longterm bacterial infections! Specifically known to be effective off-label for Bartonella.


Also reads like the paid placement of some sort of mobile phone or streaming bundle.


Look I'm speaking here as a career designer:

I think design as a "signaling function" for determining the quality of a thing was already broken. It was already possible to put up an impressive-looking site for anything; already possible to to dupe people with cheap product wrapped in fancy packaging.

Movies with insane budgets that spend forever in production are often still terrible. One of my favorite songs was written by the artist in a hotel room on a Sunday afternoon.

One thing to consider: if it's cheap and immediate to wrap any content in design, it can now also be cheap and immediate to customize the design of content. Maybe we can finally return to a user-focused internet like the one that was promised to us by browser custom style sheets.

Finally, I can see democratizing design in this way will make more content more pleasant to look a (which is a win). And we'll also make better decisions with design out of the decision matrixes it doesn't belong in (another win).


I know for most people that the big surprise here is sustained search ad revenue in the face of AI. But I’m super curious on margins because I thought for sure offering so much free AI inference would be so insanely expensive it harmed margins.


No one is losing money on inference these days. Google's vertical integration means that they have some of the lowest inference costs in the industry in any event.


Microsoft recently announced changes to copilot because, apparently, it was losing money on inference.


They were charging a flat rate per query no matter how many tokens it consumed. People naturally got very good at writing prompts that used as many tokens as possible.


They were loosing money giving absurdly generous agentic usage on expensive models to people with $10 to $40 flat rate subscriptions.

They weren't selling inference.


Conjecture, but the wording "limited subset" rarely turns out to be good news. Usually a provider will say "less than 1% of our users" or some specific number when they can to ease concerns. My guess is they don't have the visibility or they don't like the number.

I feel for the team; security incidents suck. I know they are working hard, I hope they start to communicate more openly and transparently.


“Less than 1% of our users” means 10k affected users if you have 1 million users. 10k victims is a lot! Imagine “air travel is safe, only a subset of 1% of travellers die”


I've been part of a response team on a security incident and I really feel for them. However, this initial communication is terrible.

Something happened, we won't say what, but it was severe enough to notify law enforcement. What floors me is the only actionable advice is to "review environment variables". What should a customer even do with that advice? Make sure the variable are still there? How would you know if any of them were exposed or leaked?

The advice should be to IMMEDIATELY rotate all passwords, access tokens, and any sensitive information shared with Vercel. And then begin to audit access logs, customer data, etc, for unusual activity.

The only reason to dramatically overpay for the hosting resources they provide is because you expect them to expertly manage security and stability.

I know there is a huge fog of uncertainly in the early stages of an incident, but it spooks me how intentionally vague they seem to be here about what happened and who has been impacted.


Seriously. Why am I reading about this here and not via an email? I've been a paying customer for over a year now. My online news aggregator informs me before the actual company itself does?


Please remember that this is the same company that couldn't figure out how to authorize 3rd party middleware and had, with what should be a company ending, critical vulnerability .

Oh and the owner likes to proudly remind people about his work on Google AMP, a product that has done major damage to the open web.

This is who they are: a bunch of incompetent engineers that play with pension funds + gulf money.


This industry's favored idiot children.


I just deleted my account. Their laid-back notice just is not worth it anymore. I will hold them accountable using my cash. You can get out with me. Let their apologies hit your spam filter. They need to be better prepared to react to the storm of insanity that comes with a breach or they lose my info (lose it twice, I guess..)


Says they emailed affected customers...


Via the incident page:

> Environment variables marked as "sensitive" in Vercel are stored in a manner that prevents them from being read, and we currently do not have evidence that those values were accessed. However, if any of your environment variables contain secrets (API keys, tokens, database credentials, signing keys) that were not marked as sensitive, those values should be treated as potentially exposed and rotated as a priority.

https://vercel.com/kb/bulletin/vercel-april-2026-security-in... as of 4:22p ET


The “sensitive” toggle is off by default. I’m curious about the rationale, what's the benefit of this default for users and/or Vercel?

https://vercel.com/docs/environment-variables/sensitive-envi...


Simpler for vibe coders.


Ok but it's not the original intent: that default exists since at least 2020: https://web.archive.org/web/20201130022511/https://vercel.co...


Sensitive environment variables are environment variables whose values are non-readable once created.

So they are harder to introspect and review once set.

It’s probably good practice to put non-secret-material in non-sensitive variables.

(Pure speculation, I’ve never used Vercel)


I have used Vercel though prefer other hosts.

There are cases where I want env variables to be considered non-secure and fine to be read later, I have one in a current project that defines the email address used as the From address for automated emails for example.

In my opinion the lack of security should be opt-in rather than opt-out though. Meaning it should be considered secure by default with an option to make it readable.


How does the app read the variable if it can't be read after you input it? Or do they mean you can't view it after providing the variable value to the UI?


They mean the latter. Very unclear how that translates to meaningful security.


You could have a meaningful wall between administrative/deployment interface backends and the customer server backends - only the latter get access to services that have the private keys to decrypt the at-rest storage of secure variables, and this may be fully isolated to different control planes. So it becomes write-but-not-read.

But that's just a bare-minimum defense-in-depth. The fact that an attacker was able to access the insecure variables, and likely the names of secure variables, is still horrifying.


I agree / hope that’s what they meant. It seems disingenuous, though, to describe it as unreadable, since obviously something has to read it to bake it into the deploy. And given their apparent lack of effective security boundaries in one area, why should we assume that they’ve got the deploy system adequately locked down?

It’s not like I had a ton of trust in them before, but now they’ve lost almost all credibility.


Last year Vercel bungled the security response to a vulnerability in Next's middleware. This is nothing new.

https://news.ycombinator.com/item?id=43448723

https://xcancel.com/javasquip/status/1903480443158298994


Security is hard and there are only three vendors I trust: AWS, Google and IBM ( yes IBM ). Anything else is just asking for trouble.


Having worked both public and private, I can agree with this.

Google in particular has been staggeringly good, and don't sleep on IBM when they Actually Care.


Oracle too


Oracle? Oracle?

The Oracle that published an announcement that said "we didn't get hacked" when the hackers had private customer info?

The Oracle that does not allow you to do any security testing on their software unless you use one of their approved vendors?

The Oracle that one of my customers uses where they have to turn off the HR portal for 2 weeks before annual performance evaluations because there is no way to prevent people from seeing things?

The only reason Oracle isn't having nightmarish security problems published every other week is because they threaten to sue anyone that does find an issue.

Oracle is a joke in every conceivable way and I despise them on a personal level.


I love a good cathartic rant


> The only reason to dramatically overpay for the hosting resources they provide is because you expect them to expertly manage security and stability.

This and because it's so convenient to click some buttons and have your application running. I've stopped being lazy, though. Moved everything from Render to linode. I was paying render $50+/month. Now I'm paying $3-5.

I would never use one of those hosting providers again.


Looking at linode, those prices get you an instance with 1Gb of ram and a mediocre CPU. So you are running all of your applications on that?


Personal projects/MVPs/small projects? Absolutely. For what I'm running, there's no reason to need anything beyond that.

The point is, I used to just throw everything up on a PaaS. Heroku/Render, etc. and pay way more than I needed to, even if I had 0 users, lol.


> Looking at linode, those prices get you an instance with 1Gb of ram and a mediocre CPU. So you are running all of your applications on that?

I ran a LoB webapp for multiple companies on a similar setup. Turns out 1GB of RAM is insufficient to run even the most trivial Java webapps, like Jenkins, but is more than sufficient for even non-trivial things using Go + PostgreSQL.

Your stack may be slow, not the machine.


Most of my services run with 1vCPU and 512Mb of ram. You don't need huge specs for most normal applications.


For $3.5, Hetzner gives 2 vCPU, 4GB RAM, 40 GB SSD, and 10 TB of bandwidth.


Pretty oversold iirc, but then again, that's the same for Linode


Do you mean these are shared instances, and the stated resources are not actually available?


how much work should the GP do to migrate if Linode is good enough, to potentially save up to $1.50/month (or spend 50 cents more)?


If you're only paying $3-5 on Linode then your level of usage would probably be comfortably at $0 on Vercel.


It could be $0 on Render too, but then there's going to be a 3 minute load time for a landing page to become visible, lol. So if you don't want your server to sleep, you're going to have to pay $20/month.

Does Vercel do the same?


No, I run several small websites on Vercel for free for years, always served static pages very quickly


Static pages, sure. But what do you do if you want a contact form or something? Yeah, you can use services like formspree, but then you may end up paying $20/month for that alone. Perhaps I'm just ignorant.


Render offers free static sites that are served via a CDN and load instantly: https://render.com/docs/static-sites


When I said landing page, I had contact forms and more in mind, not documentation sites.

But that is news to me. Interesting. Although for static sites, I always use Netlify or even GitHub pages.


No.


Repeating a prior comment I've made about this[0]: I run a rust webserver on a €4 VPS from hetzner that serves 300M (million) requests a day.

From what I can figure out, Vercel charges "$0.60 per million invocations" [1], which would cost me $180 per day.

[0] https://news.ycombinator.com/item?id=47611454 [1] https://vercel.com/docs/functions/usage-and-pricing#invocati...


I run a Rust webserver on a literal Pi3 in my basement and I think I managed to bench it up >1000 rps for standard loads. And that includes a bunch of tanvity querying as well.

I suspect I could do 3000+ rps with some tuning and a more modern CPU or hetzner VPS, but there's some fun cachet from running on an old Pi while there's still headroom.


Makes sense considering the quality of Vercel's security response and customer communication.


What if they have an actual back-end with long-running processes and scheduled tasks?


exactly people paid the premium so somebody else's OAuth screwup wouldn't become their Sunday. and here we are.


Completely agreed. At minimum they should be advising secret rotation.

The only possibility for that not being a reasonable starting point is if they think the malicious actors still have access and will just exfiltrate rotated secrets as well. Otherwise this is deflection in an attempt to salvage credibility.


Yeah, given there insane pricing I think the expectations can be higher. Although I know it is impossible to provide 100% secure system, but if something like that happens, then the communication should at least be better. Don’t wait until you have talked to the lawyers... inform your customers first, ideally without this cooperate BS speak, most vercel customers are probably developers, so they understand that incidents like this can happen, just be transparent about it


Welcome to the show.

While a different kind of incident (in hindsight), the other week Webflow had a serious operational incident.

Sites across the globe going down (no clue if all or just a part of them). They posted plenty of messages, I think for about 12 hours, but mostly with the same content/message: "working on fixing this with an upstream provider" (paraphrased). No meaningful info about what was the actual problem or impact.

Only the next day did somebody write about what happened. Essentially a database running out of storage space. How that became a single point of failure, to at least plenty of customers: no clue. Sounds like bad architecture to me though. But what personally rubbed me the wrong way most of all, was the insistence on their "dashboard" having indicated anything wrong with their database deployment, as it allegedly had misrepresented the used/allocated storage. I don't who this upstream service provider of Webflow is, but I know plenty about server maintenance.

Either that upstream provider didn't provide a crucial metric (on-disk storage use) on their "dashboard", or Webflow was throwing this provider under the bus for what may have been their own ignorant/incompetent database server management. I guess it all depends to which extend this database was a managed service or something Webflow had more direct control over. Either way, with any clue about the provider or service missing from their post-mortem, customers can only guess as to who was to blame for the outage.

I have a feeling that we probably aren't the only customer they lost over this. Which in our case would probably not have happened, if they had communicated things in a different way. For context: I personally would never need nor recommend something like Webflow, but I do understand why it might be the right fit for people in a different position. That is, as long as it doesn't break down like it did. I still can't quite wrap my head around that apparent single point of failure for a company the size of Webflow though.

/anecdote


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: