> All the students at Harvard are all selected from the tail end of the distribution and are very capable.
It seems like there is a pretty good way to handle this. Make the only letter grades A and F, i.e. it's pass/fail, but then additionally provide class rank percentile.
Even if everyone gets an A, in a class of 1000 students, someone is going to be at the 90th percentile and someone is going to be at the 10th and you can't inflate your way out of that.
If I get a group of 30 kids together that are incredibly intelligent and highly motivated and have had “you must be the best and you must get A” beaten into their success and livelihood since before they could talk by their parents (and let’s be real that is a good chunk of Harvard grads) - do you really think that telling them that they are going to be stack ranked against each other is a healthy and productive thing that will produce the best outcome?
There are only two things you can do here. One is that some Harvard students will have better marks than other Harvard students, and the other is that the school provides no other student evaluation than pass/fail, with the general expectation that approximately everyone will pass. You can't simultaneously give them different marks than each other and not.
Try making the fan speed curve mostly flat in the middle at whatever fan speed keeps the system from hitting the far end of the temperature range under moderate usage. Let it ramp all the way down at idle and all the way up at all-core full load but anything in the middle gets a fixed medium speed. If that medium speed is too loud then what you need is a larger fan that provides that amount of cooling at a lower fan speed.
> Right now, at this point in time, for applications like local AI and certain types of gaming, I would argue for most people having more VRAM is more useful than having faster VRAM. I personally now do more AI stuff and gaming on my M5 mac with its 24 GB shared (300 GB/s) RAM pool than my 12 GB 5070 Ti (900 GB/s).
The issue is that this in no way requires soldered memory. CAMM2 supports speeds up to 9600 MT/s. You can get over 300 GB/s from two CAMM2 sockets.
> The way to manufacture more efficient compute now is do things like put DRAM closer to the chip and even closer integration between CPU and GPU.
People have been hyping things like this for decades, but then it turns out the number of applications that need to frequently share data between a CPU and GPU at a faster speed than PCIe can handle are pretty uncommon. Meanwhile putting them closer together has some pretty significant real disadvantages, because then you're trying to deliver more power and dissipate more heat over a smaller area instead of putting more physical separation between the two largest loads in the machine.
Notice that high end PC GPUs are significantly faster than any of Apple's integrated GPUs, and that's why.
> There are also latency and bandwidth benefits how they setup their RAM just from pure physics.
Soldering RAM has a modest latency advantage over SODIMMs at the most extreme timings and CAMM turns even that into basically nothing.
> And chip manufacturing is moving towards chiplets where you have cores manufactured separately and then wired together at nanoscale level on top of a silicon interposer.
You're describing a move to less integration. They were originally on the same die, and the change has no real effect on modularity. The user doesn't even have to know that some Ryzen CPUs have a separate I/O die or more than one compute die, they all still fit into the same socket and are even interchangeable with the ones that have only a single die.
- For high end AI inference chips, DRAM already goes onto the interposer right next to the GPU to bring the bandwidth as high as possible. Apple will eventually do this for the exact same reasons. It's not just soldering RAM to a PCB
- The chiplet technique and putting everything on an interposer is less integrated from the perspective of the chip manufacturer, but for the consumer -- folks who are going to buy Framework laptops, this is a far less integrated package. CPU, GPU and RAM will sit on the same interposer and purchased together as a unit with no upgrade or swap path for any component. This is not the same as simply soldering everything together on one PCB. The level of intergration is far higher
> For high end AI inference chips, DRAM already goes onto the interposer right next to the GPU to bring the bandwidth as high as possible.
The high end AI inference chips use HBM and cost tens of thousands of dollars. HBM uses 1024 data pins instead of 64, which is crazy expensive, which means that to the extent that consumer devices get it at all, it would be in addition to rather than instead of ordinary DRAM, e.g. you might have 12GB of HBM on the CPU package but then 64GB of less expensive DRAM. Increasing the number of cache hierarchy levels is a long-term trend. HBM as L4 cache is pretty plausible for high end CPUs as a supplement rather than replacement for DRAM.
There are already servers that work like this, e.g. Xeon Max has 64GB of HBM but then further supports up to 4TB of DDR5.
Moreover, the AI inference hardware integrates the CPU into the GPU because it's really just a giant GPU. They're not getting some major advantage from that, they just know nobody is going to want to swap out the CPU on a system where the CPU is mostly irrelevant. If you wanted that level of inference performance on a normal PC which is used for other purposes where the CPU actually matters then you would drop the AI accelerator with the HBM or GDDR into a PCIe slot.
I think the long term trend is typically the high end technology of today will be the mid to low tier technology of the future.
If putting 1024 data pins all connected via a nanoscale manufactured silicon interposer right now seems complicated and expensive, that doesn't mean we won't see it in tomorrow's consumer devices. If anything we will be MORE likely to see this one day. Apple and other companies are gradually working towards moving AI models to be more local which means memory bandwidth has a real killer app use case right now. Witness Liquid AI and their partnership with Mercedes Benz to put 8B param LLM models into vehicles.
Both Desktop PCs and the CPU are becoming less and less relevant as we move further in the decade to be honest...
> I think the long term trend is typically the high end technology of today will be the mid to low tier technology of the future.
The trend doesn't look like that. The PCI bus from 1992 had 124 pins. PCIe 5.0 x16 has 164 pins; x8 has even fewer pins than the slots from decades ago. Guess how many pins Thunderbolt has. DDR1 DIMMs from the year 2000 had 184 pins; DDR5 has 288. The number of pins goes up very slowly if at all, because it's one of the most expensive ways to increase performance, despite being effective.
Which is why the enterprise hardware has always done it and the consumer hardware hasn't.
> Apple and other companies are gradually working towards moving AI models to be more local which means memory bandwidth has a real killer app use case right now.
The real problem is that ordinary consumers don't want to pay for 128GB of GDDR or HBM, and if they did then you would attach it to the GPU rather than the CPU anyway.
What they might want is the less expensive ordinary DRAM with a wider bus, which is what Apple does, but then you're not using 1024 pins and have no need to solder it instead of using CAMM.
> Witness Liquid AI and their partnership with Mercedes Benz to put 8B param LLM models into vehicles.
8B param models don't need exotic hardware, those run on existing consumer GPUs.
> Both Desktop PCs and the CPU are becoming less and less relevant as we move further in the decade to be honest...
Less relevant to what? Making up for the inefficiency of bad JavaScript with fast hardware? Running the less parallelizable parts of PC games? Databases and other branchy server workloads? They're as relevant as ever to the things they've always been relevant to.
There are generally two ways governments hold companies accountable for dangerous products.
The first is liability. If they're selling chargers that burn down houses, they get sued, and they don't want to get sued, so they don't want to sell chargers that burn down houses.
The second is regulatory requirements. This one is generally worse. The incumbents capture the regulators to e.g. have the law require their technology or raise costs to exclude new entrants. The rules are often inefficient or poorly conceived with bad cost/benefit ratios. And companies making products that are dangerous but nevertheless comply with the rules will point to their checkbox compliance to dodge liability.
The problem with the first one is that it doesn't work well against companies outside the jurisdiction, because then you can't sue them, and the importer will be a small entity that just files for bankruptcy if you try to sue them. But the second one has the exact same problem. They sell products that don't comply with the rules; if you try to fine them they're outside the jurisdiction and the only thing in the jurisdiction is a fungible importer that will dissolve if you try to go after them.
In that environment the thing that actually works is the third thing. Customers expect some products to be dangerous and rely on product reviews to determine which ones. But this is the thing the second one inhibits, because then overpriced incumbents use their influence over the laws to target any new supplier that tries to establish a trusted brand, which causes the foreign suppliers to have to sell through dozens of unknown labels so they can continue to dissolve them if any of them get prosecuted. And then customers are stuck choosing between the overpriced incumbents and the far cheaper foreign suppliers that may or may not be safe, with many people risking the latter because they have so much lower margins.
EU CE requirements are generally (outside some universally more regulated domains like medical devices) pretty lightweight to deal with, and pretty sensible. I've gone through them, and honestly the biggest pain is finding the applicable standard. Otherwise you basically just need to follow the standard and write up how you think your design follows it, and stick it into a drawer, most likely never to be seen again. You usually have to cause a very notable problem or be very obviously breaking the rules to get the regulator's attention.
How does that help you if someone is drop shipping fire hazards and trying to prosecute them means they just dissolve and create a new shell company?
Also, how does it get you anything over simple liability for fraud and harm? Why does the honest seller have to write a document if nobody is going to look at it and the dishonest one is going to skip doing it anyway?
I think the point of the parent, correct me if i'm wrong, was precisely that current EU regulations are insufficient in protecting customers and really not a burden to put any product on the market, and that anyone arguing regulation is against the little guy is talking in bad faith.
We've already tried the third one in the US before the FDA. A ton of people kept dying.
Milk was filled with borax and formaldehyde, coffee was cut with sawdust/charred bone/lead, spices were often 100% counterfeit.
The market (heavily) incentivized fraud.
In New York, in one year (1857), 8000 infants died to "swill milk" [0].
The second option (FDA and regulation) wasn't lobbied for, and the Food Bill of 1902 actually failed through heavy (counter)lobbying initially [1], until the Pure Food and Drug Act of 1906 [2] passed.
Invoking 1857 is not a valid argument really, cause consumer priorities were different. Cheaper with some level of risk (which today's American, or German would consider excessive) was preferred option hence the market response as it was - at least it's a reasonable guess.
In less rich countries it is how things work right now.
Industries dump toxic waste into waterways if they can get away with it in the US today (literally today [0]). I agree that I might not be specifically worried about borax milk if FDA was reversed, but I would absolutely expect risky shortcuts in food offerings.
The incentives in the market has never changed. That's what regulation is for, shifting market incentives/forces to favor consumers/society.
Pollution is an externality. If Alice hires a company to do "Hydro Excavation" and they pollute Bob's water, even if Bob knows all about it and is entirely opposed, he can't prevent it by not patronizing them because he isn't a party to the transaction. So the solution to this has to be to prohibit pollution.
Product safety is about information. When Bob knows that a particular brand of milk is adulterated with chalk, he doesn't buy it. Which means that all you need for this is product labeling and liability. If the ingredients list chalk, the customer who doesn't want chalk doesn't buy it. If the ingredients don't list chalk then the seller is not in compliance if the product contains it. If the battery is unsafe then you can both not purchase it because the reviews concluded it was unsafe or sue them if your house burns down. And the compliance process is simple: You list the actual ingredients on the label and have responsibility for damages caused by your product. No massive regulatory bureaucracy with thousands of pages of rules, just liability for fraud and harm.
The problem for all of these is that the perpetrator has to be in your jurisdiction. If companies in China are emitting copious amounts of CO2, regulations in Europe can't do much about it. If those companies are making unsafe products that end up on the world market, you can't sue them in the US because they have no real presence in the US. But complex product regulations don't solve that either, because they too are subject to the same problem; foreign companies drop ship things that don't comply. Nor does putting the liability in the wrong place, because generic transportation or payments intermediaries are in a worse position than the government itself to be the ones evaluating things that come over the border.
Consider it this way: Why doesn't US customs exclude unsafe products from being imported from other countries? Consider what they would have to do to actually accomplish that.
> Customers expect some products to be dangerous and rely on product reviews to determine which ones.
.. which are of course the easiest thing to fake.
> then overpriced incumbents use their influence over the laws to target any new supplier that tries to establish a trusted brand, which causes the foreign suppliers to have to sell through dozens of unknown labels so they can continue to dissolve them if any of them get prosecuted.
This is not an accurate description of new market entry for .. well, anything? And what are the new entrants being prosecuted for? Is it by any chance unsafe products?
> .. which are of course the easiest thing to fake.
How do you get Consumer Reports to publish a fake product review? Can you point to even one instance of that actually happening?
> This is not an accurate description of new market entry for .. well, anything?
Huawei is a pretty conspicuous example of it actually happening. They were starting to establish a brand and then regulatory destruction was imposed. Meanwhile there seem to be a huge number of other products from the same country with white labels or rotating unknown brands for some reason even though they probably come out of the same factory.
> And what are the new entrants being prosecuted for? Is it by any chance unsafe products?
That there is a difference between regulatory compliance and actual safety is obviously the point. All the incumbents need is for the rules to be complicated enough that compliance requires you to be a massive bureaucracy, or that nobody is really complying but selective enforcement gets imposed when someone undesirable is starting to look like a real challenger.
> How do you get Consumer Reports to publish a fake product review?
No need. Just have dozens of companies produce hundreds of new junk products every year. Then there's no way all products can be reviewed, and no way they can be properly reviewed: what's interesting about a review is the failure mode of the products, which you have no idea about when you have 10 new Samsung smartphones coming out every year.
> [Huawei] were starting to establish a brand and then regulatory destruction was imposed. selective enforcement gets imposed when someone undesirable is starting to look like a real challenger.
Yes, selective enforcement is the problem. Not regulations. Regulations are not strong enough. But they need to be applied evenly. Why is it legal for Apple and Samsung to produce junk, but not Huawei? (rhetorical question, please don't answer) We need proper consumer protection. Any company producing a product should have 10 years or 20 years warranty, and should be legally mandated to produce/sell spare parts for 20+ years (as in, real spare parts, not "replacement motherboard" which costs the entire smartphone).
Suddenly, the junk makers would produce less junk. Maybe there'd only be a new Samsung/iPhone every 5 years, but it would probably be as solid and repairable as older Nokias.
You sure typed a lot to say 'a few kids are going to get the skin melted off their face by an exploding battery, that's just the cost of doing business!'
Notice that you fail to present any argument and are only retreating into indignation at the existence of the problem. You present no viable solution or counterargument to the criticism of the status quo.
Jail the executives and engineers cutting corners. For some reason, you can spend years in jail for cannabis possession or an online post criticizing Macron or the police, but people who actually commit murder and ecocide by cutting costs in engineering products, or who import such bad-quality products face exactly zero consequences.
Then we can't drink our water, can't eat anything from our soil, sometimes can't even breathe our air. But we are the only ones facing consequences while the rich fuckers are partying on yachts.
The engineers are in China and thereby outside your jurisdiction. Likewise the executives of the company that actually makes the thing. The only people in your jurisdiction are tangentially related intermediaries with no real knowledge or control over the product. It's essentially proposing to punish FedEx for shipping a package or PayPal for doing a funds transfer if unbeknownst to them the vendor's product is of low quality. It's desperate and ineffective, because how is the generic transportation/warehousing/payments company supposed to tell if any of a million randos' products are junk?
I can assure you many companies pushing junk are based here in France. Actual manufacturing may take place in China for many products, but Decathlon, Leroy Merlin and Carrefour certainly have executives and engineers here. Those executives and engineers pushing negative externalities to local consumers on one side, and to producers with bad social/ecological standards in foreign lands.
To be clear, i'm not against foreign products. But a french exec deciding to employ quasi-slave labor and destroy the environment on the other side of the world to maximize profits should 100% be jailed. That's already technically a crime.
Then, that marketplace has no viable business. Society does not owe them anything. Seriously, if your business model requires you to sell illegal stuff, then your company does not deserve to survive. That’s the basics of regulation.
You're assuming the conclusion. Why is it the marketplace platform who should be the police? Should banks have to audit your life before you can open a bank account? Should you be unable to transact with anyone if you're not rich enough for them to justify that expense?
It's not Walmart you're proposing to unperson here.
The sellers are in practice anonymous, and the consumer facing Temu (or Shein, or Aliexpress, etc) very much markets to consumers, yet shirk any responsibility. They are Walmart but ignore the little accountability Walmart faces.
Of course Temu is responsible for things I buy in the Temu app, and pay Temu for, which then Temu ships to me.
> The sellers are in practice anonymous, and the consumer facing Temu (or Shein, or Aliexpress, etc) very much markets to consumers, yet shirk any responsibility. They are Walmart but ignore the little accountability Walmart faces.
They are not Walmart.
> Of course Temu is responsible for things I buy in the Temu app, and pay Temu for, which then Temu ships to me.
If you send money to someone in the PayPal app, are they responsible for what you bought? Not just for giving you a refund; for having liability if your house burns down. If the seller keeps their inventory in a rented space, should you be able to sue their landlord? If FedEx delivers a package to you, are they responsible for the regulatory compliance of what's inside?
Consider what would happen if you did that. Could a normal person buy or sell something or rent space or send packages, if the intermediary had to take on liability for anything you do with it?
> I've fought for a free internet. I've fought for the right to anonymous posting. To be a voice, without an identity. But? That time is over, or we won't have a democracy.
The con is claiming that this has anything to do with anonymity. There are 8 billion people. Someone with money can get a hundred, or a thousand, or ten thousand of them to lend their names to bots. The cost of a random human name is less than the cost of the years worth of tokens they'd be putting under it. Sacrificing anonymity over that is a fraud.
On top of that, we shouldn't pretend this is a new issue. In 1975 the local magnate owned the local newspaper and radio station and had relatively unbiased coverage of issues until it came to the ones that affect their own business dealings or the political ambitions of their associates.
And your disaster scenario doesn't even work when people are getting information from multiple sources. If the sources are >50% lies and exaggerations then you can spot check a small sample of the stories and notice that. (Sadly all too many current outlets fail this test.) But if, as you posit, they offered unbiased coverage almost all of the time then their coverage would be similar to every other source offering mostly unbiased coverage, until they try to mislead you, and then your "reliable sources" would be saying different things from one another because less than 100% of them are controlled by the one trying to manipulate you, which is a major red flag that somebody is lying to you.
The actual problem is that people don't bother to do the spot checks when the source is telling them the pretty lies they want to hear. Which is nothing new and has very little to do with AI.
It's a consequence of having platforms instead of protocols.
Suppose you want delivery notifications for your packages. The seller, by contrast, wants to spam you with marketing.
If getting the notifications requires you to install their app, they're going to shovel any spam into it that they can, and then they're writing the code that runs on your device. Whereas if the software on your device is controlled by you and the notifications are received using a standard protocol, you (or someone like uBlock) can create filters to only show the notifications you actually want and discard the spam.
But for that to actually work you need the software running on the client to be under the control of the user independent of which device or service they're using, and subject to competitive pressure. Otherwise the platform uses is as a means for lock-in and then filters your notifications in the ways that benefit them rather than you, or just does a lazy job because they know you've been deprived of having a lot of other alternatives.
Unless the task is extremely well defined, protocols don't really work.
Imagine you're a shipping company and lock yourself into a parcel tracking protocol. You then decide to offer the innovative feature of parcel lockers, which need a code (or an action on your device) to open. How are you going to make the thousands of weird homebrew clients that people are using on their jailbroken Nintendo Switches or whatever to behave?
That's easy. You publish the API documentation and supply a reference implementation. Anyone can use your reference implementation immediately and the person who wants to use their own code on a jailbroken Switch can do that as soon as they implement the API, or their own fork of the reference implementation.
The service doesn't have to maintain every implementation, they just have to document a stable API and not actively impede third party code.
> But for that to actually work you need the software running on the client to be under the control of the user independent of which device or service ...
In other words, you need the user of the software to pay for it's development. Since that won't happen ...
Having the incentive to do something and having the ability to do it are not the same thing.
It's not like human-generated content is made of carbon and AI-generated content is made of silicon and the science of chemistry can unambiguously tell them apart. If you asked a million humans and a million LLMs to write a sentence on a specific subject, it's not implausible that one of the LLMs and one of the humans would output the exact same sentence. Maybe more than one.
A thing that can take only the output and accurately tell you if it was AI-generated or not is therefore impossible, because if it said no it would be wrong when the LLM generates that sentence, but if it said yes it would be wrong when a human generates the exact same sentence.
All it can do is try to calculate a probability. But then what do you want to do with that? Suppose the probability it estimates for some content is 45%, and that probability estimate is an accurate measure of the true probability, i.e. can't be improved when the only information you have is the content itself. Do you want to ban the 55% of that content which is human-generated, or allow the 45% which is AI-generated?
Right now the problem is the flood of low-quality AI spam that might (or might not) be low hanging fruit. We can worry about high quality AI artifacts later if that becomes a problem. (and yes, there is no guarantee that YouTube won't fail due to these spammers)
I get the idea: get 10k each samples of human data and AI data, train a simple classifier until it gets 99.9999% accuracy or <10k false negatives per day at your scale, ship it as a screening tool.
Is such tool feasible at all with current state of AI technology, or is it just a reasonable take from the past that may not be so reasonable anymore?
> I get the idea: get 10k each samples of human data and AI data, train a simple classifier until it gets 99.9999% accuracy or <10k false negatives per day at your scale
The issue is, that's not a thing. AI-generated content and human-generated content have significant overlap. No amount of training data can allow you to distinguish them with that level of accuracy because many outputs exist that could have been generated by either one. Additional training data allows you to say that the probability is 55.0374% plus or minus 0.0001, rather than only being able to say that it's 55% plus or minus 5%. It can tell you with greater precision exactly how ambiguous it is. What it can't do is remove the ambiguity.
We will find out shortly? YouTube is the one saying they are going to implement this:
"If a creator doesn’t specify whether or not they used AI, but our systems detect significant photorealistic AI use, we will now automatically apply a label."
It seems like there is a pretty good way to handle this. Make the only letter grades A and F, i.e. it's pass/fail, but then additionally provide class rank percentile.
Even if everyone gets an A, in a class of 1000 students, someone is going to be at the 90th percentile and someone is going to be at the 10th and you can't inflate your way out of that.
reply