I have a few black wattle rounds that have been sitting around for years. I have a go at them whenever I feel like I need to be humbled. There's also a fallen tree at the bottom of our property that blunts chainsaws. It's been there for years and nothing seems to eat it. I harvest what I can from it, and a good sized chunk will burn through the night.
Series hybrids can compete with parallel hybrids, because the full decoupling of the engine from the wheels claws back some of the efficiency you lose through the energy conversion. It's series-parallel hybrids they can't really compete with, because those are able to do the same trick, but they also lose less energy in conversion, because the engine does some of the heavy lifting.
Series hybrids are great for packaging, though. Parallel and series-parallel commit to certain packaging decisions like having a transmission, or a long, monolithic unit, because it's the mechanical coupling that buys them smaller motors and potentially better efficiency. Series hybrids don't care about any of that, so even though you have bigger motors and potentially higher losses, you have more freedom over where things go.
Personally, I think there's a massive untapped market in converting old cars to hybrid engines. You wouldn't try to upgrade the old engine, you'd design a smaller and more power dense package and rip all of the original gear out. Because electrification lets you cut the size of the engine down so aggressively, this is probably a feasible strategy. As you pointed out, series hybrids are probably best suited to this because of their packaging flexibility. As others have pointed out, there's tremendous potential there for replicating original driving characteristics using software and the electric motor. And if we're being honest, off-road vehicles probably should get rid of the transmission and low range, because electric motor torque is just better. As is, the potential for cars is enormous, but we're getting the worst possible outcomes thanks to legislation.
It's hard to know for sure. There are good information theoretic reasons to suspect that general models will always be better than smaller expert models, but maybe a MoE can claw some performance back, albeit with redundant computation. The properties of conditional entropy, for instance, always favor more generality. This assumes that the harness isn't a factor, or is at least equivalent across different models.
Yep. These days, simplicity is a massive part of my development style. I don't want to be looking at a codebase, even my own, and thinking "shit, this guy was way smarter than me".
There are good information theoretic reasons to suspect that general models will be better than specialized ones, because knowledge and skills often overlap different areas, sometimes in surprising and unintuitive ways.
And yes, I'm aware that that statement might seem to fly in the face of much of the past two years of industry development, where specialized models have been in vogue. I think they'll settle to being appropriate for low cost "good enough" applications, but I'm less convinced they'll have anywhere near the fidelity of larger frontier models.
It is, but they have different use cases. CadQuery uses a geometry kernel that does boundary representation, which you need for path generation for modern manufacturing tooling. OpenSCAD produces a standard mesh representation (i.e. triangles), which is insufficient for cutting and subtractive manufacturing, but often fine for additive manufacturing (3D printing).
Google is in an incredibly strong position. They're a top tier AI vendor, and in a world where content creation is largely commoditized and outsourced to AI, advertising companies will determine what gets seen, and what gets buried in the noise. They control both generation and visibility of what gets generated. Facebook could be in the same position, but they aren't as strong in AI. OpenAI wants to be Google, but they don't have the advertising reach.
Yeah, they aren't perfect or always necessarily the best in a given area, but to compare them to IBM is probably missing the forest for the trees.
> Yeah, they aren't perfect or always necessarily the best in a given area, but to compare them to IBM is probably missing the forest for the trees.
I think comparing them to IBM is reasonable, just maybe not... today's IBM.
IBM was an absolute hardware and software behemoth leading up to the PC / early Internet era, after which they pivoted from making groundbreaking real things to providing "enterprise support."
They also outlasted almost all of their contemporaries with that pivot, for better or worse.
IBM got to where it is today by being complacent and not keeping up with innovation. Google is notably at the forefront of innovation, in a driving seat, and that innovation directly stands to benefit one of their core businesses in a way that the market is probably only just beginning to understand. It's an entirely different situation, imo.
A technology company that's profitably doing over $50 billion in sales after more than 100 years with no obvious signs of impending doom sounds like an okay position to me, even if the tables have turned since 1984[1].
The context here is that at one point, IBM was an innovator and global leader in the technology space before it got outcompeted. It was the first company to cross a $100 billion market cap. If you think of the IBM of 50 years ago as being roughly analogous to Apple today, the difference is pretty clear. Google is much closer to Apple than it is to IBM, and I don't see that changing.
The policy is how you select your actions -- in this case, the next token. It can be random, but it doesn't have to be. "Deterministically choose the best action" is a valid policy (we would call it the greedy policy), as long as you have some other means of injecting stochasticity so the model explores the space. Uniform random is also a valid policy, as is always selecting the same token (it obviously wouldn't be very performant, and would defeat the purpose here, but it might be fine in, for example, a multi-armed bandit scenario). Most of the time, the policy is a parameterized distribution, and we want to learn the model parameters that maximize some measure of success (the reward component).
Off-policy versus on-policy refers to what data the model is trained on. On-policy training is where the training data is collected by the policy. Off-policy training is where the data was collected by a different sampling process (e.g. we have a standard dataset that we're going to use for supervised training).
Sort of. Garbage collectors can be fallible too, especially where release optimization is used.
reply