While I have had my time fighting the OOM killer, I believe overcommit would have always won. To torture the metaphor a bit more, airlines have OOF mechanism - they just eject the overcommitted passengers before the plane takes off.
A passenger buying a ticket is malloc(), but passengers don't always utilize the seat (use the memory). Normally this works out fine, but occasionally, there are too many passengers. Thankfully though instead of executing a couple passengers they give you a voucher.
1. If postgres shutsdowns uncleanly, your entire table is truncated; you lose everything.
2. You should check if your backup method backs up unlogged tables. For example, RDS Snapshots on AWS do not backup unlogged tables.
These 2 are a double whammy where if you aren't aware of these tradeoffs you can find that a bad restart has deleted all your data, plus your unlogged tables were never backed up.
2. If the function `throws` you have to wrap it in try/catch, or make your function `throws`
3. Your function is `red` if it `throws` the same exception.
4. see (2)
5. See the FileReader class in core.
Now, C++ exceptions might not satisfy all of these, but the problems CheckedExceptions were meant to solve still exist in C++ and as a result some style guides forbid them entirely. Like async, the biggest problem with exceptions were the ergonomics.
> Like async, the biggest problem with exceptions were the ergonomics.
I know it's not a popular take, but I prefer the idea of Checked Exceptions over unchecked ones [0], and suspect current opinions would be vastly different if Java had shipped with some sweet syntactic sugar for: "If an exception that is of kind A or B or C occurs, automatically throw another checked exception X with the original exception as a cause."
> Ex: If I'm writing a tool to try to analyze and recommend music that has to handle multiple different file types, I might catch an MP3 library's Mp3TagCorruptException and wrap it into my own FileFormatException.
This would reduce the temptation for developers to ruin the type-safety characteristics by wrapping everything in a RuntimeException just to get the ticket out the door.
The problem with checked exceptions is that they don't compose with the rest of the type system. Hence the infamous problems with things like Streams. Result types have basically all the virtues of checked exceptions without the problems.
Result types do have one problem that checked exceptions don’t. Checked exceptions automatically combine into union types in a throws or catch clause. I haven’t seen a language that lets you be generic like that.
T fn() throws E, F, G
vs
Result<T, E | F | G> // not even Rust lets you do this.
That's more a consequence of Rust needing its tagged unions declared up front so it can lay them out consistently in memory without runtime type information. Python and TypeScript have untagged unions (that are discriminated at runtime by the RTTI attached to all objects in the underlying dynamic language); they don't happen to have an equivalent of Rust's ? operator, but if they did it'd work like you're describing.
People often call Python/TypeScript unions "untagged unions" even though they're a very different creature from C/Rust unions. Ideally there'd be a term specifically for them but I'm not aware of one.
The problem with Java's checked exceptions is that it has too many kinds of exceptions to choose from and they're overly specific. Compare with Go, which has a single error interface and had it from the beginning, so it's used everywhere. Returning a new kind of error is always a local change, unless it's a function that didn't previously report errors at all.
Type systems permit either standardization or fragmentation and that's an ecosystem issue. Another example is that a language without a strong consensus on which string type to use will result in a fragmented ecosystem when each library goes its own way.
I don't understand, why would you need to pick a checked exception? It's the dual or mirror of feeling paralyzed over a return-type because there are "too many kinds of Object to choose from."
If you're writing a CrystalBall class with a gaze_deeply() method, you'll probably return your own VisionResult (extends Object) unless it throws your TooCloudedException (extends Exception).
When someone else writes a wrapper or higher-level layer that uses your code, then it'll be up to them to convert or wrap those results and exceptions into something suitable for their level of abstraction.
> Returning a new kind of error is always a local change
One of my axioms here is that return-values and checked-exceptions are two sides of the same architectural type-system coin. While I'm not familiar with Go, that sounds like something that would be a symptom of bad architecture if it occurred for return values.
In other words, suppose all Java methods always returned Object [0]. That would also ensure that a new return type is "always a local change" to the compiler, but I think most developers would be rightly horrified if they came across code that worked that way.
> you'll probably return your own VisionResult (extends Object) unless it throws your TooCloudedException (extends Exception).
> When someone else writes a wrapper or higher-level layer that uses your code, then it'll be up to them to convert or wrap those results and exceptions into something suitable for their level of abstraction.
Why though? What do you gain other than longer stacktraces with all those wrappers? People always trot out some theoretical notion that a caller is going to catch that framework's different exceptions and handle them differently, but have you ever seen calling code that actually did that?
> In other words, suppose all Java methods always returned Object [0]. That would also ensure that a new return type is "always a local change" to the compiler, but I think most developers would be rightly horrified if they came across code that worked that way.
There are many different kinds of values. There really aren't that many different kinds of error - there's "transient error that you might want to retry", "programmer called the API wrong", and that's about it, most other cases (like bad user input) probably shouldn't be exceptions.
> Why though? What do you gain other than longer stacktraces with all those wrappers? People always trot out some theoretical notion that a caller is going to catch that framework's different exceptions and handle them differently, but have you ever seen calling code that actually did that?
You've never seen a try that has more than 1 catch block for different exception types?
> There are many different kinds of values. There really aren't that many different kinds of error - there's "transient error that you might want to retry", "programmer called the API wrong", and that's about it, most other cases (like bad user input) probably shouldn't be exceptions.
Do you think bad user input should be a result type? Because exceptions are essentially the same thing.
You've hit on a couple of problems with exceptions in Java though. The first is I think the default for checked exceptions should be no stack trace. As the designer of the method you've left it to the caller to decide to handle it or not, if they choose to turn it into an unchecked exception then I believe that is where the stack trace should start from. Assuming there's enough context in the checked exception the designer of the method gave you everything you needed to handle it so why do we need to capture that part of the stack trace? If it ends up getting logged the source of the issue was where it was changed to an unchecked exception.
The other issue comes down to usability. Try catch blocks aren't expressions so if you want to default something in the case of a checked exception it's a lot of low information density lines. Converting to an unchecked exception is also more ceremony than it really needs to be, but there's not really a reason why it couldn't be made simpler with some syntax sugar.
> You've never seen a try that has more than 1 catch block for different exception types?
I've seen it for APIs that throw exceptions for bad input or whatever. But what I've never seen is more than one catch block for wrapper exceptions (except perhaps to unwrap the cause), where the calling code handles FrameworkNetworkError differently from FrameworkDatabaseError or what have you.
> Do you think bad user input should be a result type?
Yes
> Because exceptions are essentially the same thing.
Well, except for all the ways they're not that you mentioned in the following paragraphs.
1. "Why bother throwing a new exception of a different class, and not bubble up the original as-is?"
2. "Why would you use the standard feature of all Java exceptions which allows you to chain them, and not just throw away the original exception after copying some of its message string?"
For #1, it should be obvious in almost any language, if not instinctive: The library for managing customer records should return `Customer` objects instead `some.database.FetchResult` ones, and likewise it should throw `CustomerAccessException` instead of a `some.database.DatabaseException`. It's a matter of abstraction and preventing weird coupling.
For #2, surely you've been debugging something before and cursed at how the log is missing crucial details that could have saved you hours of trying to reproduce the problem? By chaining exceptions, you get automatic access the inner exception type, message string, additional properties, and deeper stack trace.
* Discarding the 'cause' at this point is a waste, the work was already done, the memory already allocated, why not benefit from it?
* Sometimes it's not your bug, and having the inner exception makes it much easier to get the necessary cooperation of someone else and get it fixed faster.
* Since the declared type is Throwable, it's not creating a compile-time dependency between layers. You could do a softer runtime test on the cause's type, but usually that's a temporary workaround while you complain that someone else's code is hiding critical details.
> there really aren't that many different kinds of error
I feel that's naive, handling edge cases and errors becomes more important as any system gets larger. Some are emergent from the complexity, others always existed but we can't afford to ignore them anymore.
There are many errors because all errors are contextual! Bad-input in the HTML form-submit is not the same as bad-input in the SQL query. A failed invariant of a tree that somehow made a loop is not the same as a failed invariant of something reporting negative length. An SSH handshake error is not a TLS handshake error.
> It's a matter of abstraction and preventing weird coupling.
Does that actually work though? Who ever handles CustomerAccessException specifically, except by calling getCause() and looking at the underlying DatabaseException?
> There are many errors because all errors are contextual! Bad-input in the HTML form-submit is not the same as bad-input in the SQL query. A failed invariant of a tree that somehow made a loop is not the same as a failed invariant of something reporting negative length. An SSH handshake error is not a TLS handshake error.
But the way you handle them is the same. All you can ever really do is a) retry it or b) fail it and alert the developer. And the wrapper doesn't make either of those any easier.
Note: This subthread has become about exceptions in general, rather than checked-vs-unchecked. I don't mind, but I wanted to make it explicit.
> Who ever handles CustomerAccessException specifically, except by calling getCause() and looking at the underlying DatabaseException?
Ideally you call a method like `getDetailFoo()`, and using the inherited `getCause()` is for hacky workarounds, like when you need a change in behavior much sooner than Customer classes will be changed to give you the information you need in a proper way.
As I said before about abstraction and coupling, the programmer here shouldn't need to know that SomeDatabase v.1.2.3 is an implementation detail of Customer. Their code shouldn't need to have a direct dependency on some.database.DatabaseException to compile either. Sure, a Java programmer can use runtime reflection instead... but at that point they should definitely be having second-thoughts about whether they're on a path to the Dark Side.
Regardless of how you write the catch-logic, the chained cause remains important for logging and diagnosis.
> But the way you handle them is the same. All you can ever really do is a) retry it or b) fail it and alert the developer. And the wrapper doesn't make either of those any easier.
I'm going to assume this is equivalent to: "All reactions to errors usually fall into a few broad descriptive categories, and encapsulating the original exception doesn't help the handler make precise decisions."
Does that sound right? Because my initial readings went in much less charitable directions like "the person throwing the exception knows better than you do about how you should handle it later" or "all retries are the same basic logic."
> All you can ever really do is a) retry it or b) fail it and alert the developer. And the wrapper doesn't make either of those any easier.
The wrapper is quite important. Imagine these layers exist:
1. 1-5 different HTTP libraries used by...
2. 5 different API clients for 5 different fortune cookie service, used by...
3. A library that promises a wide and resilient range of fortunes by putting many sources (today, 5) together and semi-randomly picking between them.
4. The website you're making that shows fortunes when people log in.
Do you want layer #1 exceptions to travel all the way to #4 as-is, so that you get an HTTP 500 and you won't actually know which remote service is at fault? (Without grepping the stacktrace, which is *ick*.)
No, because either #2 or #3 should be wrapping the HTTP exception in a new one that can carry that extra "which one did I randomly pick" information in a defined way.
Do you want a #2 exception to travel to #4 without changing, so that you have to write catch-clauses (or reflection) for 5 or more exceptions from the 5 different services, code that will be wrong when a version update turns it into 7?
No, because #2's job is to abstract those N services away and throw its own much set of exceptions to cover common cases.
So after all that I realize I didn't specifically address retry-vs-fail choices, but I think this still sufficient to show that layering is necessary.
I know that's the theory, but have you ever seen it work that way in practice? Because I haven't.
> Regardless of how you write the catch-logic, the chained cause remains important for logging and diagnosis.
Sure. But if it's not being caught and handled but only logged or investigated then you're not gaining anything from the wrapper.
> Do you want layer #1 exceptions to travel all the way to #4 as-is, so that you get an HTTP 500 and you won't actually know which remote service is at fault? (Without grepping the stacktrace, which is ick.)
Yes? If it's a fail-and-alert-the-developer case then I'm going to look at the stacktrace first anyway.
I might have retries at any layer, but I don't see any case where #4 uses information from a wrapper exception from #3 to make a decision. The only thing #4 is going to do with a failure from #3 is retry or fail, and it's not going to make a different decision based on which service it was. (Maybe you want metrics or circuit breakers on the individual fortune cookie services, but in practice what I've seen is that you'd always find a way to plumb them in at layer #1 or #2, by reflection if need be, rather than have them work of the exceptions coming out of #3. I know the theory you're talking about, I've just never seen it actually work that way)
This isn't like returning Object. It's more like returning a String. After using a language with a common String type, who wants to go back to writing code to convert between between different kinds of strings? Having to choose among different string implementations because there's no standard usually leads to boilerplate code doing conversions at the borders.
Usually you just want to propagate or log errors, so having a generic error interface is sufficient. It's true that in Java, you can wrap exceptions, but that's extra boilerplate.
(And yes, Go does notoriously have error propagation boilerplate that they should fix, but that isn't a type system problem.)
Exceptions are a very good comparison because they also perform non-local control flow.
Checked exceptions are a form of coloring, while unchecked aren't. But Java (which has checked exceptions) has an escape into unchecked land in the form of RuntimeError, most async languages do not, short of spawning a background thread (for sync->async) or force blocking (async->sync).
Interestingly, Result<T,E> based error models are semantically (and even syntactically, mostly, except for the call-site annotation) equivalent to checked exceptions. Usually these languages have enough abstraction capabilities (HKT for example) to make coloring not an issue, or, again, an escape into unchecked land (for example panic in rust or Go, although the latter hardly counts as having Result-like error handling).
#3 is not satisfied, as you noted in #2. You can call `throws` methods from non-`throws` methods by wrapping the call in a try catch, and `throws` methods can call non-`throws`. There isn't an exclusivity asymmetry like there is for JavaScript async.
That only applies to Javascript, which, is mostly only red functions anyways (there are no blocking apis in javascript). Javascript doesn't have the coloring problem in the way Python or Rust has it.
In Python, you can wrap the call with asyncio.to_thread, in rust with tokio::spawn_blocking.
I think you got it backwards: JavaScript has the coloring problem while other languages don't.
"Red functions are more painful to call" alludes to async functions. Every await yields back to the event loop, which adds overhead. Making every function red/async adds a performance cost (and makes it harder to reason about race conditions), which is why JavaScript has a mix of blue and red functions.
Other languages can escape the "red functions can only be called by red functions" trap, like Python asyncio.run or Tokio block_on. JavaScript has no such alternative, not even in Node. Therefore, Python and Rust don't have function coloring, but JavaScript does.
I think the big difference is that in an application that cares about throughput and being non-blocking simply blocking isn’t really possible as it affects the whole performance of the application. Wrapping a checked exception in an unchecked one doesn’t do that.
Ok, sorry it's been about 20 years since I last javad IIRC you didn't have to declare exceptions in your function signatures. However, wrapping in try/catch seems to violate #3. Try catch is not a heavy lift of a seam between red and blue
To be fair, #3 seems to have shades of grey. In some pls, you can call an async function from a sync one by wrapping it in a whole damn event loop system. Should that count?
> Ok, sorry it's been about 20 years since I last javad and you didn't have to declare exceptions in your function signatures.
You're probably remembering RuntimeExceptions, which are a subgroup [0] that are exempt from "checking" by the compiler, which means it does not require method signatures to declare "I might emit this."
Your question just kicks the can down the road. The problem with the article, to me, is the author doesn't want to accept a certain amount of complexity and has erected arbitrary road blocks.
To you, a "whole damn event loop system" is too high a price to pay, but try/catch is not. The complexity of exceptions is invisible to you. However there are certain environments (e.g. FFI) where I dont want "the whole damn exception runtime".
i mean no? the coloring problem appears to be solvable in zig. i maintain an FFI binding library for the BEAM vm, where once zig finishes its stackless coroutine support, the same zig function written once should be fully interchangeable between non-async, threaded-async, or async-with-yieldpoints-wrapped-in-the-BEAM's-scheduler.
> Try catch is not a heavy lift of a seam between red and blue
> To be fair, #3 seems to have shades of grey. In some pls, you can call an async function from a sync one by wrapping it in a whole damn event loop system. Should that count?
I think you have to count any extra overhead where you can't just write f(), including try/catch. It's always possible to call whatever kind of function from whatever other kind of function if you put enough effort and hackery in, so if we can't use functions of kind x in functions of kind y as normal "f()" function calls then that has to be what we mean by colouring.
I think you are just reflexively trying to argue your point without even thinking about it.
30 million tiktoks are posted by day. What do you mean your are going to allow "users to filter". This "regulation" will be trivially defeated by TikTok-Videos LLC uploading videos and TikTok-DataScience, providing the most popular filtering algo.
At the end of the day, many children will simply default to using the best algorithm, and all this regulation helps no one.
Wow clearly this a problem that can never be solved, better to not regulate these tech giants that have anti-democratic and anti-human beliefs. We're simply too powerless to regulate these entities!
Yeah, I clearly prefer non-hamfisted regulation under the guise of "protect the children". Half-assed attempts like this are worse than useless.
Just like with GDPR, the tech giants will put their foot on the scale and continue to operate how they fit, and the smaller guys will die under a thousand paper cuts. I'd rather not add more regulation that cements Meta as the sole media platform of the internet.
>I don't for a second believe that tiktok (or facebook or any of the others) employs a primitive algorithm
Is your contention that whatever future law have some mechanism to decide the complexity of the algorithm? How would you design a law such that the reddit ranking algorithm is primitive, but tiktok's algorithim is "advanced".
You're changing the subject. I said nothing about the law, only objected to a claim about the internal mechanisms of tiktok.
If we're discussing hypothetical laws then my preference is for several. Banning various dark patterns (what the EU is doing here), banning opaque individualization outside the control of the individual in question, and banning motivated editorialization (such a intentionally promoting a particular political position). And yes, a straightforward application of what I wrote there would make the netflix recommendation algorithm as it currently stands illegal. I have no problem with that.
Respectfully, I think this is more of a use case where you aren't the target audience for a service like AWS.
I've been working with AWS for nearly 10 years. Many people I know, both small and large, just don't even use the console. If I need to figure out how much a project costs I use the AWS pricing calculator. Having an ec2 pricing on the pricing page is meaningless once you spend any meaningful amount of time in AWS. Once you add discounts and reserved instances, that number is going to be inaccurate anyways.
If you just need a VPS provider, there are better, less complex options. I find these complaints kind of like stepping into an F1 car and complaining that the F1 car is deceiving you because theres no fuel gauge.
I'm a contractor, so my deployment complexity is whatever my current client's complexity is.
> If you just need a VPS provider, there are better, less complex options. I find these complaints kind of like stepping into an F1 car and complaining that the F1 car is deceiving you because theres no fuel gauge
That's fine if you feel that way. The article and following discussion is clearly about the smaller audience, and I think you're underestimating how far up these little problems stack and scale. If a couple grand is a rounding error to you, that's great. Most businesses fall firmly in the place where that would be a problem.
I think there is a value add for large companies on AWS, but for smaller ones, I don't particularly feel like AWS is an F1 car, more like a self driving Tesla that locks you inside when it's on fire. And I find the cavalier attitude that these companies aren't important enough to add the distinction to be exhausting. AWS is being pushed on everyone.
AWS is being pushed on everyone the same way Hadoop was pushed on everyone in the 2010s and IBM in the 90s. Everyone sees themselves as webscale, when their data can reasonable fit in Excel. If the only product on AWS you are using in EC2 and S3, you are choosing the wrong tool.
The complexity of AWS is because a service like AWS is complex. Neither Azure or GCP has any less complexity. DigitalOcean offers way less services and as a result is way less complex.
>And I find the cavalier attitude that these companies aren't important enough to add the distinction to be exhausting
They aren't important in the same way a F1 car doesn't think families are important enough to add a back row seat. No company is going to have fidelity to serve a perfect product to every market. The frustration comes from the misplaced belief that a product should serve every kind of user in the market.
>I don't know of anyone saying you should buy an F1 car for your family, do you?
It's a metaphor. Your clients telling you they need you to deploy on AWS are the kind of people I believe are telling you to buy an F1 car to daily drive to whole foods. You said it yourself: "AWS is being pushed on everyone".
>I do see people in this very thread with very different ideas of when AWS makes sense for you.
Naturally. However, 99% of, what I believe are illegitimate complaints of AWS (AWS has tons of legitimate complaints), are from people who were probably better served by a using a simple VPS provider than a cloud provider. A VPS provider is simpler, easier to understand wrt to pricing, and cheaper. Most of the complexity in AWS comes from the fact that AWS itself is a very complex tool targeted to large organizations and deployments where people aren't using EC2 instances, or are using 100s of them. The complaint that the UI doesn't have enough affordances when trying to create a single EC2 instance is kind of ridiculous when you consider it's a tool designed for people launching 100s of instances. Nobody is reasonably launching 100 instances through the dashboard. Furthermore, if vendor lock-in is a concern you have AWS is the wrong tool.
Likewise for IAM. People complain a lot about IAM. But AWS has thousand different user types, and a 1000 different services. I've written my fair share of permission systems with a fraction of the amount of permutations. They always become complex due to the combinatorial nature. GCP manages to somehow be even worse. But you wouldn't need to deal with something like IAM if you just stuck with a VPS.
>In an alternate universe, messaging on the Internet could have been:
I don't think so, and I think the very reason is because the people who opt for these decentralized solutions never really sit down and try and design a product, they just want decentralization for the sake of decentralization. For them decentralization is the product. Your alternate universe evaporates when you ask the question "what happens if Alice's device is offline?".
If you squint, the exact system you are describing is e-mail, and that has become effectively centralized, and it happened long before we had tech mega corps.
The issue with email is that too little is specified, and is instead just left up to providers and clients.
This specification vacuum forces centralization because the only way to build essential usability features is to own both the mail server and the mail client.
If email had evolved to move with the times such that basic QoL features were part of the spec rather than proprietary extensions, then it could have stayed decenteralized.
Contrast with what happened on the web. Yes it's imperfect but there is a standard that evolves to move with the times and there are multiple implementations of that standard.
A passenger buying a ticket is malloc(), but passengers don't always utilize the seat (use the memory). Normally this works out fine, but occasionally, there are too many passengers. Thankfully though instead of executing a couple passengers they give you a voucher.
reply