I recently migrated from Cloudflare Workers to AWS Lambda + Dynamo for my relatively large pet project.
It was surprisingly hard - the local development of lambdas is still very raw, documentation is scarce, and there are various small issues appearing here and there. I should probably write a blogpost about how to setup decent developer environment for AWS CDK and Lambdas, because there's not much on the Internet about it.
I set up the whole AWS infrastructure via AWS CDK. I have one TypeScript file, that creates a stack with lambdas, dynamodb tables, API gateways, S3 buckets, wires up the secrets manager, etc - all of that in 2 environments - dev and prod.
AWS CLI also can generate a Cloudformation YAML file from the CDK file (via `cdk synth`), which could be fed into SAM. So, I generate a `template.yaml` Cloudformation file this way, then run SAM like `sam local start-api`, and it runs exactly the same lambda locally, as in AWS, using that Cloudformation YAML file. SAM also supports live reload, so if you change any source files, it will automatically get those changes.
So, okay-ish developer experience for lambdas is possible. There're caveats though:
* I couldn't figure out how to use "nameless" Dynamodb tables (i.e. when CDK assigns the name to them automatically), because if I omit a table name in CDK template, then the local lambda and lambda in AWS assume different names for some reason.
* Locally, the binary outputs don't work. It ignores `isBase64Encoded` by some reason, and lambda just returns Base64 output, instead of binary.
* And the main problem - local lambdas are SLOW. It seems like it restarts some container or something under the hood on each API call, which adds 2-3 seconds to each API call. So, the calls that should be like 30ms, are actually 2 seconds now. This is super frustrating.
I could have written your message word for word. I recently had a very similar experience moving to AWS Lambda using AWS CDK and SAM. It is pretty shocking how unhelpful the AWS documentation can be at times.
Each use case is so unique, that it is likely difficult for them to account for every single situation where Lambdas are being used. Instead they opt for a generalized and non-specific documentation site, which results in docs I find practically useless.
I frequently find solutions to problems for AWS on someone's obscure blog, who after many days of suffering stumbled onto the answer, rather than finding anything of use directly from those who wrote the software.
Does a site exist where programmers can submit useful findings? Similar to stackoverflow, but rather than asking for questions, you just submit answers. I would think being able to search across such a crowdsourced utility would save programmers a ton of time.
> Does a site exist where programmers can submit useful findings? Similar to stackoverflow, but rather than asking for questions, you just submit answers. I would think being able to search across such a crowdsourced utility would save programmers a ton of time.
SO kind of lets you do this. You can write a question and immediately answer it yourself - there's an option in the question UI for it.
Pros of Cloudflare Workers - they're way simpler. Their docs are nice, it's straightforward to set up a dev environment for them, it's easy to set up a key-value store for them (Cloudflare KV), secrets management is simple. Pricing is nice - it's flat $5 a month, and you get access to almost all of it. It works fast and distributed across the whole world, admin console is nicely integrated with the Cloudflare CDN dashboard.
But they're too limiting. Cloudflare Worker should be a pure JS script (that looks like a service worker script in a browser). You cannot use any Node libraries. In my app I need to generate images on the fly, so I couldn't figure out how to do that in a Cloudflare worker, so I ended up creating lambdas for those, and proxying the calls from Cloudflare worker to those lambdas.
KV (their key-value store) is way too simple. No secondary indexes, no sorting, no querying (you can only sorta query list of keys, but then you have to fetch a record by each key individually). My app hit the limits of KV very quickly.
Also, there's no monitoring or any good way to do backups for KV. So, it's almost unusable in a real-world production system, except as maybe a cache layer.
Monitoring was a big pain for Cloudflare Workers. Like, even accessing production logs is weird - you have to "tail" them from your CLI. Even Cloudwatch is 100x better here.
Development environment is okay, I guess, but the local server crashes constantly. Also, it kinda forces you to use Webpack and bundle the sources. It also doesn't have anything like layers in AWS lambdas, and they have a limit of 1MB for a bundle, which I almost hit (I had like a 900kb JS file), so that part got me worried too.
Basically all of that made me worry I will hit a wall soon with Cloudflare Workers, so I moved to AWS. It's not piece of cake too, but at least I get a way to run proper Node scripts, a decent document database, continuous backups and pretty good monitoring out of the box.
I tried Cloudflare Workers and found these pain points:
- local dev server (“wrangler dev”) crashes often
- local dev server executes code always on Cloudflare -> seeing changes takes a few seconds, you need an Internet connection
- forces Webpack on you
- managing KV store is difficult (How to back up? How to update schema of millions of entries?)
- KV store is slow (400 ms in Europe) unless keys are being hit every couple of seconds to keep the edge cache warm
Currently I’m trying an alternative – Deno Deploy – and love it so far. Local dev server executes your code locally, quick compile times, native TypeScript support due to Deno runtime.
> Like, even accessing production logs is weird - you have to "tail" them from your CLI.
Which seems to fail for access-protected sites since `tail` uses the same domain as the real endpoint. Also no error reporting - the exception may be in the logs, but it's missing the stack trace unless you log it explicitly.
The monitoring side of workers is extremely poor currently.
Also unit testing workers is hard to get right - there are no good mock libraries for request and kv which behave exactly like the production versions, so you end up with surprises after deployment.
It was surprisingly hard - the local development of lambdas is still very raw, documentation is scarce, and there are various small issues appearing here and there. I should probably write a blogpost about how to setup decent developer environment for AWS CDK and Lambdas, because there's not much on the Internet about it.
I set up the whole AWS infrastructure via AWS CDK. I have one TypeScript file, that creates a stack with lambdas, dynamodb tables, API gateways, S3 buckets, wires up the secrets manager, etc - all of that in 2 environments - dev and prod.
AWS CLI also can generate a Cloudformation YAML file from the CDK file (via `cdk synth`), which could be fed into SAM. So, I generate a `template.yaml` Cloudformation file this way, then run SAM like `sam local start-api`, and it runs exactly the same lambda locally, as in AWS, using that Cloudformation YAML file. SAM also supports live reload, so if you change any source files, it will automatically get those changes.
So, okay-ish developer experience for lambdas is possible. There're caveats though:
* I couldn't figure out how to use "nameless" Dynamodb tables (i.e. when CDK assigns the name to them automatically), because if I omit a table name in CDK template, then the local lambda and lambda in AWS assume different names for some reason.
* Locally, the binary outputs don't work. It ignores `isBase64Encoded` by some reason, and lambda just returns Base64 output, instead of binary.
* And the main problem - local lambdas are SLOW. It seems like it restarts some container or something under the hood on each API call, which adds 2-3 seconds to each API call. So, the calls that should be like 30ms, are actually 2 seconds now. This is super frustrating.