Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

*.txt.gz is not exactly a modern data format.

I share the parent's sentiment; finding easy access to good weather data is hard.

Do you want your side project to be an experience in learning how to parse obscure formats and coordinating obsolete dependencies, or HTTP GET some JSON and immediately do something with the data?



You're shitting me, right?

They're literally giving you all of the data in a plaintext format with documentation, and you're still complaining?

Sure, having everything handed to you on a silver platter is nice, but why should they do your project for you?


Imagine what is going to happen when they discover the entire financial system is built on copying CSV files around by FTP.


Don't forget the emails back and forth and manual fixes and resubmissions from either end when there's a problem causing the CSV to not parse!


> *.txt.gz is not exactly a modern data format.

You know what you get with a format that's not "modern"? A huge, mature set of tooling that works. For instance, without trying because just responding to this whinging is barely worth the wear on my keyboard, I'd be disappointed if OS X wouldn't cheerily open the file, and unarchive it into the original directory structure simply by typing "open weather_data.txt.gz" from a terminal window. In other words, one could script it. I'd show you how, but holding your hand leaves me with only one hand for typing.

> HTTP GET some JSON and immediately do something with the data

Well, that's the magic of JSON: any code I write just wondrously knows what to do it without me telling it how to parse it. How can poor ol' ASCII text possibly compete?


You don't even need the terminal, just double click the file.


Can't script a double-click in Finder. :-) (Well, there's AppleScript...). But, yeah, point taken.


Not modern doesn't have to mean difficult to parse, it depends on your personal outlook. You can also download individually customized csv files if you want, which my original link led to. csv and txt files seem eminently reasonable to me when dealing with a huge historical cache of data that's being put out by a public institution for free.


I could write a node server that converts all that fixed length ASCII data into JSON faster than you guys can complain.


  wget -O - http://cdiac.ornl.gov/ftp/ushcn_daily/state01_AL.txt.gz | gunzip -c |  in2csv -s schema.txt | csvjson
Fixed that for you. And in less space than it took for you to complain.


What entitled nonsense. *.txt.gz is not a problem for anyone of even minimum competence.


Wow, you guys really missed the point.

Why can't we ask for nice things, even if they are "free"?

How are you going to get a kid interested in programming historical weather analysis when he's got to figure out how to configure an entire stack just to query your dataset. If you really want my opinion, this should be an AI problem, not yet another tedious MVC application.

Furthermore, when I jump into projects, I am most definitely appreciative that there is a responsive system on the other end, no matter how much I'm getting paid. I'm not some noob and yes I have imported gigabytes of minimally-structured data into my own applications, but I don't want to have to do that every time. Especially just to tinker.

If you're not thinking about how your software development experience can be made quicker, more direct, and less painful, I pity you. Shit on JSON all you want, but I'll take a populated native object and a blinking cursor over FTP'd text files any day.


"Shit on JSON all you want, but I'll take a populated native object and a blinking cursor over FTP'd text files any day."

I'm not sure this particular hill is worth dying for. Remember, text was the universal transport long before JSON was (and probably will be long after).




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: