Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Out of interest, what created a 90GB JSON file?


Not OP, but I routinely call a specific HTTP API for millions of entities or pull down entire Kafka topics - all in JSON format. For various reasons those are the canonical sources of data and/or the most performant, so I end up ripping through GBs and GBs of JSON when troubleshooting/reporting on things.


I don't think it's quite 90GB, but I've processed Wikidata dumps in the same order of magnitude before (which are one JSON object per line) with jq, and it could've certainly been faster.


I was wondering about that - whether they are one single JSON array/object or one per line.


A Firebase Realtime Database backup file




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: