Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Thank you for your advice, that's handy to know.

What's the database format it uses, sqlite3? (I tried looking on the repo but couldn't any obvious reference)



I believe it is all flat-file. When I say "Database" I am just referencing what they call it:

    -d DB_DIR, --db-dir DB_DIR
But from what I see the structure looks like this:

    db/
        YYYY-MM/
           1234554543262346.eml.gz - I assume the meat and potatoes of the email along with attachments, not sure
           1234554543262346.meta - JSON file with msg_id, thread_id, flags, labels, subject, etc


if you want to try something that downloads Gmail via imap and indexes it into an sqlite3 db (with FTS5 fulltext of from/to/cc/bcc/subject/body fields) and extracts attachments to filesystem, take a look at a recent project of mine:

https://github.com/good-3g-apps/email/tree/master/imap-impor...

It also saves the raw .eml files to disk. No support for labels (yet), but it does properly link up threads in the db using `References` from the parsed headers (setting both MPTT and adjacency-list fields)

It's WIP, so contributions welcome :)


FWIW regarding speed, i was able to download, index & extract my entire INBOX + Sent Items (14k emails, 3.5GB total) in < 10min on a fast connection. the limiting factor by far was connection/imap speed.





Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: