Do you literally just do this deduplication with your brain alone? Or do you have some kind of clever indexing scheme and, if the latter, have you ever written it up? Because that would be interesting to read about.

It is all explained at https://news.ycombinator.com/item?id=27284079 and the links back from there.