Turbocharging Solr Index Replication with BitTorrent

Many of you probably use BitTorrent to download your favorite ebooks, MP3s, and movies. At Etsy, we use BitTorrent in our production systems for search replication.

While the entertainment industry has been busy paying off US senators to legislatively undermine the domain name system, their nemesis BitTorrent has continued to be a remarkably powerful technology for efficiently and securely replicating all kinds of “intellectual property”, such as multi-gigabyte search indexes for handmade goods (a source of dignified, creative jobs).

Where some see only a bucket brigade for thieves, others recognize one of the most significant innovations in the last decade of network computing.