The online platform Vinted reported on the process of transitioning their data retrieval system after using Elasticsearch since 2015. Despite years of successful use, they encountered limitations and decided to switch to Vespa, another open-source search system, in a process that took nearly a year.
Before the transition, Vinted utilized 6 Elasticsearch clusters, each with 20 machines. Managing these large clusters was time-consuming, prompting the team to opt for Vespa due to its simplified deployment process. Vespa’s Vespa Application Package (VAP) bundles configurations into a single package, making it easy to add additional nodes and automatically distribute data across nodes, reducing the need for multiple clusters to a single cluster with 60 data storage machines, 3 configuration machines, and 12 container machines.
The advantage of Vespa lies in its faster and safer re-indexing compared to Elasticsearch. Vespa supports high input rates, with new document ingestion taking just 4.6 seconds. Vinted helped patch Vespa to support data analysis with Lucene, maintaining search efficiency similar to Elasticsearch.
Vespa is an internal project by Yahoo! used for search, data recommendation, and advertising. Yahoo! later open-sourced the project, with the team separating to form a company last year.
Source: Vinted.Engineering
TLDR: Vinted switched from Elasticsearch to Vespa for improved search efficiency and simplified data management processes, with Vespa’s quick re-indexing and support for high input rates being key advantages.
Leave a Comment