Efficient Session Search using Topical Index Shards

EFFICIENT SESSION SEARCH USING TOPICAL INDEX SHARDS Gijs Hendriksen, Djoerd Hiemstra, and Arjen P. de Vries∗ Radboud University, Nijmegen, The Netherlands ABSTRACT Retrieval is often considered one query at a time. However, in practice, queries regularly come in the context of sessions with coherent topics. By dividing a collection into topical index shards and matching the topical context of a session with the right shards, we may reduce the amount of resources required for answering each query. We consider two alternatives: (1) starting with exhaustive search and pruning unnecessary shards after each session turn, and (2) applying a resource selection algorithm to pre-select shards at the start of the session. The first approach, which we call shard pruning, uses pseudo-relevance feedback of retrieved documents to select shards for later turns. It starts with exhaustive search for the first query in the session. After each query, we register which shards did not contribute any documents to the top 1500 retrieved documents, and remove them from consideration for subsequent session turns. In other words, each time we process a query, we prune the set of shards from which documents are retrieved. The rationale behind this approach is that the session topic should become more pronounced as the session proceeds, and thus the number of shards under consideration can be reduced as we go. The possible downside is that we prune shards that are not relevant to a specific query in the session but would be useful for a later one. The second approach is based on the selective search setting, in which a resource selection algorithm is used to predict the set of relevant shards for a single query. In our case, we select the shards to use throughout the session after receiving user input at the start of the session. Note the difference with selective search: instead of performing resource selection for every query individually, we only perform it once for the first query in the session. The resulting list of shards is used for all queries in the session. We evaluate both approaches on the TREC Conversational Assistance Track (CAsT) datasets, which contain conversational search sessions with coherent topics. We focus on CAsT 2019 and 2020 and use the manually rewritten queries provided by the task organizers to focus on retrieval effectiveness (instead of conversational query rewriting). We apply the QKLD-QInit clustering algorithm to partition the collection into a set of topical index shards. Table 1 shows the mean recall (R@1000), cost-in-shards (how many shards were used) and cost-in-postings (how many postings were used) obtained by our shard pruning system on the CAsT 2019 dataset. Our setup is able to reduce overall cost by nearly 50%, while keeping recall within a 5% margin of exhaustive retrieval. Similar trends were observed on the CAsT 2020 collection. ∗{gijs.hendriksen, djoerd.hiemstra, arjen.devries}@ru.nl Table 1: CAsT 2019 performance of systems that iteratively prune shards after each conversation turn. R@1000 CiS CiP (×103) Exhaustive 0.84 94.0 1197.5 Shard pruning 0.83 35.5 614.3 (–49%) Table 2: CAsT 2019 performance of systems that select shards for the whole session using only the first query. R@1000 CiS CiP (×103) Exhaustive 0.84 94.0 1197.5 SRBR 0.81 5.7 206.9 (–83%) CORI 0.83 58.0 877.6 (–27%) ReDDE 0.82 32.0 596.4 (–50%) Rank-S 0.82 36.8 642.1 (–46%) Taily 0.82 51.0 778.6 (–35%) L2R 0.82 25.0 516.7 (–57%) Table 2 shows the same metrics for our system that preselects shards using the first query, using a number of popular resource selection algorithms: CORI, ReDDE, Rank-S, Taily and L2R. SRBR is an oracle method that ranks the shards based on the number of relevant documents they contain for the whole session. This setup is extremely effective when we use oracle resource selection. However, in practice, existing resource selection algorithms struggle to select the right shards using only the first query. Our experiments show the viability of using topically partitioned document collections to make conversational question answering more efficient: high recall can still be achieved with a 50% reduction in costs. A system tuned for early precision requires even less resources. Our work was accepted to ECIR 2025 [1] and our code is published to GitLab.1 ACKNOWLEDGMENTS This work has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101070014 (OpenWebSearch.EU, https://doi.org/10.3030/101070014). REFERENCES [1] Gijs Hendriksen, Djoerd Hiemstra, and Arjen P. de Vries. Efficient Session Search Using Topical Index Shards. In ECIR 2025. To appear. 1https://gitlab.science.ru.nl/informagus/ efficient-session-search https://doi.org/10.5281/zenodo.17238317