'Sequential Query Prediction based on Multi-Armed Bandits with Ensemble of Transformer Experts and Immediate Feedback'
Published: 2 July 2024
Our Sequential Learning paper 'Sequential Query Prediction based on Multi-Armed Bandits with Ensemble of Transformer Experts and Immediate Feedback', has been accepted in Data Mining and Knowledge Discovery journal, authored by S Parambath, C Anagnostopoulos, and R Murray-Smith. Keywords: Multi-armed bandits, Query recommendation, Immediate User Feedback, Large Language Models (LLMs), Transformers.
We study the problem of predicting the next query to be recommended
in interactive data exploratory analysis to guide users to correct content.
Current query prediction approaches are based on sequence-to-sequence
learning, exploiting past interaction data. However, due to the resourcehungry
training process, such approaches fail to adapt to immediate
user feedback. Immediate feedback is essential and considered as a signal
of the user’s intent. We contribute with a novel query prediction
ensemble mechanism, which adapts to immediate feedback relying on
Multi-Armed Bandits (MAB) framework. Our mechanism, an extension
to the popular Exp3 algorithm, augments Transformer-based language
models for query predictions by combining predictions from experts,
thus dynamically building a candidate set during exploration. Immediate
feedback is leveraged to choose the appropriate prediction in
a probabilistic fashion. We provide comprehensive large-scale experimental
and comparative assessment using a popular online literature
discovery service, which showcases that our mechanism (i) improves
the per-round regret substantially against state-of-the-art Transformerbased
models and (ii) shows the superiority of causal language modelling
over masked language modelling for query recommendations
First published: 2 July 2024
Link: paper