Huseyin Yurtseven

Off-policy evaluation ( OPE) estimates the performance of new recommendation policies using logged data, thus enabling fast, safe and inexpensive iteration prior to costly A/B tests. To evaluate ranking policies, existing OPE estimators all make structural assumptions about user behavior, leading to a spectrum of trade-offs between bias and variance.

Personalization is a key requirement for most customer experiences in a music streaming service, such as landing page, station song sequencing, or search. A common approach is to develop dedicated personalization ML models, one for each experience, that directly integrate with all the personalization signals alongside experience-specific signals. However, this is not scalable as it is costly for each product team to replicate improvements and experiment with new personalization features.