← Paper page User Welfare Optimization in Recommender Systems with Competing Content Creators Ask this paper arXiv ↗

User Welfare Optimization
in Recommender Systems
with Competing Content Creators

Creators cannot see what all users want, but the platform can. How can it steer creators toward the users nobody is serving?

Fan Yao, Yiming Liao, Mingzhe Wu, Chuanhao Li, Yan Zhu, James Yang, Jingzhou Liu, Qifan Wang, Haifeng Xu, Hongning Wang · KDD 2024

Press → or click to step through

Creators compete on a map of user tastes

user tastes (schematic) majority taste niche niche niche creators choose what to make (a point on the taste map) platform shows each user one of their top-K matches creators are paid by how satisfied users are creators try small changes, keep them if they pay more

The platform sees the whole map. Each creator only sees her own feedback.

This information gap, not creators’ selfishness, is what the paper blames for poor outcomes.

Without help, creators get stuck on a “popular” user

how much the platform counts each user centerrighttopleftbottom halved served by a creator left unserved

The paper’s toy example: 5 users, 5 creators who improve by trial and error, top-3 matching. Best for users: one creator per user. Animation simplified from the paper’s Figure 1.

The fix: re-weight users by how well they are served

1 measure each user group’s satisfaction big niche 2 re-weight: well served count less big niche 3 send the signal through rewardsrankingretrieval three levers, next slide 4 creators adapt by trial and error toward under-served users repeat every period

The platform turns its knowledge of who is under-served into incentives creators can feel.

In a simplified setting (many creators, users with clearly distinct tastes), this update provably raises total user welfare (Theorem 2). In practice users are grouped and weights are kept within a fixed range.

Three ways to send the signal

Reward re-weighting (UIR) pay creators more when an under-served user is happy needs control over payments Softer ranking (SMT) strict softer for an under-served user, spread matches more evenly over the top-K more creators get to meet that user Wider retrieval (HMT) top 3 top 5 for an under-served user, retrieve more candidates before ranking only changes a retrieval threshold: easy to deploy

All three make serving an overlooked user more rewarding. Only the first touches payments; the other two just adjust matching.

In simulations, creators escape the trap

total user welfare over time (schematic) time → no intervention with re-weighting weights build up… …creators move
  1. ◆Wider retrieval (HMT)Largest gain, least variance; on synthetic data every user group gained.
  2. ◆Reward re-weighting (UIR)Smaller gain, very stable; niche groups gain a little at large groups’ expense.
  3. ◆Softer ranking (SMT)Moderate gain, more variance, and less even across groups.

All three beat no intervention by up-weighting under-served groups (on synthetic data, the small ones).

Synthetic data: 2,000 users in 10 groups (from 1,000 down to 10 users), 200 creators starting near the largest group. MovieLens-1M: 1,578 users, 20 creators. Figures 2–3; curve shapes are a schematic of the reported “plateau, then climb again” pattern.

Three weeks on a leading short-video platform

+1.13%like-through rate
+3.7%views of videos under 2 hours old
+0.76%views of new (cold-start) videos
+0.71%topics watched per user

Users liked more of what they saw and got fresher, more varied videos.

A/B test of the wider-retrieval lever on new videos: each arm pairs 3% of creators with 3% of users, kept fully separate. The four user-side gains are statistically significant. Popular creators’ daily activity rose 0.17% on average (others 0.06%); the paper notes three weeks may be too short for most creators to respond (Section 5.2, Table 2).

What this means for platforms

  1. 1Creators get stuck because they cannot see the whole mapPopular users act as safe bets that trap creators, leaving others unserved.
  2. 2Share what you know through incentivesRe-weight users by how well they are served, and pass the weights on through rewards, ranking or retrieval.
  3. 3It works beyond the whiteboardGains in simulations and in a three-week online test: more likes, fresher and more diverse content.

Paper page · arXiv · PDF