Overview
In the world of technology, ensuring equal opportunity is crucial for attracting the best talent and positioning yourself well in the market. Meta (formerly Facebook) has developed an innovative RL-based system to address potential biases in ad delivery.
Challenge
Online advertising platforms aim to provide visibility to users about products and opportunities that might interest them. However, these systems can inadvertently create disparities in how ads are served to different demographic groups. This can lead to certain groups being underexposed to important opportunities, such as job listings or housing ads, raising concerns about equal access to information.
Solution:
To address this issue, Meta developed a Variance Reduction System (VRS) that leverages Reinforcement Learning to help ensure that the audience for a given ad closely reflects the eligible targeted audience across various demographic groups. This system aims to balance ad delivery in real-time, adapting to changing conditions and user interactions.
Meta's team created a simulated version of their ad delivery system to develop and test the RL solution. The state space combines a user embedding (a compact representation of user characteristics) with a measurement of the current delivery variance for the ad. This allows the system to make decisions based on both individual user factors and overall ad performance across demographics.
The action space is notably straightforward: a binary choice to either increase the value of an impression or make no adjustment at all. This facilitates quick and frequent decisions in the fast-paced ad auction environment. This approach directly connects the RL agent's success to the goal of equitable ad delivery.
Results
The effectiveness of the VRS was demonstrated through rigorous testing. During a two-week A/B testing scenario, the system significantly reduced Non-Conforming Ad Coverage (NCAC), a metric used to measure delivery imbalances, achieving a 76.36% reduction in NCAC across the gender variable and a 53.95% reduction across the race variable.
Factored AI
At Factored, we constantly push the boundaries of what’s possible, applying cutting-edge research from labs worldwide to real-world applications for our customers.
Our team of RL enthusiasts are excited about how these equity-aware RL principles could extend beyond ad delivery. The same techniques that Meta uses to balance demographic representation could be applied to ensure equity in automated hiring systems, loan approval processes, or resource allocation in public services. Factored is exploring how to combine value-aligned RL with variance reduction techniques to help organizations make more equitable automated decisions.
Factored AI
Center of Excellence: Machine Learning
Expert Group: Reinforcement Learning
Team Lead: Alejandro Aristizabal
