Multi-Armed Bandit

One real-world example of a multi-armed bandit problem is when a news website has to decide which articles to show to its readers. With no information about the visitor, all click outcomes are unknown. The website’s goal is to maximize engagement, but it doesn’t have data on what exactly its readers like.

The news website could also have a problem choosing which ads to display. The ads are on too many pages, and there is too much variation in the ads that can be effective. They want to maximizes revenue from their advertising, but they may not know enough about their users. A problem similar to that of news articles is the lack of information that they have about their visitors. They often have a large number of ads that can be effective at driving revenue for them.

Meaning Of A Contextual Bandit

A 'multi-armed bandit' is a situation in which there are multiple actions to choose from and each offers different rewards. In this type of situation, the contextual bandit refers to the specific environment that shapes how we perceive the rewards for taking certain actions.

For example, contextual bidding lets you select an ad or piece of content to show your customers based on where they've recently been or what they're currently doing. The context is any historical or current information you have about the user, such as previously visited pages, a purchase history, device information, or geolocation.

If you know a user has browsed Entertainment-related content in the past, you can display top-performing Entertainment articles at the top of the page. If you know a user is in Miami, you can display local weather or other relevant information.

A/B Testing & Multi-Armed Bandits

In deciding whether to use multi-armed bandits or A/B testing, you must weigh the benefits of experimentation vs. exploitation (sometimes known as ‘earn or learn’.) When using multi-armed bandits, you can test multiple versions simultaneously, which means that you get the best of both worlds: exploration and exploitation.

Once you declare a winner, you move into a long period of exploitation. The percentage of users who go to the winning variation can be measured and used to understand how to optimize future marketing campaigns. You can learn from the winners and apply this knowledge to your next campaign using the winning variation as the best one.

Multi-armed bandit testing is a type of test that does not require a winner to be declared after the end of an experiment. This means it can be more efficient and faster to get high-quality results with minimal traffic wastage.

One of the biggest flaws of multi-armed bandit testing is its complexity. The more arms, the harder it is to conduct an experiment and the more calculations you have to perform.

The #1 platform for delivering high-quality software releases, instantly.