A/B Testing Strengthens Digital Transformation Improves Organizations

Creating a culture of experimentation at your organization can be challenging. From determining who’s leading the experiments to who executes, ideates, and shares the findings, it’s important to have a set model for experimentation so your team can have a repeatable process where everyone knows their roles and responsibilities regarding experimentation. 

To help you implement an effective A/B testing program at your organization, we’ve broken down a few structures that could help you organize your team. These frameworks are great jumping-off points and don’t have to be followed perfectly. Each company is different, so it’s up to you to pick the framework that works best for your team or to take ideas from each of them to tailor it to your organization.

The models we will be covering are:

• Central Experimentation Model
• Execution Model
• Dispersed Model
• Assigned Ownership Model
• Hybrid Model

Central Experimentation Model

The companies that use this framework are typically decentralized. They have single owners whose work rarely overlaps which allows the owners to work independently, reducing the amount of time needed to sync and work together. This framework is used mostly for companies that need a dedicated experimentation team and who are aiming to create a culture of experimentation and scale their current experimentation practices. 

Since there are several single owners that work separately, it’s incredibly important to have a central experimentation team. This central team is key to push the adoption of testing for all parties. They facilitate the learnings and outcomes of an experimentation with everyone. 

This model could be right for your team if:

• Your team is decentralized
• Teams don’t need to consistently be in contact with each other (they have independent roadmaps)
• You have a good understanding of experimentation and how it could help your company
• Centralized experimentation could negatively impact individual teams speed

An example of a company using a Central Experimentation model could be an experimentation program manager whose role is to inform all the individual product managers on best practices and learnings for running experiments. The experimentation program manager is in charge of setting the goals for the program and reporting back to the product managers and companies leadership team on the results of the experiments run. They are also responsible for improving and growing the organization’s experimentation moving forward.

If this is the model you’re looking to use, a good starting point would be to write where ownership comes from for the central experimentation team rather than for individual teams. This clarifies who is in charge of leading execution.

Execution Model

The Execution and Dispersed Model are very similar, but differentiate based on who owns building and launching experiments.

With the Execution Model, teams must collaborate with each other and be in constant communication. Meeting consistently on a weekly or bi-weekly basis is crucial to the success of your experiments. 

Similar to the Central Experimentation framework, there’s often a team dedicated to managing experimentation. What differentiates this model is that the managing team helps with every aspect of experimentation. They help with ideation, design and analysis, something that the Central Experimentation team rarely does.

The Execution Model has a team that owns building and deploying experiments. An example would be that your team has determined which tests you’re going to run in the upcoming quarter. The experimentation team owns the responsibility in making it happen. This requires dedicating engineering or development resources. This is impactful for teams since you’re putting the ownership on the experimentation team and everyone else simply has to focus on creating a hypothesis or taking action from the experimentation’s learnings.

This model could be right for your team if:

• You want people in your organization to focus on understanding learnings and contributing ideas rather than using their resources on experimentation
• You have the resources on your team to run experiments
• You don’t fully have buy-in from your engineering and development team for them to execute on their own

Dispersed Model

The Dispersed Model is the opposite of the Execution Model meaning that the ownership of execution is on the engineering and development team. This model is best if you don’t have engineering and development resources on your experimentation team.

Despite the engineering or product teams building the tests, oftentimes, the creative teams such as design and marketing will have ideas on what to experiment on which is useful for growing your teams experimentation program. With this model, regular weekly or bi-weekly meetings are still important to keep everyone up to date with the status of the experiments.

This model is often used if a team has several product teams that are running experiments with the same top-level domain. An example of this would be having multiple teams who individually own steps towards the same goal such as having customers add items to their cart.

This model could be right for your team if:

• You have buy-in and resources from engineering and development teams to run experiments
• You have a big team where several people are working towards the same goal or on the same product
• You work collaboratively on prioritized items and want everyone to review results, even if they may not have been on the team working on experimentation

Assigned Ownership Model

If you have a smaller team where your organization is new to experimentation, the Assigned Ownership model is perfect for you. With this model, you may have to educate your team on the value of experimentation and you may not have all the resources you need. Don’t stress if you’re in this position, everyone’s gotta start somewhere!

Since you’re low on resources, this means you may have to use resources and findings as they become available to you. The Assigned Ownership model is a group of people whose jobs are only partially around experimentation. This means they may have another role on the team, but people in the Assigned Ownership’s experimentation team  should be spending at least half (or more) of their time on running experiments.

This model could be right for your team if:

• Your organization is new to experimentation
• Experimentation resources are tight on your team
• The number of tests you can run is limited

Hybrid Model

Fitting to the name, this model is a hybrid between the Central Experimentation, Execution and Dispersed models. It’s for multi-tired organizations with multiple products. This model will have the Central Experimentation team measuring the success of the experiments and supporting the experimentation teams. With the Hybrid Model, this could mean supporting multiple individual teams or several Execution or Dispersed experimentation teams.

All the information about the models above should be applied to the Hybrid model. It’s all about taking bits and pieces from each one and making it work for you. 

This model could be right for your team if:

• You have several products or experiences that need to be collaborated on
• You have a high level of experimentation maturity and have a history of running successful experiments

A Certain Level of Centralization Creates a Successful Experimentation Program

Having a central owner or central team owning your experimentation program is key to establishing focus on the roles and responsibilities of not only your experimentation team, but how everyone in your organization works on experimentation. Having a central owner or ownership team allows responsibilities to be clearly defined and allows for smooth cross-functional communication and collaboration.

It’s important to keep in mind that your structure may change over time. As your organization grows, the needs of your experimentation team will change to ensure experiments are run in the most effective way possible.

Ready to start experimenting? Chat with a Taplytics experimentation expert and learn about how our Customer Success team will position you with the necessary onboarding and training to get you started.