Storyblok native AB testing vs. Optimize app: choosing the right experimentation architecture

At Croct, we have always admired Storyblok’s vision. Their component-based approach to content management is exactly how modern digital experiences should be built. Recently, Storyblok took another exciting step forward by launching its native AB testing feature as part of Storyblok Labs.

We share a core belief with the Storyblok team: content creation and experimentation should happen in the same place. Marketers should not have to jump between disjointed tools to validate their ideas.

With Storyblok’s new release, teams now have two distinct paths to bring experimentation into their headless CMS: using Storyblok's native AB testing feature or the Optimize app.

While both solutions share the goal of enabling experimentation, they take fundamentally different architectural approaches. Here is a transparent, objective comparison to help your team choose the right path.

Storyblok app for dynamic content

Add native personalization and AB testing to your blocks.

Learn more

Experimentation workflow

The most significant difference between the two approaches lies in how much of the experimentation pipeline your team is required to build and maintain.

Optimize app

Croct is a complete, end-to-end optimization engine. When you use the Optimize app for Storyblok, it handles the entire lifecycle of the experiment.

• Content: Marketers enable dynamic content on Storyblok blocks using the visual interface, without relying on developers.
• Decisioning: Our server-side engine automatically handles bucket assignment and cross-device stickiness.
• Analytics: We process data using our native Bayesian statistical engine, calculating performance per goal in real time without requiring third-party analytics tools.

Native AB testing

Storyblok’s native feature provides a fantastic UI for content editors. It allows marketers to create story variants, assign traffic weights, and define a control version directly in the visual editor.

However, it is a content-only feature. Your engineering team must absorb the operational heavy lifting of experimentation. To run an AB test, your developers need to:

1. Build the logic to randomly assign users to a bucket.
2. Implement custom routing to fetch the correct story variant.
3. Manage first-party cookies to ensure variant stickiness across sessions.
4. Integrate an external analytics platform to track goals, calculate statistical significance, and push the results back to Storyblok via their Management API.

Integration and code customization

Agility in marketing is directly tied to the complexity of your tech stack. How much custom code is required to actually launch an experiment?

Optimize app

Croct requires a one-time integration. Your developers install the SDK, and that is it.

import {storyblokInit, apiPlugin} from '@storyblok/js';import {withCroct} from '@croct/plug-storyblok/js';
storyblokInit(withCroct({  accessToken: 'YOUR_DELIVERY_API_TOKEN',  use: [apiPlugin],  components: {},}));

After this initial setup (which takes minutes), marketing achieves total autonomy. They can launch dozens of new AB tests, change variants, and track conversion goals directly from the UI without a single new line of code or developer request.

Native AB testing

Because the decision and tracking logic must be handled on your end, integration is an ongoing process. Implementing your first experiment requires writing significant custom code to handle the API response, calculate weights, manage state, and trigger webhooks.

Experiments fetch

import { storyblokInit, apiPlugin } from '@storyblok/js';
const { storyblokApi } = storyblokInit({  accessToken: 'YOUR_DELIVERY_API_TOKEN',  use: [apiPlugin],});
// Fetch storyconst { data: storyData } = await storyblokApi.get('cdn/stories/home', {  version: 'published',});
let story = storyData.story;
// Fetch experimentsconst { data: experimentsData } = await storyblokApi.get('cdn/experiments');
const experiments = experimentsData.experiments;
// Determine whether an active experiment includes the fetched storyconst experiment = experiments.find((experiment) =>  experiment.story_ids.includes(story.id),);

Variant assignment

const pickVariant = (variants) => {  const total = variants.reduce((acc, variant) => acc + variant.weight, 0);  let random = Math.random() * total;  for (const variant of variants) {    random -= variant.weight;    if (random <= 0) return variant;  }  return variants[variants.length - 1];};

Variant rendering

if (experiment) {  const chosenVariant = pickVariant(experiment.variants);
  // Unless the chosen variant is the control, its slug differs from the main story  if (!chosenVariant.is_control) {    const mapping = chosenVariant.story_mappings.find(      (mapping) => mapping.original_slug === story.full_slug,    );    if (mapping) {      // Fetch the variant story      const { data: variantData } = await storyblokApi.get(        `cdn/stories/${mapping.variant_slug}`,        { version: 'published' },      );      // Overwrite the main story data with the variant story data      story = variantData.story;    }  }}

Variant persistence

You should persist the variant ID for each experiment in your system, using a cookie or any other method.

Developer dependency

A successful culture of experimentation relies on marketing autonomy. If every test requires a developer ticket, you simply will not run enough tests.

With Storyblok Native, whenever marketing wants to launch a new test on a different story, developers must intervene to ensure that routing, state management, and event tracking are correctly mapped. After the initial integration of the Optimize app, marketers can independently launch dozens of tests.

Availability across plans

Budget and stage of growth are critical factors when designing your tech stack.

• Native AB testing: The new AB testing feature is part of Storyblok Labs and is exclusively available to customers on their Premium and Elite enterprise plans.
• Optimize app: The Optimize app integration is available across all plans, including our generous free tier for startups.

Beyond AB testing: the path to personalization

AB testing is usually the first step toward a more mature growth strategy.

Eventually, you will want to move from finding the "average winner" to delivering targeted 1-to-1 experiences.

While Storyblok’s native feature is strictly designed for AB testing story variants, the Optimize app unlocks our full real-time personalization engine. Your marketing team can instantly transition a winning AB test into a personalized experience and explore visitor profiles and the event timeline. You can target users based on their real-time behavior, firmographic data (such as Clearbit or 6sense), geolocation, or custom profile attributes, all seamlessly integrated with your existing Storyblok components.

Which should you choose?

If you are on a Storyblok Premium or Elite plan, have a large engineering team with the bandwidth to build and maintain a custom decision engine, and want to keep your tech stack limited to your CMS and a third-party analytics provider, Storyblok’s native testing is a beautifully designed foundation.

However, if you want a plug-and-play optimization engine that gives your marketing team total autonomy, provides native Bayesian analytics, and scales effortlessly into advanced behavioral personalization, all with virtually zero ongoing code maintenance, the Optimize app is built for you.

Ready to see how Croct and Storyblok work together? Create your free account and install the Optimize app today.