In their influential 2017 paper Aleksander Fabijan et al postulate the three paramount requirements for any experimentation service. I consider these in detail and see how Variant stacks up and even challenges some of the assumptions that seem inherent in these principles.
The experiment’s configuration, lifecycle, and runtime should be handled externally of the host application, leaving it completely unconcerned with those details. Moving experimentation-related details out of the host application’s domain achieves the complete separation of experience implementation (handled by the host application) and experiment instrumentation (handled by the experimentation server)—the holy grail of any successful experimentation framework.
The fundamental flaw of all existing commercial A/B tools is that they all started out as SaaS, which was a reasonable approach to the shallow experiments—their beachhead market,—but that architecture proved untenable when they attempted to extend down the host application’s stack. Without cogent support for full-stack experiments enterprise customers by enlarge opted to roll their own.
Although many tech-capable enterprises have been able to develop custom experimentation platforms, many tech-second companies have had to spend sweat and treasure on what are essentially stop-gap solutions. Here are the three principal reasons why implementing an A/B testing practice is a particularly gnarly problem to solve in the wild.
Scala programmers are in an advantageous position to learn Java quickly: they are likely to know a lot about the JVM and to have used Java’s standard library. This article is intended to help experienced Scala programmers to grasp Java’s key concepts quickly, by introducing them by analogy with the familiar Scala idioms.