Send A/B test data using the Analytics Toolkit API
Learn how to use the easy to use Analytics Toolkit API to get your test data into tests in the A/B testing hub. Applies only to sequential evaluation (AGILE) tests.
This article will help you
- configure a test to use the Analytics Toolkit API as a data source
- implement an Analytics Toolkit API endpoint to push data for your A/B tests
The Analytics Toolkit API allows you to push A/B test data from any programmable source. It is a powerful way that advanced analytics teams and CRO agencies use to employ the statistical engine of Analytics Toolkit for the analysis of their online experiments.
Prerequisites for using the API
If you can extract your data from some sort of data storage and then make HTTP requests when certain conditions are met (e.g. at certain time intervals), you should be able to unlock the power of our custom API.
At the cost of a minimal investment to program your end of the API you can apply the powerful AGILE statistical engine to the analysis of your A/B tests. The API accepts data via both GET and POST requests, though POST is recommended due to inherent limitations of GET handling. The data payload should be a JSON encoded string.
The API is currently only available to clients on our Agency and Enterprise plans.
Setting up the API for the first time
To begin, navigate to Account > Data Sources > Analytics Toolkit API. If this is the first time setting up the API, you will see a screen like the one below:
initial API configuration details
You should download the API reference and go through the document to understand the parameters supported by the interface. It also contains detailed information on responses and error codes.
The two pieces of information which are shared across all A/B tests that you use the API to push data for are the endpoint and the authentication hash. The endpoint URL should therefore be hardcoded in your code. Ideally the code will handle redirects gracefully so that in case it changes your API implementation continues to work as before, but it is a forward looking requirement and we will make sure to communicate any API changes to all users in advance.
The second piece of information is the authentication string which is required for your API requests to be honored. It is a precautionary mechanism against unauthorized data pushes.
You can then create a new test and choose the Analytics Toolkit API as its data source so you have a test you can play with to debug your API endpoint.
Once you start using the API the same page under Data Sources will contain a list of your most recent API calls, including their outcome and any errors encountered. This can be useful in monitoring and debugging your API calls.
configuration details for the Analytics Toolkit API
Should you need to get an updated documentation file or retrieve the endpoint URL or your authentication hash, these will be available in the right sidebar, alongside the API call log, as shown above.
API calls are time-agnostic
API calls are time-agnostic, given the test is past its start date. This means that if you have missed sending the data at the expected time, you can always do it later. E.g. your data base was down or slow to catch up with processing so test data which should have been available and pushed via the API on March 1 only became available on March 3. You can send the data on March 3 without creating any issue for the engine.
This also opens up the opportunity to reanalyze old tests using AGILE to see how they would have been managed by Analytics Toolkit.
Using the Analytics Toolkit API for a new test
To use the Analytics Toolkit API as a data source for a test set up in the A/B testing hub, simply select "Analytics Toolkit API" as the data source:
select the Analytics Toolkit API data source
If a test has incidentally been configured with a different data source, you can reset it either via the "Manage data link" from the test actions drop-down in a list of tests in the hub, or from the test's results page.
After confirming the choice for the API as a data source, you will see the test ID which needs to be used to identify the correct test when making calls to the API. Copy it and paste in the platform where your API endpoint is coded. It will be needed on every call to the API.
test ID for the Analytics Toolkit API
That should be all that is needed.
If you need to check the current API reference, a link to it is also displayed. Clicking on the "show" button will reveal the API endpoint and authentication hash, which, since they are the same for all A/B tests, should not be needed at this point.
- once the API has been set up, you only need the test IDs to start pushing data for additional A/B tests
- check API responses to know when to stop gathering more data in a test
- recent API responses are also available in the Analytics Toolkit interface under Account > Data Sources > Analytics Toolkit API
What to watch out for
- make sure the start data of the test precedes the date on which you first start pushing data via the API
- for A/B tests with a non-binding futility boundary, a "PendingAction" action response will be issued on crossing of the lower boundary. Separate action needs to be taken in your platform to stop the test in case the user decides to do so.
Last updated: Feb 07, 2022