Filling in the blanks with 7.1 Analytics

Sometimes you may want to take your analytics data beyond the console.

Since version 6.2 there has been the ability to export your raw analytics data with the export API. This API allows querying directly into our underlying datastore, Elasticsearch.

In prior versions of Analytics we were able to satisfy all desired permutations of your data. New requests have come up showing different breakdowns that are not easily supported with the current architecture. Until we have a chance to satisfy these feature requests, we have created a small python program in the interim for version 7.1.

The python program, known as mfp_query_helper, queries directly into Elasticsearch and aggregates the desired results in memory. This can be a slow and expensive process, and thus mfp_query_helper should normally be run remotely from outside your cluster. The program is available for download here.

What kind of data permutations can I achieve with this program?

This tool is primarily focus on breaking down device data. It's main functions are to query for new devices, and to break down devices by application name and version.

Preparing your cluster for querying

The Elasticsearch http endpoint must be enabled for the tool to query your datastore. To enable to the http endpoint, add the JNDI property "analytics/http.enabled" to your cluster with value "true".

Using mfp_query_helper

Installation

To install the mfp_query_helper CLI:

$ sudo python setup.py install

To uninstall, use the uninstall.sh script with root:

$ sudo sh uninstall.sh

You can also create a python virtual environment, if you do not wish to install globally. Make sure the virtual environment uses python 2.7

$ virtualenv venv
$ . venv/bin/activate
$ python setup.py install

Basic Usage

$ mfp_query_helper -h

Currently, there are 3 queries that are supported: newDevices, mfpAppVersions, and distinctMfpAppVersions.

To run one simply use:

$ mfp_query_helper [QUERY_NAME]

By default, this will search Elasticsearch host localhost:9500 on index 'worklight'.

Query Information

• newDevices

  The newDevices query returns an array of new devices per day. A device is considered new the first time it connects to your MFP server.

  Response payload looks like:

  [
      {
          'date': 0000000000000, // timestamp of day
          'count': 0 // number of new devices on that day
      },
      ...
  ]

• mfpAppVersions

  The mfpAppVersions query returns a count of all devices that have connected to a an MFP application. An MFP application is considered a unique combination of app name and app version.

  Response payload looks like:

  {
      'AppName1': {
          '1.0': 20,
          '2.0': 25,
          ...
      },
      ...
  }

  It is important to note that if a device connects to AppName1 version 1.0 and version 2.0, it is counted in both. If you would like distinct devices, use distinctMfpAppVersions query.

• distinctMfpAppVersions

  The distinctMfpAppVersions query returns a count of unique devices that have connected to an MFP application. An MFP application is considered a unique combination of app name and app version. For this query, the latest app version will be used.

  For example, if a device connects to AppName1 versions 1.0, 2.0 and 3.0, it will only be counted in version 3.0.

  Response payload looks like:

  {<br />
      'AppName1': {
          '1.0': 20,
          '2.0': 25,
          ...
      },
      ...
  }

Command Line Arguments

There are command line arguments for Elasticsearch configuration, and to filter the given queries

To change the Elasticsearch host to query, use -esHost

$ mfp_query_helper [QUERY_NAME] -esHost myhost.com:9500

To change the index to query, use -esIndex

$ mfp_query_helper [QUERY_NAME] -esIndex otherindex

You can filter by app name, app version, device model, device os, device os version, and time range.

A basic filter would look like:

$ mfp_query_helper [QUERY_NAME] --mfpAppName myApp --mfpAppVersion 1.0

The keys for filtering are as follows

• mfpAppName
• mfpAppVersion
• deviceModel
• deviceOS
• deviceOSversion

Time range filters use

• startDate
• endDate

startDate and endDate parameters can be epoch timestamps or date strings of the format year,month,date (i.e. 2015,4,24). Both startDate and endDate need to be used together.

$ mfp_query_helper [QUERY_NAME] --startDate 2015,1,1 --endDate 2015,2,1

Some Examples

To get all new devices for the app "Insurance App" with Elasticsearch running on mysite.com, port 9500, index myindex

$ mfp_query_helper newDevices -esHost mysite.com:9500 -esIndex myindex --mfpAppName InsuranceApp<br />

Download mfp-query-helper