Best Practice - Capturing Client Logs and Analytics

Do you need to capture raw debug logs from devices in production from the field? Do you want to capture analytics events from applications deployed in production?

The answer is yes! At some point, you absolutely will want to gather crash logs, or debug logs from devices in production in the field, because you don't have that device or the environment to be able to reproduce a user-reported problem. I hope the value of analytics is obvious. You want to know how many times the application makes network requests, or if you've instrumented your application to report custom analytics, then you've clearly invested in its value already!

This post gives a brief introduction to the technology, but more importantly, gives some best practice guidelines on how to (or if you should) expose the underlying capture-and-report to the end user based on real-world applications you probably already use.

The MobileFirst Platform Foundation client-side SDKs have built-in ability to capture and report both raw debug logs and analytics. The client SDK has both a logger API and analytics API to report more data than what is reported out of the box. It is your job to configure each using the appropriate function calls. In fact, you actually have four jobs:

• instrument
• configure
• capture
• send

The objective-c (for iOS) codebase, Java (for Android), and JavaScript (for hybrid) codebases all have equivalent capabilities, in that they each have a raw debug logger API, and an analytics API built into the MobileFirst client SDK. The names are:

I'm going to show the examples using Java to keep this post shorter, but please understand that the usage is similar across the languages.

First let's talk about your four jobs.

The logger API is a log framework that is similar to other well known log frameworks, such as java.util.logging or log4j in Java and Cocoa Lumberjack in objective-c. It includes features such as level filtering and package filtering.

The core MobileFirst codebase is already instrumented with debug log calls, so your only job is to consider if you want to instrument your code with the public logger API provided from MobileFirst SDK (you do!). The obvious benefit of instrumenting your code with logger API calls is that it gives you the opportunity to capture logs in production from your applications already deployed in the field.

You certainly can take advantage of the public analytics API to report structured data you intend to display or analyze later. Unlike the logger API which takes unstructured human-readable text, the analytics API takes structured data intended to be used in a chart or to detect trends.

You could just take the defaults, which is why this is optional. The default configuration for both the logger and analytics is:

Capture is on by default. This just means the MobileFirst SDK is recording the output reported through the logger and analytics API calls to a persistent store, nothing more. The performance cost of having capture on is negligible.

This is the most important consideration! There's no sense turning verbosity way up, heavily instrumenting your application with logger output and custom analytics, if you never send that data anywhere!

However, you need to decide if you want all that debug log data all the time (you probably don't), and all analytics records (you probably do). The debug log data is only really useful if you have a specific problem you need to solve.

Finally, we get to see some real-world examples, and apply them to the MobileFirst client SDK.

Chrome asks the end user if they desire to send usage and crash reports:

In this case, you'd always capture, but only send if the user has agreed to it. Your configuration is:

1
2
3
4
Logger.setAutoSendLogs(false);  // only send on-demand, and only if the user agreed to it
Logger.setCapture(true);
Logger.setLevel(Logger.LEVEL.FATAL);  // user agreed to "crash reports" only
WLAnalytics.setCapture(true);

Then of course you'd send any captured data from the device up to the server periodically:

1
2
3
4
if (userAgreed) {
    Logger.send();
    WLAnalytics.send();
}

When to send is a decision left to the developer. Should you send only on wifi? Should you send on an interval? Should you send always on app open? It's up to you!

Nova Launcher is slightly different than Chrome Mobile in that it also includes "errors", and gives a more verbose explanation to the end user as to what they have agreed.

The configuration is very similar, except for the logger level:

1
2
3
4
Logger.setAutoSendLogs(false);  // only send on-demand, and only if the user agreed to it
Logger.setCapture(true);
Logger.setLevel(Logger.LEVEL.ERROR);  // user agreed to error and crash, and like any logger, the ERROR level is "error and above"
WLAnalytics.setCapture(true);

Then of course the send strategy is up to you.

Tasker is an Android "automation" application, that listens for device events and triggers an action. It is extremely powerful, intended for a highly technical end user. I include it here because Tasker chose to be extremely verbose about what the user has agreed to.

The text strongly hints that Tasker is recording at debug level, but only when the user agrees, so their initial configuration would be:

1
2
Logger.setLevel(Logger.LEVEL.DEBUG);  // always record at high verbosity
Logger.setCapture(false);

Once the user presses the button that turns on logging, your configuration changes. In the button press callback:

1
Logger.setCapture(true);

You may wish to be as verbose in both your log output and your end user explanation.

The Tasker dialog box never implies any sending to a server. It is your choice whether to do so in your use of the MobileFirst infrastructure or not, considering your end user's privacy.

I cannot find evidence in the Evernote user experience of giving the end user the choice of opting in or out of "usage reporting", so let's assume for the sake of this example that Evernote never collects anonymous usage data (which is probably a bad assumption, but go with me anyway). Evernote does, however, send raw debug logs up to the server, but only on end user demand.

Notice that Evernote uses the phrase "activity log" and "error log". To the end user, this may be confusing. Is it my activity or just application errors? It's not completely clear what the end user is sending when they use this button. So, the configuration is:

1
2
3
4
Logger.setAutoSendLogs(false);  // only send on-demand, and only if the user agreed to it
Logger.setCapture(true);
Logger.setLevel(Logger.LEVEL.ERROR);  // user agreed to error, and like any logger, the ERROR level is "error and above"
WLAnalytics.setCapture(false);  // remember our assumption!

The send strategy is to only send upon end user demand:

1
2
// inside the button on-click callback:
Logger.send();  // but not WLAnalytics.send() because of our assumption

This strategy probably makes the most sense for users who have paid for your application, as a paid-for application implies some level of support. Perhaps the "send activity logs" button is not even available in the free version of your application.

In general, don't set your logger to maximum verbosity and send all that data from all your in-production applications. You don't need all that data, nor do you want to manage the storage burden that comes with it! Consider the end-user's privacy, their technical abilities, and apply a good user experience for both both you and the end user to satisfy your need to capture logs and analytics!