Scalability and Hardware Sizing for MobileFirst Foundation 8.0

This document contains information about scalability-related parameters of IBM Mobile Foundation, and the scalability tests performed.
It serves as a guide to the provided hardware sizing calculator that is used to determine the required hardware for an Mobile Foundation deployment.

Jump to:

• Architecture
• Scalability Tests
• Using the hardware sizing calculator
• Push Notification
• Analytics
• Direct Update
• Database usage and size
• Application Center
• Security Model

Architecture

The Mobile Foundation topology is based on the following principles:

• MobileFirst Server is installed in the organization LAN, connecting to various enterprise back-end systems.
• MobileFirst Server can be clustered for high availability and scalability.
• The database is used for storing administration and configuration data, push notifications, registered clients data, and the security context of the clients. A single instance of the database is shared by the MobileFirst runtime environment in all MobileFirst Server cluster nodes. It is possible to cluster the database by using the tools and methods that are provided by the database vendor.
• MobileFirst Server is installed behind a web authentication infrastructure, which acts as a reverse proxy and provides SSL.
• The database server and the MobileFirst Server must be installed on separate host machines.

Topologies can vary depending on the corporate network architecture, disaster recovery, different back-end systems, and so on.

Scalability Tests

IBM conducted a series of scalability tests before releasing Mobile Foundation. The following table contains a short summary of the tests performed.

Test Flow

The test, which is run repeatedly, involves the following actions:

1. A typical banking application is simulated. It logs in, makes multiple calls for account list and transaction lists, and then logs out.
2. The Mobile Foundation application server uses an HTTP adapter to call a simulated back end.
3. The back end is called via HTTP and returns with a predefined response.

Hardware

The following hardware was used for Mobile Foundation application server:

• x3250 M4, Xeon 4C E3-1220v2 69W 3.1GHz/1600MHz/8MB Note: With the specified number of cores, Mobile Foundation performs thousands of transactions per second. If your throughput is low, you can use fewer cores than the specified number.
• 32 GB 1333 MHz Note: Not all RAM was used. Use the accompanying calculator to compute required RAM.
• NetExtreme II BCM5790 Gigabit Ethernet
• 2 X 500 3.5in SATA, SR C100

The following hardware was used during testing for the Mobile Foundation relational database:

• 2 Intel Xeon 8C 2.3 GHz/20 MB
• 20 GB DDR3 RAM
• 6-7 500 GB 7.2 Krpm SATA disks
• 1 RAID 5 controller

Mobile Foundation Client:

• Simulated by twenty virtual machines that run multiple client threads. Running Ubuntu 10.04 64-Bit 4 GB RAM.
• The client load is achieved with Apache jMeter, an open source server performance testing tool by Apache.

Test Results

Throughput was measured at 2,000 requests per second per server with an average CPU usage of 70%.

Using the hardware sizing calculator

About sizing

The hardware sizing calculator is based on a scenario that was tested in IBM Labs. This scenario is based on a real customer scenario.
However, MobileFirst Server is a that is augmented with server-side developer code. Therefore, the sizing is largely influenced not only by client calls, but also by the efficiency of the server-side code. The server performance is also influenced by the security checks used by the application and the security configuration. Hence, you must consider the calculator as a baseline from which you can proceed to customer-specific scalability tests.

This document is accompanied by a Microsoft Office Excel spreadsheet. The purpose of this spreadsheet is to help organizations calculate the required hardware for a Mobile Foundation deployment.

The calculator computes:

• The number of required servers. The server hardware baseline spec is defined in the preceding test.
• The required database size.
• The push notification throughput.
• The analytics server disk space and the number of servers.

Memory requirements

The MobileFirst Server version 8.0 does not store data inside the client’s HTTP session. Memory is therefore not a performance bottleneck. In the scalability tests conducted by IBM, the server was run with a maximum JVM heap size set to 6GB.

Push Notifications

What are push notifications?

A push notification is an alert that indicates a change or an update that the MobileFirst Server sends to users’ mobile devices. Push notifications are popular with all mobile operating systems. Mobile Foundation has APIs that maintain the lists of mobile devices and their tag subscriptions, and send push notifications. Mobile Foundation supports push notifications for the following vendors:

• Google's GCM for Android
• Apple APNS for iOS
• Microsoft's MPNS for Windows Phone 8

Mobile Foundation does not currently support push notification API for BlackBerry or Windows 8.
Possible architectures for push notifications Two architectures are possible for push notifications. They differ by the method that the enterprise back end uses to provide push notification data to the MobileFirst Server:

• Push notifications are sent to the MobileFirst Server by a call to a Mobile Foundation procedure.
• The MobileFirst Server pulls data by using a MobileFirst JMS adapter. The enterprise back end puts the push notifications in the JMS queue.

For more information about these alternate architectures, see the IBM MobileFirst Foundation user documentation.

The primary difference between the two methods is that the first method is limited to a single server, which is pulling data from the JMS queue. With the second method, the procedures in the MobileFirst Server are called through a load balancer. Therefore, the architecture can scale to more than one server.

Broadcast, tag-based, and unicast (1:1) notifications

Mobile Foundation V8.0 supports broadcast, tag-based, and unicast (1:1) notifications.

• Broadcast notifications are sent to all the registered mobile devices of a specific application.
• Broadcast notifications are sent to all the registered mobile devices of a specific application.
• Unicast notifications, are sent to the devices of specific users. The "userID" (or) "deviceID" is specified in the introduced server-side REST API, which enables to send push notification messages to a specified target.

Testing assumptions

Push providers were simulated with a delay that is typical to these services:

• APNS: 0.05-0.1 milliseconds
• GCM: 20-25 milliseconds
• MPNS: 20-25 milliseconds

Mobile Foundation can associate multiple devices per user. On average, each user is assumed to have 1.1 devices.
Starting IBM Worklight Foundation V6.2.0 onwards, devices can subscribe to tags or topics of interest. Performance of tag-based notifications was tested with tags that had 600,000, 300,000, and 150,000 devices subscribed to them.

For broadcast or tag-based push notifications, a MobileFirst Server can push a notification message to over 22,500 iOS devices, 50,000 Android devices, and 350 Windows Phone devices per second. This is with all of the device types together.

For unicast or event-source based push notifications, an instance of MobileFirst Server can push 1,750 notifications per second to APNS, and 350 notifications per second to GCM and MPNS, when it is under load. The load includes 2,400 regular transactions per second, as well as proportional device calls for subscribing to, and unsubscribing from, the push service.

The CPU and the memory are mostly unaffected for unicast and event-source based push notifications. However, for broadcast and tag-based push, if the number of mobile devices that would register for push notifications is large (for example, more than 100,000), then there could be bursts of 100% CPU usage and higher memory usage for a short duration during which the notification message would be sent to all the registered mobile devices. If your requirements are such that you do not want to delay your mobile users because of push notifications, you might want to have an instance of MobileFirst Server dedicated to handle all the push notification needs, and configure your back-end server to send all the push notification requests directly to this dedicated server.

The database is unaffected, except for the storage of devices and tag subscriptions, as mentioned in section 8. Database usage and size.

Analytics

IBM MobileFirst Foundation Operational Analytics

The Analytics is a WAR file that is bundled with MobileFirst Server. The Analytics is based on file system storage rather than database storage. The hardware that is required to run the Analytics relies on several factors including:

• The frequency of connectivity to the server
• The verbosity set for logging
• The number of days that data is kept before being deleted

There are three distinct data types:

• Analytics collected from server network connections
• Client Logs
• Server Logs

The analytics data is always collected and forwarded to the MobileFirst Analytics Server. MobileFirst Server produces debug logs that are optionally forwarded to the MobileFirst Analytics Server. Client logs are sent when the client app is optionally configured to collect and send logs.

An event of network connection to the analytics server is recorded in the following cases:

• A device successfully authenticates with the MobileFirst Server instance.
• A device makes an adapter invocation.
• A push notification is sent to a device.
• A client-side API is called to create a custom activity.

Server network connections occur in the following situations:

• A device successfully authenticates with the MobileFirst Server instance.
• A device makes an adapter invocation.
• A push notification is sent to a device.
• A client-side API is called to create a custom activity.

Server network connections are always collected and forwarded to the IBM Analytics Platform.

Server network connections occur in the following situations:

• A device successfully authenticates with the MobileFirst Server instance.
• A device makes an adapter invocation.
• A push notification is sent to a device.
• A client-side API is called to create a custom activity.

Server network connections typically take up most of the disk space in the Analytics Platform. Device authentications and adapter invocations are the most common types of network connections that are recorded in the Analytics Platform.

When you estimate hardware requirements, it is important to take into account the number of devices that will be connecting to your server and the number of adapter invocations each device will be making.

Client logs are sent when the client app is optionally configured to collect and send logs.
By default, client logs are set to “FATAL”, which means that only crash logs are forwarded to the MobileFirst Server instance. With this default setting, client logs use less space than analytics data that are created by server network connections.

Changing this level and adding custom client logs drastically affects the disk space used by client logs. With more custom logs and higher verbosity log levels, client logs can consume much more disk space (by several magnitudes) than server network connections. When you estimate hardware requirements, you must take into consideration the size client logs and the frequency of sending these logs.

Server logs, which are produced by MobileFirst Server and are saved to disk, are also forwarded to the MobileFirst Server by default. Server logs will have the least impact on the disk space sizing.

On MobileFirst Analytics, you can configure automatic purging of data.

For more information, see the IBM MobileFirst Foundation user documentation

Different data types can be purged at different intervals. This means that if you wish to keep a year's worth of server network event data but only three weeks of log data, you can set these values separately.

Log data takes up more space than server network event data. Depending on the configured verbosity, MobileFirst Server could be sending a very large amount of log data. If disk space is limited, you must consider purging data more frequently or adding nodes to your Analytics cluster.

You can add a new node to the analytics cluster with zero downtime. Therefore, you might want to start with a single node and add nodes as necessary. The total number of nodes that you can add to the cluster is equal to the number of shards that were created when the first node was installed.

For more information about shards and how to set up a cluster, see the IBM MobileFirst Foundation user documentation

Direct Update

What is Direct Update?

When the administrator redeploys an app to the MobileFirst Server instance without changing its version, MobileFirst Server updates the web resources folder. Then, when an app connects to MobileFirst Server, the server directly pushes the web resources (HTML, JavaScript, CSS and images) of the newly deployed app to the device. The push happens only when an app contains different web resources and the same native code.

Direct Update scalability challenges

When the administrator triggers direct update by redeploying an app to the MobileFirst Server instance, it creates a surge of web resources download requests from the connected devices to the MobileFirst Server. The IT organization has to be ready for it.

Factors that affect direct-update:

1. # of concurrent direct-update requests. The # of downloads and especially the frequency of usage of the application (e.g. once an hour vs. once a month) will determine the number of requests the MobileFirst Server can handle when the new version is pushed.
2. Application size. There is an accumulative size limitation on the ability of the MobileFirst Server to push data to clients.
3. Difference in content. Starting with MobileFirst 6.3, the difference in content between the old and the new version is also important (see below).

Direct Update lab results

An instance of MobileFirst Server can push a total of 250 MB/s. As an example, if the application size is 5 MB, MobileFirst Server can serve 50 direct updates per second. This means that if the cluster contains four MobileFirst Server nodes, the cluster can serve 200 direct updates per second. This is the limit for a single server rather than the network limitation.

Starting with IBM MobileFirst Foundation V6.3, direct update was improved to allow users to receive only the web resources that have changed between updates, instead of downloading the entire web resource package. This enhancement reduces download time, conserves bandwidth and improves overall user experience. When the delta size is between 10k to 300k the MobileFirst Server TPS (Transactions per second) rate remains almost the same (6% decrease) with an additional 200 to 300 direct update requests per second.

The customer IT infrastructure has to be ready for both incremental direct update and full direct update requests. Subject to the exact customer scenario, most of the updates are likely to fall under the incremental scenario.
Small B2C deployments and most B2E deployments can use the above product features as is. As your application grows, you will need to use other means.
As a first approximation, you can get ready by adding servers and increasing outgoing network bandwidth. A more scalable solution would be to use a CDN.

Using a CDN

The best scalable solution for Direct Update, both full and incremental, is to use a content delivery network (CDN) in front of the MobileFirst Server instance. When caching the direct update URL, the request is served from the CDN and does not get to MobileFirst Server. The expiration policy must be specified in the CDN. The CDN's primary use is for full direct update requests but it may also for incremental direct update requests especially if the factors mentioned above influence your system.

For more information, see the "Serving direct update requests from CDN page" topic in the user documentation.about Direct Update, see the user documentation topic

Database Usage and Size

Mobile Foundation database usage profile

The Mobile First database stores administration and configuration data, registered clients data, the security context of the clients and push notifications. If the Application Center is used, its data is also stored in a database.

For a comprehensive description of the database tables, including the estimated number of rows per table, see the topic Internal runtime databases in the user documentation.

The main data items that require more database resources are the following:

• Client registration data: information is stored about every registered client instance (device and application pair) including information about the device, the application, user names associated with the client, last activity time and additional custom attributes.
• Client security context: the authentication state of the client. The size of the data depends on the number of the security checks used by the application, and the size of the state data stored by each security check.
• Push notification information: the list of devices that registered for push notifications, and the subscriptions from each device.

The hardware sizing calculator attempts to help you determine the expected size of the database, based on the usage profile.

Application Center

The Application Center is the Mobile Foundation application store. The concept of the Application Center is similar to the concept of the Apple public App Store or the Google Play Store, except that it targets only private usage within a company. By using the Application Center, users from the same company or organization download applications to mobile phones or tablets from a single place that serves as a repository of mobile applications.

The Application Center and MobileFirst Server can be installed on different systems. This installation choice ensures that the setup of the Application Center app store does not affect the performance of Mobile Foundation applications on the server, when it is installed on a different hardware.

The performance of the Application Center is mainly tied to the quality and bandwidth of the network, and to the number of users. The distribution of large mobile applications must be done preferably through 3/4G or WiFi networks. On low networks, the Application Center client tries to solve problems that are related to timeout. If the Application Center cannot reach the server, it tries again several times. The download of an application might also be interrupted when the connection to the network is lost. In such case, the download can restart from where it stopped (for Android, iOS).

The Application Center requires one database schema to store the application catalog and the user information. The size of the database depends mainly on the average size of the applications that are stored in the catalog and the number of users that are connected to the system. This is reflected in the Hardware Calculator document.

Security model

The security model in MobileFirst Foundation is based on the OAuth 2.0 protocol, which has two distinct phases:

• Acquiring an access token
• Using the access token to access protected resources

In the first phase, an access token is generated by the MFP server. In the second, token validation is performed to allow access to protected resources running either on the MFP server or as external resources.

Token generation is more CPU consuming than token validation. Accordingly, the token expiration affects the overall performance of the MFP server, and should be configured taking into account both performance and security considerations.

For example, a very long token expiration will reduce the token generation overhead on the MFP server (thus improving performance), but might be less secured in the sense that client authorisation against the MFP server will be infrequent. On the other hand, a very short token expiration can be considered very secured (since it will require frequent interaction with the MFP server), but it will come at a performance cost.