Scalability and Hardware Sizing for MobileFirst Platform Foundation 6.3
This document contains information about scalability-related parameters of the IBM MobileFirst Platform Foundation, and the scalability tests performed. It serves as a guide to the provided hardware sizing calculator that is used to determine the required hardware for an IBM MobileFirst Platform Foundation deployment.
This document is not a replacement for the IBM MobileFirst Platform Foundation user documentation, the Knowledge Center, which describes an IBM MobileFirst Platform Server architecture, server setup and daily maintenance requirements.Jump to:
- Scalability Tests
- Using the hardware sizing calculator
- Push Notification
- Direct Update
- Database usage and size
- Application Center
ArchitectureThe IBM MobileFirst Platform Foundation topology is based on the following principles:
- MobileFirst Server is installed in the organization LAN, connecting to various enterprise back-end systems.
- MobileFirst Server can be clustered for high availability and scalability.
- It is possible to install different MobileFirst Platform project .war files in the same application server. Each IBM MobileFirst Platform Foundation project configuration must use a different MobileFirst Platform database or schema. The database is used for storing push notification information, devices information, and statistics for reporting and analytics. A single instance of the database is shared by the MobileFirst Platform runtime environment in all MobileFirst Server cluster nodes. It is possible to cluster the database by using the tools and methods that are provided by the database vendor.
- MobileFirst Server is installed behind a web authentication infrastructure, which acts as a reverse proxy and provides SSL.
- The database server and the MobileFirst Server must be installed on separate operating systems.
Scalability TestsIBM conducted a series of scalability tests before releasing IBM MobileFirst Platform Foundation. The following table contains a short summary of the tests performed.
|1||Single user performance test||This test creates a performance baseline for future tests from a typical user scenario.|
|2||Basic stability||This test checks the basic stability and endurance of the server under some load. No crashes, memory leaks, or errors for 12 hours.|
|3||Regression from former version||This test verifies that the system performs at least as well as the former version.|
|4||Multiple user performance test||This test checks the performance of a typical user scenario under typical load.|
|5||Scalability tests||This test proves that IBM MobileFirst Platform Foundation can linearly scale to multiple servers under a single database.|
|6||Endurance testing||This test proves that the server can stay up without memory leaks for a duration of 72 hours under a typical load scenario.|
|7||Stress testing||This test increases the load until the server reaches 100% CPU and starts failing. Reduce the load and see that the server is recovering.|
|8||Recoverability||This test stops various components of the IBM MobileFirst Platform Foundation solution (internet, database, and back end). It starts components again and ensures that the server recovered.|
Test FlowThe test, which is run repeatedly, involves the following actions:
- A typical banking application is simulated. It logs in, makes multiple calls for account list and transaction lists, and then logs out.
- The IBM MobileFirst Platform Foundation application server uses an HTTP adapter to call a simulated back end. On average, a client-server call generates 1.1 requests that are sent to the back end.
- The back end is called via HTTP and returns with a predefined response.
HardwareThe following hardware was used for IBM MobileFirst Platform Foundation application server:
- x3250 M4, Xeon 4C E3-1220v2 69W 3.1GHz/1600MHz/8MB Note: With the specified number of cores, IBM MobileFirst Platform Foundation performs thousands of transactions per second. If your throughput is low, you can use fewer cores than the specified number.
- 32 GB 1333 MHz Note: Not all RAM was used. Use the accompanying calculator to compute required RAM.
- NetExtreme II BCM5790 Gigabit Ethernet
- 2 X 500 3.5in SATA, SR C100
- 2 Intel Xeon 8C 2.3 GHz/20 MB
- 20 GB DDR3 RAM
- 6-7 500 GB 7.2 Krpm SATA disks
- 1 RAID 5 controller
- Simulated by twenty virtual machines that run multiple client threads. Running Ubuntu 10.04 64-Bit 4 GB RAM.
- The client load is achieved with Apache jMeter, an open source server performance testing tool by Apache.
ResultsTests were successful in achieving the goals that are defined in Table 2.1. Specifically, the final test on a single IBM MobileFirst Platform Foundation server ran for three days, without memory loss or performance loss. Throughput was measured at 2,000 requests per second per server with an average CPU usage of 50%.
Using the hardware sizing calculator
About sizingThe hardware sizing calculator is based on a scenario that was tested in IBM Labs. This scenario is based on a real customer scenario. However, MobileFirst Platform Server is a platform that is augmented with server-side developer code. Therefore, the sizing is largely influenced not only by client calls, but also by the efficiency of the server-side code. Hence, you must consider the calculator as a baseline from which you can proceed to customer-specific scalability tests. This document is accompanied by a Microsoft Office Excel spreadsheet. The purpose of this spreadsheet is to help organizations calculate the required hardware for an IBM MobileFirst Platform Foundation deployment. The calculator computes:
- The number of required servers. The server hardware baseline spec is defined in the preceding test.
- The amount of RAM required for each server.
- The MobileFirst Platform Server average request latency.
- The required database size.
- The push notification throughput.
- The analytics server disk space and the number of servers.
What are push notifications?A push notification is an alert that indicates a change or an update that the MobileFirst Platform Server sends to users’ mobile devices. Push notifications are popular with all mobile operating systems. IBM MobileFirst Platform Foundation has APIs that maintain the lists of mobile devices and their tag subscriptions, and send push notifications. IBM MobileFirst Platform Foundation supports push notifications for the following vendors:
Possible architectures for push notificationsTwo architectures are possible for push notifications. They differ by the method that the enterprise back end uses to provide push notification data to the MobileFirst Server:
- The MobileFirst Server pulls data by using a MobileFirst JMS adapter. The enterprise back end puts the push notifications in the JMS queue.
- Push notifications are sent to the MobileFirst Server by a call to an IBM MobileFirst Platform Foundation procedure.
Broadcast, tag-based, and unicast (1:1) notificationsIn IBM Worklight V6.1.0 and earlier, a device or user had to subscribe to an "event source" to receive push notifications. This is called "event source" based notification. In addition to event-source based notifications, IBM Worklight Foundation V6.2.0 supports broadcast, tag-based, and unicast (1:1) notifications.
- Broadcast notifications are sent to all the registered mobile devices of a specific application.
- Tag-based notifications are sent to all the devices that subscribed to a specific tag.
- Unicast notifications, like event-source based notifications, are sent to the devices of specific users. However, no subscription to an "event source" is required. Instead, the "userID" (or) "deviceID" is specified in the newly introduced server-side API WL.Server.sendMessage, which enables to send push notification messages to a specified target.
Testing assumptionsPush providers were simulated with a delay that is typical to these services:
- APNS: 0.05-0.1 milliseconds
- GCM: 20-25 milliseconds
- MPNS: 20-25 milliseconds
For broadcast or tag-based push notifications, a MobileFirst Server can push a notification message to over 22,500 iOS devices, 50,000 Android devices, and 350 Windows Phone devices per second. This is with all of the device types together.
For unicast or event-source based push notifications, an instance of MobileFirst Server can push 1,750 notifications per second to APNS, and 350 notifications per second to GCM and MPNS, when it is under load. The load includes 2,400 regular transactions per second, as well as proportional device calls for subscribing to, and unsubscribing from, the push service.
The CPU and the memory are mostly unaffected for unicast and event-source based push notifications. However, for broadcast and tag-based push, if the number of mobile devices that would register for push notifications is large (for example, more than 100,000), then there could be bursts of 100% CPU usage and higher memory usage for a short duration during which the notification message would be sent to all the registered mobile devices. If your requirements are such that you do not want to delay your mobile users because of push notifications, you might want to have an instance of MobileFirst Server dedicated to handle all the push notification needs, and configure your back-end server to send all the push notification requests directly to this dedicated server.
The database is unaffected, except for the storage of devices and tag subscriptions, as mentioned in section 8. Database usage and size.
Version notesThe testing data for event-source based push notification that is provided in this document relates to IBM Worklight V6.0.0 and onwards. The testing data for broadcast, tag-based, and unicast push notification that is provided in this document relates to IBM Worklight Foundation V6.2.0 and onwards. Customers who use earlier versions of IBM MobileFirst Platform Foundation (IBM Worklight 5.0.5 and 5.0.6) must be aware of known scalability issues. Some of these issues have workarounds. Therefore, if you intend to use push notification with earlier versions of IBM MobileFirst Platform Foundation, contact IBM customer support or see the technote about improving performance of IBM Worklight push notifications.
ReportingIBM MobileFirst Platform Foundation contains the following reporting mechanisms:
- Reports that are based on databases.
- Reports that are based on IBM MobileFirst Platform Foundation Analytics Platform.
Database-based reportsDatabase-based reports are an older style of MobileFirst Platform analytics reports. These reports use a single table to store large quantities of raw data in the database. This table is then aggregated periodically to produce reports. It is recommended that you use the newer Analytics Platform reports listed below. Sizing for these reports is provided in section 8, Database usage and size.
IBM MobileFirst Platform Foundation Analytics PlatformThe Analytics Platform is a WAR file that is bundled with MobileFirst Server. The Analytics Platform is based on file system storage rather than database storage. The hardware that is required to run the Analytics Platform relies on several factors including:
- The frequency of connectivity to the server
- The verbosity set for logging
- The number of days that data is kept before being deleted
There are three distinct data types:
- Analytics collected from server network connections
- Client Logs
- Server Logs
- A device successfully authenticates with the MobileFirst Server instance.
- A device makes an adapter invocation.
- A push notification is sent to a device.
- A client-side API is called to create a custom activity.
- A device successfully authenticates with the MobileFirst Server instance.
- A device makes an adapter invocation.
- A push notification is sent to a device.
- A client-side API is called to create a custom activity.
Server network connections typically take up most of the disk space in the Analytics Platform. Device authentications and adapter invocations are the most common types of network connections that are recorded in the Analytics Platform.
When you estimate hardware requirements, it is important to take into account the number of devices that will be connecting to your server and the number of adapter invocations each device will be making.
Client logs are sent when the client app is optionally configured to collect and send logs.
By default, client logs are set to “FATAL”, which means that only crash logs are forwarded to the MobileFirst Server instance. With this default setting, client logs use less space than analytics data that are created by server network connections.
Changing this level and adding custom client logs drastically affects the disk space used by client logs. With more custom logs and higher verbosity log levels, client logs can consume much more disk space (by several magnitudes) than server network connections. When you estimate hardware requirements, you must take into consideration the size client logs and the frequency of sending these logs.
Server logs, which are produced by MobileFirst Server and are saved to disk, are also forwarded to the MobileFirst Server by default. Server logs will have the least impact on the disk space sizing.Purging data
On IBM MobileFirst Platform Foundation Analytics Platform, you can configure automatic purging of data. For more information, see the IBM MobileFirst Platform Foundation user documentation. Different data types can be purged at different intervals. This means that if you wish to keep a year's worth of server network event data but only three weeks of log data, you can set these values separately.
Log data takes up more space than server network event data. Depending on the configured verbosity, MobileFirst Server could be sending a very large amount of log data. If disk space is limited, you must consider purging data more frequently or adding nodes to your Analytics Platform cluster.Clustering
You can add a new node to the analytics cluster with zero downtime. Therefore, you might want to start with a single node and add nodes as necessary. The total number of nodes that you can add to the cluster is equal to the number of shards that were created when the first node was installed. For more information about shards and how to set up a cluster, see the IBM MobileFirst Platform Foundation user documentation.
Direct Update scalability challengesWhen the administrator triggers direct update by redeploying an app to the MobileFirst Server instance, it creates a surge of web resources download requests from the connected devices to the MobileFirst Server. The IT organization has to be ready for it. Factors that affect direct-update:
- # of concurrent direct-update requests. The # of downloads and especially the frequency of usage of the application (e.g. once an hour vs. once a month) will determine the number of requests the MobileFirst server can handle when the new version is pushed.
- Application size. There is an accumulative size limitation on the ability of the MobileFirst server to push data to clients.
- Difference in content. Starting with MobileFirst Platform 6.3, the difference in content between the old and the new version is also important (see below).
Direct Update lab results
An instance of MobileFirst Server can push a total of 250 MB/s. As an example, if the application size is 5 MB, MobileFirst Server can serve 50 direct updates per second. This means that if the cluster contains four MobileFirst Server nodes, the cluster can serve 200 direct updates per second. This is the limit for a single server rather than the network limitation.
Note: You must also validate that the network bandwidth is not the bottleneck.
Starting with IBM MobileFirst Platform Foundation V6.3, direct update was improved to allow users to receive only the web resources that have changed between updates, instead of downloading the entire web resource package. This enhancement reduces download time, conserves bandwidth and improves overall user experience. When the delta size is between 10k to 300k the MobileFirst Server TPS (Transactions per second) rate remains almost the same (6% decrease) with an additional 200 to 300 direct update requests per second.
Notice: The MobileFirst server will only maintain changes between the current release and the one before that. This feature is available only for users who are upgrading an application with web resources that are only one build behind those in the application now being deployed and for Android and iOS only.
The customer IT infrastructure has to be ready for both incremental direct update and full direct update requests. Subject to the exact customer scenario, most of the updates are likely to fall under the incremental scenario.
Small B2C deployments and most B2E deployments can use the above product features as is. As your application grows, you will need to use other means.
As a first approximation, you can get ready by adding servers and increasing outgoing network bandwidth. A more scalable solution would be to use a CDN.
Using a CDN
The best scalable solution for Direct Update, both full and incremental, is to use a content delivery network (CDN) in front of the MobileFirst Server instance. When caching the direct update URL, the request is served from the CDN and does not get to MobileFirst Server. The expiration policy must be specified in the CDN. The CDN's primary use is for full direct update requests but it may also for incremental direct update requests especially if the factors mentioned above influence your system.
For more information about Direct Update, see the user documentation topic Serving direct update requests from CDN page.
What is Geolocation?
Location services in IBM MobileFirst Platform Foundation provide you with the opportunity to create differentiated services that are based on a user location, by collecting data about geolocation and WiFi, and by feeding the location data and triggers to business processes, decision management systems, and analytics systems.
Application code on the mobile device controls the collection of data from device sensors. The collected data is referred to as the device context. When a change occurs in the device context, such as a change in the geolocation of the device, or the fact that it has just entered a WiFi zone, triggers can be activated. The triggers specify that an action should occur: either a callback function is called, or an event is sent to the server, based on the device context.
Events are created by triggers and application code and include a snapshot of the device context at the time of their creation. Events are buffered on the client and are transmitted to the server periodically.
To handle the events, the server uses IBM MobileFirst Platform Foundation adapter application code. This code sets up event handlers on the server, which filter event data and pass matching events to a callback function. The code also accesses the device context of the client, that is to say its location, and WiFi network information. It then sets an application context. Server activities and received events are logged, together with the device and application contexts, for future reporting and analytics.
How Geolocation affects the server performanceAnalytics and reports
Additional entries go to analytics and reports. Entries are added each time an event handler is run. Additional data passed when invokeProcedure is called
Additional data is passed when the
invokeProcedure method is called from the device, to transfer the device context of the client to the server. The amount of data that is transferred is usually in the tens of bytes. This amount of data might be larger, depending on how the WiFi acquisition of the device is configured and what WiFi access points are visible to it. Events are also transmitted occasionally, according to the event transmission of the client.
Database Usage and Size
IBM MobileFirst Platform Foundation database usage profileAn IBM MobileFirst Platform Foundation project is a set of apps, adapters, and project-specific configuration. A project is associated with a WAR file.
IBM MobileFirst Platform Foundation requires one database schema per MobileFirst Platform runtime environment, which is called WRKLGHT, and one database schema for all MobileFirst runtime environments, which is called WLADMIN. The WLADMIN database stores all the management data, for example the applications and adapters, and their state. Optionally, you can add a database schema for database-based reports, which is called WLREPORT. The reports database is used only with the classic IBM MobileFirst Platform Foundation reports. For more information, see section 5, Reporting.
In addition, when the Application Center is used, one more database schema, called APPCNTR, is used. This database schema is global, which means that it is not project- dependent.For Oracle and DB2, the default configuration uses three schemas instead of three actual database instances. The IBM MobileFirst Platform Foundation database is rather small, with a number of exceptions:
- Push notification information: the list of devices that registered for push notifications, and the subscriptions from each device.
- SSO state information: SSO (Single Sign-On) is the ability for business customers to have multiple apps from the same organization. When they log in with one app, they automatically log in to the other device. Turning on the SSO feature puts a larger load on the database because SSO state information is read and updated to the database at the start of each client access to the server.
- The Reports database contains a single table that is called APP_ACTIVITY_REPORT, which is append-oriented. Each transaction that is called from a mobile device generates an INSERT statement into this single table. This data feed accumulates a large amount of data on what the user does. The data is aggregated at specific intervals. The default interval is 20 minutes.
- Device licensing is used to count licenses and control user access to applications. This feature is used with B2E customers (internal enterprise apps) but is enabled by default for all customers and must be explicitly disabled for B2C. The feature uses a few more database queries and stores additional information in the database.
Database sizeDatabase size is determined almost solely by the size of the data feed that is described earlier. The hardware sizing calculator attempts to help you determine the expected size of this feed, as well as push notifications and SSO. Notes on the report tables:
- The application developer might also choose to add custom log messages, in which case the table size becomes bigger.
- The size of the aggregated tables is negligible when compared to the raw data.
The Application Center is the IBM MobileFirst Platform Foundation application store. The concept of the Application Center is similar to the concept of the Apple public App Store or the Google Play Store, except that it targets only private usage within a company. By using the Application Center, users from the same company or organization download applications to mobile phones or tablets from a single place that serves as a repository of mobile applications.
The Application Center and MobileFirst Server can be installed on different systems. This installation choice ensures that the setup of the Application Center app store does not affect the performance of IBM MobileFirst Platform Foundation applications on the server, when it is installed on a different hardware.
The performance of the Application Center is mainly tied to the quality and bandwidth of the network, and to the number of users. The distribution of large mobile applications must be done preferably through 3/4G or WiFi networks. On low networks, the Application Center client tries to solve problems that are related to timeout. If the Application Center cannot reach the server, it tries again several times. The download of an application might also be interrupted when the connection to the network is lost. In such case, the download can restart from where it stopped (for Android, iOS).
The Application Center requires one database schema to store the application catalog and the user information. The size of the database depends mainly on the average size of the applications that are stored in the catalog and the number of users that are connected to the system. This is reflected in the Hardware Calculator document.▲