Mobile Test Execution Grid
Enterprises today face the challenge of delivering high-quality mobile applications across a fragmented device ecosystem. While public device farms offer one solution, they often come with high recurring costs (nearly $250 per thread per month) and significant limitations, such as the lack of native support for two-factor authentication (2FA), SMS retrieval, GPS manipulation, and deep device-level interactions. To overcome these challenges, the Mobile Test Execution Grid provides a cost-effective, flexible, and enterprise-ready alternative that can run on-premises or in the cloud.
At the heart of this architecture is the Server, which works closely with the Scripting Bot. These two components are co-located and can be hosted either in the Nogrunt cloud or within a customer’s private cloud, depending on security and compliance needs. The Scripting Bot translates test scripts into actionable steps and communicates with the Node Controller on distributed test nodes.
The Node Controller plays a pivotal role by managing connected devices or emulators. It not only executes scripted actions on the mobile interface through the Execution Engine, but also provides direct access to native phone functions such as GPS, network settings, battery state, SMS retrieval, and two-factor authentication flows. This ensures that end-to-end user scenarios, including secure login flows and device-level behaviors, are reliably tested—capabilities that are not natively supported in most device farms.
The Execution Controller orchestrates the overall process, ensuring that scripts are scheduled, distributed, and run efficiently across multiple nodes. This design enables enterprises to keep testing environments elastic and adaptable to real-world use cases.
Parallel Execution and Horizontal Scalability
One of the most powerful features of the Mobile Test Execution Grid is its ability to run tests in parallel. Instead of sequential execution, enterprises can distribute workloads across multiple devices and emulators, reducing overall execution time and accelerating release cycles. The grid is horizontally scalable—customers can choose how many parallel threads they want to run simultaneously and expand capacity simply by adding more devices or emulator instances.
This flexibility ensures that organizations can strike the right balance between speed and cost. For example, a combination of physical devices and emulators can be used: emulators are efficient for scaling large regression suites, while real devices are critical for validating hardware-level interactions, such as biometric authentication or push notifications.
For iOS testing, Apple’s ecosystem restrictions require the node hosting the execution to run on macOS. The Mobile Test Execution Grid seamlessly accommodates this, allowing mixed environments where Android tests run on commodity hardware while iOS tests are executed on Mac nodes. This makes it practical to scale across platforms without over-relying on costly third-party device farms.
Multi-Device and Multi-Version Support
Modern apps must work seamlessly across a diverse ecosystem of mobile environments, and the Mobile Test Execution Grid is designed with this in mind. The grid supports:
- Android and iOS platforms, ensuring coverage across the two dominant ecosystems.
- Physical devices for high-fidelity testing, covering scenarios like 2FA, GPS, network switching, camera, and SMS verification.
- Emulators and simulators for scalable regression and non-hardware-dependent testing.
- Different OS versions and device models, addressing fragmentation in the mobile ecosystem.
- Multiple app versions, allowing enterprises to validate production releases, beta builds, hotfixes, and custom client builds in parallel.
By supporting such a wide range of device configurations and app versions, the grid enables enterprises to replicate real-world conditions and ensure that applications deliver consistent performance and reliability across the devices and builds their users actually experience.
Why On-Premise Grids Excel at 2FA and Device-Level Testing
Public device farms are optimized for generic app validation but fall short when it comes to deep device integration. Critical enterprise workflows often require:
- Two-Factor Authentication (2FA): Automated handling of SMS-based or app-based OTP flows is unreliable on device farms, as they do not provide direct access to the device’s messaging or notification systems. On an on-premise grid, testers can directly capture and validate OTP messages, ensuring secure authentication paths are thoroughly tested.
- SMS and Call Retrieval: Many enterprise apps depend on phone number verification. On-premise nodes can access and validate these messages natively, something device farms restrict for privacy reasons.
- GPS and Network Simulation: Enterprise apps that use location-based services or need to be tested under varying network conditions benefit from direct control of device sensors, which device farms rarely expose.
- Battery and System Events: Real-world conditions like low battery prompts, background app behavior, and power-saving mode can be simulated only when enterprises have control over the devices themselves.
By hosting a mobile grid on-premises (or in a private cloud), enterprises gain full control over device functions, ensuring they can automate scenarios that directly impact security, reliability, and user experience.
Balanced Strategy: On-Premise Grid + Device Farms
While the on-premise grid is ideal for the majority of testing—covering regression, functional flows, 2FA, and device-level scenarios—enterprises can still benefit from limited use of public device farms. Before a production release, customers can procure a small number of threads on a device farm to validate their app across a wide variety of niche devices, OS versions, and screen sizes that may not be available in-house.
This hybrid strategy offers the best of both worlds:
- Cost efficiency by running most tests on the on-premise grid without paying high per-thread costs.
- Comprehensive coverage by leveraging device farms only for last-mile validation.
Conclusion
This distributed execution model has two major advantages. First, it helps enterprises keep costs under control by eliminating the need to rely on expensive external device farms that charge per thread. Second, it offers deeper coverage of real-world mobile interactions by enabling direct communication with the devices themselves, including features critical for modern apps like 2FA, push notifications, app-version regression testing, and system-level events.
In summary, the Mobile Test Execution Grid combines scalability, cost efficiency, and functional completeness. By placing the server and scripting components in the cloud while distributing execution across enterprise-controlled nodes, organizations gain the ability to run high-fidelity mobile tests in parallel, on-premises or in hybrid setups, without compromising on security or depth of functionality.
| Criteria | On-Premise Execution Grid | Cloud Device Farm |
| Cost | One-time infra + scaling costs; no per-thread subscription. Much cheaper long-term. | Expensive recurring cost (~$250 per thread/month). |
| Scalability | Horizontally scalable — add more nodes to increase parallel executions. Flexible mix of devices & emulators. | Limited by purchased threads. Scaling requires buying more subscriptions. |
| Device Control | Full access to devices: GPS, battery status, SMS retrieval, 2FA, network throttling. | Restricted access; most farms do not support low-level device features or 2FA. |
| App Versions | Can test multiple app versions (production, beta, hotfix) simultaneously in one environment. | Supports multiple versions, but limited by number of threads and environment restrictions. |
| Environment | Can be hosted in Nogrunt Cloud or customer’s private cloud/data center. Runs securely inside enterprise infra. | Vendor-managed, multi-tenant cloud environment. |
| Multi-Device Support | Parallel testing across different device models, OS versions, and emulators in one grid. | Wide device model availability, but limited in parallel unless more threads purchased. |
| iOS Testing | Requires macOS nodes but fully supported in enterprise setup. | Provided by farm vendor; no infra setup required. |
| 2FA & Security Testing | Fully supported (SMS, push notifications, authenticator apps, device-level APIs). | Not natively supported — major gap for enterprise apps requiring 2FA. |
| Network/Real-World Testing | Full control over network throttling, battery simulation, and GPS/location switching. | Mostly not supported or limited in scope. |
| Best Use Case | Daily regression, functional, and integration testing. Cost-efficient for enterprise-scale. | Pre-release validation on wide variety of devices and OS combinations. |
