Which MAST tool is DevSecOps ready?

The Mobile Application Security Testing (MAST) tool intended for DevSecOps shall automate the execution of routine test tasks and find the maximum number of vulnerabilities with a minimum effort. This is exactly the goal of DevSecOps tools that use scan automation to reduce labor costs. Automation implies the fastest and most complete receipt of information about the vulnerabilities of a mobile application with minimal human involvement in the process.

A “DevSecOps-ready” MAST tool shall perform automated security testing and support integration with different types of DevOps tools. If the requirements for security testing automation and readiness for integration with software engineering tools are met, a MAST tool can be considered ready for use in modern mobile application development systems.

The main conditions for “DevSecOps ready” for a MAST tool are:

• Automation shall work end-to-end
• The MAST tool shall be prepared for smooth integration with DevOps tools, including enterprise and open source software engineering tools

End-to-end Automation

All DevSecOps is based on automation of all operations, including application security scanning. DevSecOps requires security checks to be included in the CI/CD process.

In order to automate the security testing of mobile applications, the tool shall completely cover and automate its target MAST practices (see the “MAST Practices” article next to this article for details). This allows you to find the maximum number of vulnerabilities in a mobile application without human participation.

In order to provide high test coverage for all the practices, the MAST tool shall record test scenarios based on the business logic of the application, as Mobix does, or provide integration with existing automation tools (for example, Appium, iOS Automator, etc.). These test scenarios (or auto tests) can then be played back for security checks as part of the CI/CD process. This is necessary to gather enough test data to ensure good test coverage for each test cycle. Without this, the security check cannot be considered complete. In the manual mode, providing such coverage takes a lot of time and effort, so test automation is a must.

In dynamic testing, simply launching an application without user interaction and analyzing the data generated by the application is often referred to as automatic application analysis. Just application launch without interaction by a user is not enough to say that the MAST tool is really DevSecOps ready. In this case, we have no control over which test cases are executed. It is not clear what is the test data that the application receives as a result and what test coverage is achieved with such tests. Therefore, the target MAST practices will not be fully tested. In particular, an incomplete set of test scenarios has a direct impact on the results of the IAST and DAST practices. Also, the interaction of the application frontend with the backend will not be tested properly. Thus, the API ST practice will not provide representative test results too (see the “MAST Practices Covered in Mobix” article next to this article for details). That is why the MAST tool shall automate the user's work with a mobile application, as is the case with Mobix.

Automation of test case creation is also important in terms of test creation speed and labor costs. DevSecOps involves the rapid creation of new auto tests and correction of existing ones if required. It matters how long it will take to create these auto tests – 15 minutes by a less skilled tester or 8 hours by a test automation expert. Mobix provides a new level of abstraction during auto test cases creation. It records auto tests using Artificial Intelligence (AI) methods. Test engineers do not need to know the implementation details of the mobile application under test to write auto tests. The test engineer only needs to interact with the application as a user. There is no need to write test scripts at all. Mobix records and validates auto tests based on user actions. With Mobix, auto tests generation takes significantly less time and can be performed by less skilled staff.

Integration with DevOps Tools

As mentioned above, the first requirement for the MAST tool developed for DevSecOps is end-to-end automation with the most comprehensive vulnerability coverage. The second important point is whether this tool fits easily and is fully functional into the DevOps process.

The modern development of mobile applications is mostly based on the Agile methodology. The typical duration of software development iteration (sprint) is two or three weeks. During this time frame, it is necessary to find a vulnerability, fix it, and create a new version of a mobile application to ensure that the vulnerability was correctly fixed. If security is checked just from time to time, it would not work. Vulnerabilities will go to the production version of the mobile application and will be present in the product for at least several next development sprints. Security check needs to be performed regularly and be included into the CI/CD process. This enables the shift from DevOps to DevSecOps. This shift is possible only if security checks performed by MAST tools are integrated into CI/CD. That is why testing automation and easy integration of MAST tools into a continuous software development process (DevOps) are required. Such a process reduces reputational risk, increases the speed of identifying security problems, and reduces the cost of fixing them.

Plugins, CLI and REST API for Integration with DevOps Tools

The most convenient type of integration is native integrations with DevOps tools. However, since there are many such systems, the plugins for native integrations cannot be created for all of them. At the choice of the MAST tool vendor, the plugins can be prepared for integration with the most important DevOps tools.

There are a lot of DevOps tools, and they are all very different with different APIs and with different degrees of readiness for integration. Therefore, in addition to the plugins, the MAST tool shall be prepared for custom integration with other DevOps tools. MAST tool needs Command Line Interface (CLI) and a fully-functional documented REST API to integrate with all other DevOps tools. The command-line utility should work on any operating system. If the functionality of the CLI utility is not enough for integration, then you should use the REST API for custom integration with it.

If a MAST tool has the plugins, the CLI utility and a fully-functional REST API, it can be integrated with the existing DevOps process that is already built for application development. The plugins, CLI and REST API shall enable the automatic execution of all those MAST practices that can be done manually with this tool. This enables a simple MAST tool connection to the CI/CD pipelines and a shift from DevOps to DevSecOps.

Types of DevOps Tools to Integrate

Besides CI/CD tools, MAST tools shall support integration with different types of DevOps tools. As an example, let’s consider the list of tools integrated with Mobix, grouped by categories:

1. CI/CD tools: TeamCity, Jenkins, GitLab CI.
2. Defect Tracking Systems: Jira.
3. Distribution Systems: AppCenter, HockeyApp, Nexus Repository Manager.
4. ASOC (Application Security Orchestration and Correlation) tools: Maverix.
5. Compatible data traffic format: Mobix provides collected raw testing data in traffic formats compatible with enterprise MAST tools: Acunetix, Netsparker, AppScan, WebInspect.
6. Custom Integrations: Mobix is ready for custom integrations via REST API and CLI.

Role of Distribution Systems

The DevOps tools listed above are familiar to application developers, except for distribution systems. Let's explain their role in a little more detail. The use of distribution systems is specific to mobile applications. These systems enable the automation of mobile applications distribution to devices for testing purposes. For Mobile AST, distribution systems are part of the DevOps tools. The distribution systems are triggered by the notification of a new version of the application for the rapid delivery of the new version to devices. Communication with distribution systems can take place in both directions. The MAST tool can receive information as well as add comments or tags from its side.

Automation of Test Reports Creation

An important factor is how the MAST tool feeds the results of its work back into the DevSecOps process. The tool needs to provide feedback that can be used to control the entire DevSecOps process.

Automation of test reports creation includes providing a security scan report in different formats - both human-readable (PDF, HTML, MS Word, etc.) and machine-readable. The machine-readable format is required so that the DevSecOps system can make decisions automatically and as quickly as possible based on the results of ongoing security testing. For example, it could be a decision to continue CI/CD pipeline operation or to cancel it due to vulnerabilities found. This mechanism allows you to effectively organize the work of the DevSecOps system by setting up fail conditions to either send notifications of scan results or stop the entire build.

In addition to providing a test report in the CI / CD tool, it can also be exported to the tool for continuous inspection of code quality. For example, if you are using SonarQube, you can add a test report to it from the MAST tool. In SonarQube, you can also define a common quality gate for the results of the MAST tool scan and the SonarQube check.

A custom test report generation shall also include different types of reports - human-readable and machine-readable in a format for export to the CI/CD tools and external monitoring tools.

Summary

Of course, there are still many different requirements and recommendations for the MAST tool, which can be called DevSecOps ready. We haven't covered all of the aspects in this article and have focused on two main features that a DevSecOps ready tool should have. In our opinion, the two main criteria for automation of the recording and replay of user actions and for readiness for integration with DevOps tools that are highlighted in this article are crucial. A MAST tool that does not meet these two criteria cannot be called DevSecOps ready. Without it, the integration will either be very difficult or incomplete. Either the tool cannot find all vulnerabilities, or the integration process is very time-consuming. Some other features, like the collection of raw data for further analysis or integration with additional services are considered important but not critical.