Often companies have to conduct technical as well financial evaluation & due diligence of SaaS products before deploying to meet their requirements or while acquiring. It may be also for competitive analysis.
Some of the important points of consideration are:
- Financial viability
- Functionality mapping
- Security
- Scalability
- Third party Dependencies – like APIs and SDKs
- Licensing issues of using third party Libraries, SDK’s Frameworks (say React)
- Code Quality
- Maintanibility
- Versions of the softwares, tools, Programming languages and Databases
- Version controlling & Release management strategy
- How and where the database schema is stored
- Recurring cost of APIs, SDKs if there is any, upgrade paths of SDKs
- OSes deployed and available upgrade paths
- Technical Architecture document
- Test plan and test cases (white box as well as blackbox)
- Test automation suite & any licenses applicable
- If there mobile applications involved, device compatibility should be considered in addition to all the above
- Email delivery and IP reputation
In the above check list, first 2 items are out of scope for this discussion and few other items will be discussed here.
Security
One of the most important aspects of today’s multi-tenant applications is the security impact. A security audit at the code level & application level must be performed to ensure that the application will withstand the real world threats. A quick way to ensure this is to perform tests with OWASP’s ZAP and Nessus.
Code level audits should be ideally part of the continuous integration. If such a system is not available, which is mostly the case, static code analysis tools can be used for the purpose. A multi-language supporting platform is quite easy and CodeClimate.com is of great help here.
Scalability
Today’s applications requires “internet scale” & performance and capacity testing can reveal not just the scalability aspects of the application but it can also high light security vulnerabilities. Responding to boundary values also will be tested while a performance test is carried out. API layer must be capable of horizontally scaling in most of the scenarios and persistance at the database level or at the application level should not be a bottleneck here. (This is often the case with JSP codebases and LAMP stack.)
API performance testing automated with a tool like Postman and Jmeter can be a quick method to perform the testing. Along with the performance testing, conducting the security audits can be put any application under stress.
Dependencies
A list of all the third party dependencies and libraries should be made available to the buyer / evaluation team. Versions of the third party dependencies, licensing etc must be considered. Another aspect for modules / libraries is whether they are maintained by the original developers or are they abandoned. This can be very important for Perl CPAN modules, Ruby Gems and NPMs. Further NPMs are observed to have vulnerabilities and sources should be audited even for unwanted payloads or backdoors. ClamV, Immunet etc can be used for the purpose.
Licensing Issues
This is one of the most common and important aspects thats gets over looked. The on going discussion around React’s dual licensing and the request from Apache Foundation to make the licensing Open Source is an example. Though the implications of React’s licensing are not clear, this is a common scenario that we encounter. GPL violations (Free Software), Infringement of Patents, Trademarks etc can drag one into endless legal and financial trouble. This makes it very important perform a code level audit (if possible) to get an assurance from the vendor regarding licensing especially in the case of an acquisition.
This list is not an all inclusive one but this can be used as a high level check list before evaluating products. If you are looking for assistance in such a scenario, feel free to reach out to us
