The True Cost of Open Source Load and Conformance Testing
The True Cost of Open Source Load and Conformance Testing
Since the early days of computer software, collaborative developers have been working together on systems that would be available at little to no cost and continually built and improved by the community. Much like recipes that are passed along and improved, open source software benefits from a large community taking the software, expanding it, and sharing those expansions. In the modern computer era, that has benefited individuals in the form of free software tools like Linux, Mozilla, GNU, and many more. But can this work for businesses?
Several articles have been written on the topic, highlighting benefits such as no cost, and “try before you buy” options while also promoting the idea of thousands of developers working on the project. While all this sounds fantastic, the reality for businesses is that open source software isn’t always a fit. For telecom carriers and vendors who need to perform conformance, load or performance testing on their network or devices, it can be a high cost option.
Open source technologies best fit in environments where the organization is able to allocate time, resources and hardware for the project. Companies that understand the true costs of open source tools have already built their budget for the project. For others, the additional costs may be unexpected and cause the project to go over budget, limit the features or increase risk.
If you are considering an open source tool for your testing, ensure that you understand the hidden costs such as the ones that follow:
High Initial Costs - Businesses often invest time and expend engineering resources just to to determine if the solution is suitable. The business could rely on word of mouth, but few will and more often will invest to build, install and allocate hardware for a solution that in the end may not meet their requirements. The solution may have been free, but the evaluation has a cost.Inflexible - Most open source solutions target one or two specific protocols. Few of the technologies are cross-domain and those that are have a narrow scope. As the project requirements scale, the business will need to acquire more tools, meaning different interfaces, API's, etc; all that must be internally managed.Limited Support – Open source technologies have come a long way when it comes to support. With knowledge bases, forums, wikis, etc, the answer is probably out there, but finding it could take longer than expected, and the answer may not be specific enough to act upon without engaging technical resources. This can be devastating to a time-sensitive project. Because of this, internal expertise will be needed, and IT resources expended to support the product.Upgrade Issues – Once the open-source technology has been modified to fit for a specific business, when the business needs changes the tool will have to be changed again. If the community already developed the solution, great, but more likely the cost to do the development will be taken on by the individual company. Investments will need to be made to keep the product up to date.Missing Capabilities - Often, the business will come to a critical requirement that the open source technology cannot support. This means engaging internal engineering or working with the development community to develop the features. The community may not provide the capability exact as specified, creating costs to perform engineering changes.Why share? – Businesses leveraging an open source tool are likely investing significant sums of money to make that tool work for their situation. There is little incentive for the business to pass on improvements they have spent time, resources and money to develop. For companies that do want to contribute, the changes made may not be accepted, leaving the business with non-upgradeable version of the tool.
Tools like Valid8.com are purpose-built for telecommunications providers to test networks, devices and applications. The company works closely with customers and the industry to ensure that the technology offers a full-suite of solutions. The business is constantly innovating products to ensure a migration path for new capabilities in line with the rapidly changing market. The long-term return on investment is much higher than a “free”, open source technology. In addition, Valid8.com can be integrated into an environment where open source tools are already in use.
While a purpose-built tool may require some level of custom changes, because they are managed by a single organization, they rarely impact upgrade paths or require an internal IT or engineering resource.
