8 tips for avoiding vendor lock in
Vendor lock-in makes a customer dependent on a supplier for a product or service, and unable to use another vendor without substantial switching costs or inconvenience. While standards, bodies and an increased focus on interoperability have helped reduce the threat of application lock-in, web site and web application development can still be a minefield. In this article, we will look at eight ideas that can help companies avoid vendor lock-in, intentional or otherwise, when setting up new sites or systems.
Executive summary
Vendor lock-in makes a customer dependent on a supplier for a product or service, and unable to use another vendor without substantial switching costs or inconvenience. While standards, bodies and an increased focus on interoperability have helped reduce the threat of application lock-in, web site and web application development can still be a minefield.
In many cases, site owners only realise the dangers of lock-in once they are trapped with rising support and maintenance costs, slow and expensive change processes, and the prospect of prohibitive costs to develop an equivalent solution from scratch.
Vendor lock-in doesn’t have to be an intentional attempt by the vendor to restrain their client’s future choices. Plenty of companies are trapped in arrangements simply because their technical solutions were supposed to be short term. Lack of long-term planning can lock a customer into a vendor without that intent existing on either side.
Similarly, sometimes vendors just don’t have the time or the resources to take on work the client wishes them to do at a given moment in time. If the site owner can’t bring another agency in to do the work they are clearly suffering from lock-in.
In this article, we will look at eight ideas that can help companies avoid vendor lock-in, intentional or otherwise, when setting up new sites or systems. These are:
- Separate suppliers
- Integrate rather than extend
- Retain ownership of your data
- Retain ownership of intellectual property
- Look for mindshare
- Document the processes
- Prefer extensibility to features
- Get a second opinion
Following these guides, it is possible to realise the benefits of working with external suppliers without fear of lock-in.
Separate suppliers
One common scenario is for a company to contract a single supplier for a number of technical services, often driven by cost savings, or to simplify supplier relationships. When an issue arises with one of the services and the company wishes to improve or replace it, they find that they are locked in.
This can be due to technical or commercial dependencies, but in either case they are prevented from making improvements or switching to another system without significant overhead of money, time or hassle. Often the problem is made worse by the fact that the supplier’s other services are still delivering significant value, and the company would prefer the status quo to remain for these offerings.
This can be avoided by trying to separate suppliers, where practicable. One straightforward example might be separating hosting of a web application from its development. This makes the host of a company’s application a supplier to that company, rather than to their development partner. This ensures that the host’s first duty will be to the company, rather than to the (potentially conflicting) needs of the developer, and makes it easier to change one or other supplier should the need arise.
Another area that might benefit is support, where someone other than the original developer handles maintenance and operational support. Even where the packaging of a suite of services is preferential, steps can be taken to ensure the supply of the services is independent – by taking advantage of the integration features described in the next suggestion.
Integrate rather than extend
Web service APIs (Application Programming Interfaces) are now a common feature in many commercial and open source applications. This means that all or part of the functionality available to the users of the application is also available for machine to machine usage.
By using the protocols and standards of the web to define this interface, these services are language and platform agnostic; an application written in one language or on one operating system can be accessed by a system written completely differently. Data is transferred in a common format such as XML or JSON, and the code bases of the two systems remain completely independent.
Web services come into their own when a site owner wants to extend an existing web site or application, perhaps adding social features, mail-out functionality, moderation systems or a host of other improvements. In many cases custom work is required that increases the complexity of the system, and adds expensive requirements to any potential future replacement. For example, if CRM (Customer Relationship Management) facilities are added to a content-oriented site, replacing the content site at a future date means also having to replace the CRM – whether or not it is performing effectively as it is.
If instead the CRM functionality is provided by an existing system, like SugarCRM2, then the content site could integrate with it via Sugar’s web services, ensuring they remain independent. Replacing the content site later would simply mean reintegrating with Sugar – a task, but a much less significant one. Any custom work that had been done on the CRM would remain unaffected, and vice versa, if a new CRM is required.
The same logic holds true of integrating with a SaaS (Software as a Service) system like Salesforce3. Using the web services available in these systems decouples one piece of functionality in a web application from another, making the whole application more flexible and easier to modify. Tight coupling between different applications makes lock-in much easier, increases the complexity of the code base, and makes the flexibility of a system equal to the least flexible component of that system.
Retain ownership of your data
One risk with using a software-as-a-service, or any hosted system, is that potentially critical information is under the control of another company. Even with externally developed bespoke systems businesses can be vulnerable to the same problem if the data is held inside a black box – the application has stored it in its own way, and it can be a major task to get access.
This is one of the key factors in traditional vendor lock-in, and ensuring that data remains accessible should be made a high priority. There are broadly two approaches to the problem: control the data directly, or ensure there is a reliable way of extracting the data in a usable format.
For example, an email contact service might have the option to have user registrations happen directly through their system, or to use a web service and process registrations on a separate site. Though the first option might be easier and quicker to develop, the second means that data can be stored in a manner of the business’s choosing, and then sent on to the email contact service.
Even with a local system where the storage is well documented, having a method to export the data to standard spreadsheet, database format, or XML might make future migration significantly easier and cheaper. It is important to establish procedures for extracting the data, ensuring the exports still work, and checking that the format is well documented, as the critical moment when the data is needed is the worst time to discover that the export functionality hasn’t worked since the last update, or the perfectly exported data is encoded in an unknown format.
These kinds of requirements should be verified as part of the acceptance criteria of any new work.
Retain ownership of your intellectual property
In this case, the property in question is usually in the form of source code or design work. There is a danger with bespoke development of moving away from a supplier only to discover that they own critical parts of a system, forcing the site owner to negotiate continuing use of those subsystems. This can be avoided by a contractual agreement that the client owns all custom work, or alternatively all custom work is released under a license that allows free reuse and extension.
Even once the legal and commercial provisions have been made, if a breakdown in the relationship between site owner and vendor does occur, or the vendor ceases trading, it’s reassuring to have guaranteed access to any intellectual property. Running a source control system and wiki and obligating development partners to store project related information and code in them is the best way to ensure that the work generated for a project stays under the control of the business.
Source control systems like Subversion or Perforce are a way of tracking different versions of an application’s source code and allowing developers to co-operate efficiently. Most developers will be using one of these systems already, and should be happy to work with one supplied by their client, as long as it is well managed, maintained and backed up. A dedicated provider or a regular hosting supplier could host the system, but partner development agencies should not be hampered by the source control so it’s worth discussing the choice of system and access controls with them.
A wiki like MediaWiki or Confluence should be hosted similarly, and provide a central point to store documentation, related information, and any other assets. These ensure that the information is available whenever needed, and provide a useful starting point for future partners and suppliers. However, a sprawling, unmaintained wiki is often less of a help and more of a hindrance, so users from both parties should be encouraged to keep information up to date, and sensibly structured.
Look for mindshare
It’s relatively easy to look at proposed costs and features when considering proposals, but it’s hard to see what the cost of changing developers on a system will be, or of hiring another agency to support the project after completion.
As part of the RFP process, ask developers whether they will be building the site from scratch, or use some underlying system. Most companies won’t be building in ‘raw’ PHP, Ruby, .Net or Java - they will make use of open source or commercial systems that provide many functions ‘for free’, and implement or extend these as necessary to meet the needs of the project.
These systems broadly come in two flavours: frameworks or applications – though there is some overlap between the two.
Frameworks are component libraries that provide a set of functionality and allow the developer to rapidly create the features they need. Usually they do not work ‘out-of-the- box’, but are easy to tailor to specific requirements, with less overhead than a full-blown application. Common frameworks include Symfony and Zend Framework9 for PHP, Ruby on Rails for Ruby or Django for Python. Some companies have internally developed frameworks that fulfill a similar role.
Applications are generally higher level, and provide some sort of running system straight away. They are usually harder to customise than a framework, but offer more features with minimal development. This can range from popular open source publishing applications like WordPress to large, expensive content management systems like Documentum.
The base of the proposed system is a huge influence in how easy it is to find other development partners to work on it. Getting an idea can be as simple as searching for ‘X developer’ and seeing if there are local agencies familiar with it. It’s also worth asking development agencies for examples of other work in the described framework, as another good rule of thumb is that an agency more specialised in one framework or system will be more likely to use the standards and techniques best fitting to it. These should be reasonably consistent across developers, meaning another specialist should be at home working on the resulting web application.
Should a proposal be entirely bespoke, not be using a framework at all, or only using one developed in- house, it will be much harder for another agency to take over the work. This isn’t necessarily a deal breaker, should the rest of the proposal be fulfilling all the requirements, but it should be considered as a risk and as a potential source of lock-in.
Document processes
Documentation is one of the best, but most misused, tools in ensuring site owners truly control their systems. Often documentation is an ignored requirement, or results in an inch-thick technical manual that no one will ever read.
At the outset of a project, require that some core parts of the system are documented, and make it an acceptance condition that this documentation is kept up to date, and internally verified. This should include:
- Separate suppliers
- Integrate rather than extend
- Retain ownership of your data
- Retain ownership of intellectual property
- Look for mindshare
- Document the processes
- Prefer extensibility to features
- Get a second opinion
This provides a baseline of information that another developer who is familiar with the underlying system, but not this specific instance, can use to understand how the system is structured and used.
In general, the documentation should cover the exceptional, rather than the normal. If the system follows a standard deployment procedure adequately documented elsewhere, there is no need to have the developer repeat that work. However, if special steps are required then these variances must be accurately documented and recorded in reference to the standard procedure.
Prefer extensibility to features
One common, but unpleasant, situation is to be in an otherwise good relationship with a supplier, but for changes to their application to be prohibitively expensive. This can occur with the best of will, and is sometimes described as ‘technical debt’ – bad practices early on have led to a situation where even simple things are much more difficult.
This effectively leads to a lock-in situation: the customer’s choice is to either keep paying over the odds for changes, or to spend out on a replacement solution. Often this is caused by a misplaced sense on the part of the developer that they are helping their client by delivering every desired feature straightaway, without realising they are harming the flexibility of the application in the future.
In this case prevention is better than cure, and one way to avoid it is to discuss future changes and challenges with the development partner as early as possible – with the proviso that these are not requirements, but options. The developer should not build the features, but ensure that the system is appropriately ‘extensible’ – that it can be made to support those kinds of changes in the future.
This will likely mean spending more time on architecture and scaffolding on an application than they otherwise might, which takes more time. To ensure that the application still fulfils the needs of the business, the minimum set of features needed on the application should be prioritised, developed to a usable point, and then the remaining features should be assessed to determine which are needed next. Working in this way helps the developer realise what’s important to the business, now and in the future.
To ensure that this process is being followed, it’s worth considering auditing the code occasionally, preferably by a disinterested party, such as an internal technical resource, or an unrelated agency.
Get a second opinion
The very reason that a company would hire a development partner is the same reason that they are susceptible to lock-in – the agency has a set of skills and experience that are lacking inside the company. One way to ensure that the best decisions are being made is to hire a third party to provide technical and strategic oversight and consultation.
This agency should not be eligible for any development work, and ideally should be from a company with dedicated professional technical or strategic consultants. They can help form a strategy, create proposals, evaluate options and suppliers and ask the awkward questions at meetings. They should also be able to audit the work of the suppliers and ensure that the best practices for the site owner are being followed.
This work is mostly conducted at the beginning of the project, with a review at the end, rather than continuously throughout development. This keeps the cost to the site owner lower while delivering insights and clarity at the most effective moments.
The procedures for avoiding development lock-in should be followed with the strategic partner as well, particularly ensuring that the product of their work is owned and controlled by the site owner, and that any processes they introduce are clearly documented.
Conclusion
After examining techniques for avoiding vendor lock-in, it should now be clear that vendor lock-in isn’t just about file formats and incompatible applications – it’s about being trapped on a system that is no longer meeting your needs. Avoiding lock-in is mostly a case of being aware of the risks, and making choices that balance the features of the application with flexibility in the future. Following the principles and advice laid out herein will help IT managers better select vendors and systems.
