The Software Selection Process - Part II


The initial vendor selection

Given the number of times businesses have selected software packages it is remarkable that in the majority of cases it is an inefficient, ineffective, time consuming and costly process. Even if you hire expensive consultants to guide you through the process the implementation invariably under-delivers in terms of functionality and overruns time and cost budgets. This article looks at some of the reasons for this and suggests a few simple but little used techniques to avoid buying a package then learning its short comings after the contract has been signed.

Before we critique the process let’s be clear on the guiding motivation of the two parties involved. The business is trying to make a selection by evaluating the application’s capability to perform their business tasks: they believe they are in the driving seat and able to make an informed rational decision. The vendor is involved in a fiercely competitive sales process geared to earning sales commission. Pre sales demonstrators have honed the fine art of creating an impression you saw something when you actually didn’t, much like a magic show!

Neither approach considers how easy it will be to implement the application chosen and how easily it can perform the tasks.

A typical selection process starts with the identification of the vendors who provide relevant applications. Any more than six candidates may look like a fishing trip. A Request for Information is sent out to engage vendors and elicit their participation in the process. This is followed by a more comprehensive Request for a Proposal (or Specification of Requirements) to a short list of four to five vendors. There are two immediate drawbacks to the last step which significantly reduces its effectiveness. Firstly, the listing of functional requirements in a document is time consuming and results in an abstracted description that omits much complexity and most of the business modus operandi that gives meaning to the functionality. Secondly, the vendor’s response lies about their functional match: this is a sales process and there are no prizes for honest losers (but heaps for dishonest winners). For example, we know a vendor who confirmed their application could work offline on a laptop from the comfort of a business class aircraft seat: post contract the vendor just said it was due in a future release (and still waiting).

So after considerable time and effort, that may have wilted down the field but arguably has produced little reliable evaluative evidence, the selection-sales process is in full swing and the next event is the demonstration.

The demonstration

Given the lack of comfort available to purchasers from the previous steps the demonstration is the main way of evaluating a products capabilities.

The guiding principle here is not meant to promote a cynical approach but as we are dealing with an invisible abstract product the rule of thumb is if you don’t actually see something, it’s best to assume the reason is (in order of likelihood):

  • the vendor does not have the functionality;
  • the functionality is too technical difficult to execute in a demonstration (and hence the implementation?);
  • or the person demonstrating doesn’t know how to use the application in that area.

If the vendor has the functionality – you probably will see it. If not they have a number of ways around this from making you think you saw it, to “showing you at the end”, to not being able to show you right now.

So let’s set 5 simple rules to guide us to conduct an effective demonstration.

Rule 1 Test Data: always ask the vendor to demonstrate using a data set extracted from your business. This sounds obvious but is frequently not done as it takes a lot of time to construct logical transactions that can be passed to a vendor for processing.

If this rule is not followed the demonstration is a sales presentation of the vendor’s strengths (but not their weaknesses).

Rule 2 Live data: take a further set of transactions and insist these are processed.

If this rule is not followed there is a possibility that the results of Rule 1 could have been achieved by “smoke and mirrors”. Unfortunately in a sales process there is the temptation that Vendors will try everything to make a product look better than it actually is.

Rule 3 Configuration and process changes: it is crucial to test the ability to add a field to capture your particular data needs. A vendor may be able to add the field but without seeing data entered and edited all you’ve seen is potentially a non-working field being added to a screen. I know of a vendor that would demonstrate the addition of the new field but not actually use the field: in order to use the field there was a data design and mapping process that was too technical to show in the demonstration and was totally reliant on the vendor.  The adage that if you don’t actually see it, then assume it’s not there, will serve you well.

Rule 4 Reports live and amended: reporting requires two major components, first the ability to query all of the underlying data in the system, and second the ability to present that data in dashboards and reports. Vendors are skilled at giving an impression you can easily get the data you need for reports, but what you actually see in a demonstration is:

  • a pre-canned report, or on-screen data, showing the result of a query;
  • the slicing and dicing of datasets.

This may suggest the application meets the requirements and may even show the data you provided. However it’s easy enough for a vendor to prepare reports or datasets for analysis and it does not show the data change and flow through to the reports live in front of you something you must have to create reports and analysis yourselves.

In order to really see the functionality you would need to execute the following simple steps:

  1. view a report;
  2. modify some of the data reported;
  3. add new data fields and associated data;
  4. re-run the report to see the changes;

It cannot be overemphasised how important these simple steps are. If you don’t actually see the full cycle of data being changed, added and reported then all you’ve seen is a pre-canned report or slicing of a static dataset something any vendor can do by opening a pre-prepared pdf. This type of ‘magic trick’ is being used today and causes huge problems during implementation. The problem is so pervasive that there are products designed to replace a vendors reporting with a generic report engine. Whilst these products can be of great value you are paying a second time for functionality you thought you had purchased from the application vendor.

Rule 5 Review all core components: the demonstration may have shown personalised homepages, alerts, reports and the processing of your data. However, when you start to implement the package you often find there are significant costly and time consuming difficulties.

The reason for these implementation problems is described in the first part of this article; in most cases the core functions common to all systems are not efficient tools that can be used by the business implementation team.

So how can you evaluate the capability of the software when Vendors are adept at ‘smoke and mirror’ techniques that give the appearance of meeting your requirements.

There is only one approach that works and that is to trial the software and test all of the core functions yourself. This is a significant additional activity that will lengthen the demonstration process and

  1. this does add significant cost and time to the selection process,
  2. this will be difficult to negotiate with the vendor (unless you insist and then it will be done),
  3. this will significantly reduce the risk associated with your selection decision,
  4. this could improve the estimates of cost and time for the implementation,
  5. this could prevent the purchase of a poorly constructed package and save the business from years of frustration.

The core components that need to be thoroughly examined include:-

  • Querying/Reporting
  • MS Word/MS Excel Integration
  • Document Management
  • Auditing
  • Relationship Management
  • Permissioning
  • Workflow
  • API

The testing process requires some planning but is relatively straightforward, creating new fields, adding data, writing reports, setting up users, checking audit trails etc. With all tasks performed by the purchaser this process will shine a light on the areas that are inadequately covered by demonstrations and provide a wealth of information to guide the purchase decision and implementation planning.

It’s normally these core processes that are at the centre of the misalignment in expectations during an implementation. The client expects to write reports and configure dashboards but find they need the Vendor’s help to create queries and configure fields. The implementation is extended, there can be a loss of goodwill and the on-going enhancement of the application is inhibited.

From the vendors point of view they are not setting out to mislead you (well not much, let’s not forget this is a sales process), the pre-sales team are doing the best with the tools they have. They know many of these core processes require their technical input they just do not want to admit to this and see the implementation budget increased to their competitive disadvantage.

So in summary, irrespective of the line of business functionality provided by the package, implementations struggle because of the inefficiency of their core functions: simple tasks require costly technical support from the vendor.

Like all software developers we have worked for companies who were (and still are) guilty of these practices. However there is a different approach which by designing the core functions as part of the original architecture ensures that straight forward implementation tasks like building reports can be performed by business users.

The final stages

With the demonstrations complete you should have identified the preferred vendor and
have one vendor in reserve. There are seven further crucial steps before the
project can get underway.

References: it is vital to take independent soundings from the vendors customers. The normal practice is for the vendor to provide a list of customers and suggest which customers are suitable to ask for a reference. The vendor will often suggest you only consider companies that have completed their implementation and accompany you on any site visit: although they may not be in the room during discussions their presence may influence the independence of the reference. In short there is a risk that you will only talk to favoured customers who provide good (or not bad) references. We urge you to take a different approach. Phone one of the vendors customers and ask them who else they know who has recently bought the system. Track down customer that are still implementing and visit them independently. The intelligence gathered from these meetings will be invaluable in structuring your contractual negotiations and implementation plan.

Pricing and payment terms: the pricing of packages is often complex providing the creative landscape by which the vendor can charge what they think the customer will bear. We advise you negotiate hard and insist on deferred payment terms against the realisation of your measurable benefits. It is a buyers’ market so push for the terms that fit the project. If you do not meet the business objectives paying a single one off license fee or with a fixed contractual period can be a risk as you may be stuck with the application until the capital cost has been depreciated. An annual (or even monthly) license model may provide greater flexibility to both up and down scale the usage of the application and revisit the market at a time of your choosing.

Contractual agreement: there are many important conditions to be considered during the agreement of contractual terms and we recommend that you are advised by a legally experienced and commercial savvy expert. Amongst other things particular attention should be paid to the service levels for maintenance and version releases.

Post contract vendor reorganisation of responsibility: One of the big internal changes within the vendor that is frequently missed by purchasers is that this is the point when the vendor’s pre-sales team hands over to the implementation team. A ‘hand over’ sounds like a simple enough process however the sales team has completed its job and there may be very limited protocols for handing over information to the implementation team. This can have a huge impact on the implementation project. In many instances you can find that you are starting again with a new team that does not have a sales orientated view of the world. Things that were ‘easy’ become a bit more difficult as the implementation problems arising out of inefficient core process are no longer hidden by the ‘smoke and mirrors’ of the demonstration. This is where the implementation time and cost overruns start to raise their heads.

Project benefits: everyone knows that all implementation projects should be monitored against the achievement of measurable benefits. These benefits should guide the contractual arrangements and the project planning. The only mystery is that so few implementations rigorously follow the mechanism to measure the achievement of the planned benefits. Without empirical measures you cannot hold the vendor to account.

Project planning: this is not an exercise in gnat charts and task assignment. There are three interdependent moving parts in each project. Firstly, the scope of each phase has to be defined in line with the benefits to be achieved. With this done the project is determined by the resources allocated and the timespan the resources are to be deployed: these two parameters determine the final moving part the quality of the outcome or the likelihood of achieving the benefits. Project managers need to work to this age old formula and push back when projects are under resourced (and become known as ‘the bastard’ to make them succeed, this is not personal it is just essential to the job!).

Project resourcing: on the Customers side change projects need the people with the best business process knowledge to be dedicated full time. If the business does not spare the best resources and those it does assign have to also work their day job then the project will both over-run and under deliver.

The vendor will expect you to employ their implementation team to assist you. If you do you will need to manage and monitor them very closely as they may have a conflict of interest between implementing your system and protecting their employer. There are two issues, firstly the implementation team is not treated as a customer by the vendor’s support and development teams and they frequently do not get priority to fix software bugs or modification requests. Secondly, the vendors implementers are adept with workarounds and can write technical scripts or get code fixes that are outside of the maintenance cycle. This means that many of the implementation tasks are too technical and rely on hidden vendor knowledge to complete.

An alternative approach is to see if the vendor has attracted any ‘camp followers’ who have experience implementing the package but contract independently. These independents may have more competitive rates and are often as skilled as the vendor’s implementation team. They provide a better boundary between the customer and vendor which can mean the vendor is obliged to solve implementation development issues in a timely fashion.

Post-implementation maintenance

If an implementation has been substantially completed by the vendor’s team they may have controlled the whole process and you have limited on-going ability to change and improve processes yourself. All changes are dependent on the vendor and may have to be delivered in line with their version release program which can make the whole process both lengthy, costly and a significant management overhead. The need for ‘continuous maintenance’ is frequently overlooked after the initial implementation effort and the software package slowly degrades in usefulness over the next 4-5 years until the whole selection cycle starts again.

To avoid the nightmare of sequential software package implementations there are three key things that need to be put in place. Firstly the package must have a business sponsor who has fully responsibility for process efficiency. Secondly there should be an active program of continuous improvement which at the very least prevents MS Excel based systems being used for workarounds. Lastly the vendor has to be actively managed to ensure they maintain the package and provide frequent version releases to ensure bugs are fixed and new capabilities delivered.

The way forward

The way to ensure you select the best available software solution is to follow the selection guidelines described.

To fundamentally change the value proposition for business applications you need to actively target the next generation of software products. These solutions have three key elements;

  1. Cloud computing: which reduces cost and operational risk, and enables,
  2. Continuous maintenance: to radically improve the quality of the software and responsiveness to customer requests and,
  3. New software architecture: which builds the enterprise components as core enabling capabilities.


In summary legacy software products have a common design deficiency, they started with the business functionality and overtime added the common usability functions, this has invariably meant that ‘simple’ tasks have been difficult to achieve. New application development tools are now available that make implementing and maintaining a business system, tuned to your business’s specific requirements, achievable and affordable radically changing the value proposition.