By Alex Shusterman

Published on Wed, February 22, 2017

All posts by this person

Think about your most recent car buying experience. How much time behind the wheel did you take before making the decision to buy the specific model you were interested in? Was it a quick dealer prescribed route or did you take it home for a couple of hours and really put it through its paces? The answer for most people these days is the latter – as an avid car enthusiast and shopper, my experience in recent years has been that buyers want to experience a prospective vehicle in their driving conditions before making a final decision. My question is, why are we not approaching our ediscovery purchasing with the same type of mentality by running a real-life proof of concept (PoC)?

Recently, I've had the opportunity to assist a client with the new implementation of third-party technology. The industry of my client and the category of ediscovery software really isn't all that important for our purposes here, because the scenario they find themselves in could exist anywhere in the ediscovery purchasing cycle. Since purchasing and starting implementation of this particular piece of ediscovery technology, it has become clear that while the sales and purchase cycle was robust, it may have missed vetting of key components of proper day-to-day functionality. Seemingly each day after go live, there was a new scenario or edge case that would arise that required a conference call, a review of documentation, and a development request. To make matters worse, functionality that was expected to work (including integration with other key business technology in the organization) was not working nearly as automatically or efficiently as might have been originally thought. After inquiring a bit more, I found out that no PoC had been performed in a way intended to imitate a real-life use case.

These days most people know when they go to buy a car that the only way to know if it is the right fit is to take an extended test drive – over your roads, your long commute – before making the purchase. Purchasing ediscovery technology and services should be no different. Seeing a short demo or using a small sample set of data in a sandbox environment tells you part of the story, but fails to demonstrate all of those edge cases that tend to be identified in daily use.

So what’s the answer? Once you have narrowed down your list of finalists, invest the time (and money) in an extended test drive of the technology. Stand the technology up in your environment, create the business-technology integrations, use it on a live case or two, and see whether it actually works in practice.  You may find that it works for you 80 percent of the time, but that the remaining 20 percent makes it a non-starter from a purchasing perspective. 

From my perspective, here are some things to consider and test during a PoC:

  • Third-party systems integration – E.g. data sources, human resources systems, active directory integration. Are they as automated and accurate as you expect?
  • Edge cases – How many matters in the last year did you run that fell outside of the norm and your typical thinking? Do you need to be able to handle them?
  • Existing functionality – After a thorough inventory of existing technology functionality, does the new tool have all of the same features? If not, are they important to you?
  • Migration experience – How smoothly can you move data from any legacy platform into the new tool? Will the vendor help with that process or are you on your own?

You may think this sounds like a pretty in-depth process for a tool that has not been purchased yet, and you would be right. In fact, stakeholders may try to argue that it is not necessary. The two most common arguments you may hear against performing this kind of an in-depth PoC:

  • This is going to take too long.
  • This is going to be too expensive.

My response to both of these scenarios is, can you afford the time and cost not to do the PoC? With my current client, the go-live has taken months longer than expected, there are still lingering issues requiring resolution, and some issues that may never be resolved, requiring costly workarounds. If you have run an RFx process and are confident with how you have narrowed the choices, then the time and cost of a PoC should simply be seen as a component of that process. As a buyer you should not expect the vendor to do a PoC for free and, likewise, the seller should not expect you to foot the entire cost. The PoC experience is an important step for both parties to ensure they are going to be happy with the final decision that is made, and the best way to do that is for both parties to have some skin in the game.

What have your experiences been with PoCs?  Have they added value to your technology purchasing experience? Please reach out to me at info@lhediscovery.com to share your thoughts and discuss more.

About the Author
Alex Shusterman

Senior Consultant

Alex has several years of ediscovery experience specializing in program maturation, with detailed specialties in data collection, legacy data archive remediation and migration, and ediscovery processing technology implementation and workflow development. He has experience scoping and driving the implementation of custom ediscovery workflows for some of Lighthouse's most sophisticated Fortune 100 clients. In addition, Alex has his B.S. form the University of Pittsburgh, his J.D. from Duquesne University School of Law, and he is a licensed Pennsylvania attorney.