At the pilot phase, or a concept test phase, it is critical to define very clear what is going to be tested and what the parameters of measurement would be. Many of the parameters could be qualitative as there may not be enough data to do a quantitative analysis. But identifying what and how it is going to be measured is critical.
Below are a few things that are tested in a concept test stage/pilot phase:
- The concept – the power of the idea itself: One of the key things to evaluate in the pilot phase is whether the idea or concept has appeal among the target audience. While research is one way of gauging customer/consumer acceptance of the concept, customers or consumers actually buying/using is a stronger demonstration of the appeal of the concept e.g. for an e-commerce venture selling real gold & diamonds jewellery online, the pilot phase may want to check if customers actually buy jewellery online
- The business model: Even if the concept rocks, how good and practical the business is will make or break the business case. In many cases, it may not be possible to check the business model in its entirety in the pilot phase. But at least it would give some indicators. E.g. for a company creating advertising platforms on college campuses, the pilot phase may check on whether the revenue split between the company and the colleges is being accepted as they had anticipated or are colleges asking for more or are they not allowing advertising on the campus. Also, things around the business model could be tested in this phase e.g. ‘How long does it take to close a sale? Who decides the pricing – the college, the advertiser or the company?’
- The assumptions: How good your business case is and how close to reality it is will be entirely dependent on the quality of your assumptions. One of the critical things to test at pilot phase are the assumptions. e.g. For an e-commerce company, the assumptions around conversions from clicks, cost of customer acquisition, average ticket size, repeat purchase rates, etc. are critical factors to be tested.
- Understanding operational challenges: Actually implementing on the ground highlights some challenges that you would not have anticipated in your planning. These learnings are critical as they help plan resources, processes and infrastructure that would be required to manage the operations.
- Testing processes and operational capabilities: As the startup scales up, the dependence on processes will increase. Else scaling up will either not be possible, or will be inefficient. A pilot phase is a good time to draw some basic processes and observe the operations to understand what these basic processes should evolve to.