These methods for testing hypotheses are available

However, this only applies if hypothesis tests are used right from the start. If the finished product is tested first, this often means the failure of the entire project. In this blog post you will learn how to test hypotheses and which methods you can use.

Key assumptions, risks and hypotheses

Every business idea is based on certain basic assumptions. If these can be converted into a testable form, they are called hypotheses. Testing hypotheses is basically based on the most critical assumptions or hypotheses that need to be verified in order for the project to be successful:

• Identify basic assumptions and convert them into testable hypotheses.
• Sort hypotheses by relevance for innovation (product, service, business model).
• Test hypotheses with the highest relevance first, with the hypothesis that is most critical to the success of the idea at the top of the list.

It is therefore important to test the most critical factors that can cause the project to fail, for example:

• Problem validation:Does the identified problem really exist?
• Problem solution fit:Does the idea solve the problem?
• Product market fit:Is the investment worthwhile, will the innovation pay off?
• Technological feasibility:Can we implement the idea technically?
  
  

Given that behind every business idea there is a problem that is to be solved by innovation, the first question that usually arises is "Does the problem even exist that you want to solve with a new product, service or business model? A first hypothesis derived from this could be, for example, "My customers have problem XY" or "My customers are bothered by XY".

Once it has been verified that customers actually have this problem and are therefore interested in the innovation, the next step is the problem solution fit. A testable hypothesis could be formulated as follows: "My product in this form solves the problem. We can see this from the fact that X percent of our customers behave like "Y". After this hypothesis has been tested positively, the market relevance and technical implementation can be examined in the next step, etc.

Iterative testing of hypotheses

Testing hypotheses should always follow the Build-Measure-Learn cycle. The method basically comes from the design area, where the customer is shown three sketches, for example, and is then given feedback. Based on this, the sketches are then adapted and presented to the customer again. This is repeated until the customer is satisfied with the result.

In the area of innovation management, the concept of build measure learning is known as discovery driven learning or lean startup method. The following therefore applies to the testing of hypotheses:

• Build:The test design is determined and the test is set up.
• Measure:The test is performed in the most realistic setting possible. In order to measure resonance, target values are defined which are evaluated as success (X percent of customers behave like Y).
• Learn:The results of the test are analyzed and a decision is made as to whether the hypothesis was correct or false and how to proceed.

If the test results confirm the path taken, it is possible to move on to the next hypothesis, which is tested according to the same pattern. If the hypothesis has been falsified, an adaptation of the original idea is necessary and a new hypothesis must be formulated and tested.

Methods for testing hypotheses

There is no truth at the desk. When testing innovative ideas, the focus is on the qualitative testing of hypotheses. Of course, research in advance can also provide useful insights, but ultimately a company needs to interact with customers, show the product and get feedback to gain reliable insights into the probability of success of the innovation.

For example, a car manufacturer could hypothesize that there are problems with the sale of new cars because there are more and more sharing models. If the research shows that the rental car market is growing disproportionately to the buying car market, this reinforces the hypothesis, but does not yet mean verification.

The selection of suitable methods for testing hypotheses should always follow the guiding principle "How can I achieve the best results in the solution with as few resources as possible? In the following we will show you which methods are suitable for the respective development phase.

Step 1: Explore the market and the problem

The following methods are suitable for finding out how the market environment is to be assessed with regard to the desired innovation and whether there is fundamental interest in the idea:

• Research:In addition to the collection of technical information on the company's business field, the focus of research in the course of innovations is on the collection of information about customers and users, competitors and suppliers. In addition to classical market observation (studies, customer surveys), the Internet in particular offers comprehensive possibilities for tapping external information sources, e.g.:
  • Online business and trade journals, research and technology portals, patent databases, open innovation platforms
  • Social networks, expert and consumer blogs, blogs of relevant influencers, test portals
  • Ask questions in online discussion forums such as Quora or company-specific forums
• Interviews: Open interviews with customers are an efficient way to find out what the problem is with the current solution and what the customer's needs are. Ideally, the interviews should be conducted face-to-face or via media such as Skype.
• Test advertisements: Test ads switched on suitable platforms (Google Adwords, Facebook-Ads, etc.), which are linked to an e-mail registration or pre-orders, enable conclusions to be drawn about the interest of customers.
• Blogposts: The company can receive qualitative feedback on a planned innovation by reporting on it in blog posts. Both your own company blog and blogs addressing relevant user groups are suitable for this.
• Online customer surveys: Another very good source of feedback is customer surveys combined with incentives, for example "100 Euro discount on product launches". If customers are then willing to participate in the survey, this should be seen as confirmation that there is a need here. 

Step 2: Test solution

The further the company expands its level of knowledge and progresses with its hypothesis tests, the more something concrete should be shown to the customer and the more realistic the tests should be. The latter is best achieved by designing the hypothesis test in such a way that the user groups tested do not even notice that it is a test.

• Sketches and MVP: In a first step the customer can be shown a sketch and in a second step a Minimal Viable Product (MVP).

• A/B Testing: Another possibility are A/B tests, i.e. you create a landing page with two variants and test what works best. In this way, different user groups can be compared quickly and cost-effectively with a test group.

• Smoke testing with dummies: Smoke testing is an efficient test procedure under very realistic conditions. For example, the company presents a dummy product, a video or a concept on a landing page or crowdfunding platform (Indiegogo, Kickstarter) and then observes how many people register for "Keep me updated" or click on "I want to buy XY" - without the product already exist. Smoke testing is therefore a method for testing hypotheses that offers a high level of knowledge gain with little influence at the same time.

• Beta-Tester: Before an innovative product or service is published, it is worthwhile to carry out a usability test via beta testers or early adopters. Platforms such as BetaList or Startups List are available to attract interested parties. 

Step 3: Market introduction testing

In a final step, the market launch can also be put to the test by testing hypotheses.

• Website: Analysis of your own website with Google Analytics (which keywords are most relevant, demographic information, segments, cohorts, etc.).

• Marketing campaigns: Here it is important to analyse which campaigns have the greatest traction. This is not only about understanding the customer, but also the behaviour of influencers. Various social media tools such as Bottlenose are a valuable support here.

• Referral Programs: To find out which recommendation programs reach the customer, you can experiment with monetary and non-monetary incentives. An example of a very successful recommendation program is Dropbox. The company uses a reciprocal incentive for sharing the referral link: The person who logs in to Dropbox via a referral link will receive more storage space than a normal login. Also, the person who recommended Dropbox gets additional storage space.

Conclusion: These methods for testing hypotheses are available

Plans that are only created at the desk often lead to failure. Tests and experiments are required to evaluate innovative ideas on a sound basis. The focus is on the most critical hypotheses that need to be verified in order to prevent failure and create real added value for the customer.

"Significantly, the most innovative companies conduct the best experiments."

Michael Schrage, Research Fellow, MIT Center for Digital Business