Startups Should Test and Learn Their Way to Success

I’ve been involved with 51 startup tech companies as a founder, executive, investor or consultant since the 1980s. Most were entrepreneurial startups but a few were “intrepreneurial” businesses inside Fortune 500 companies. The primary difference between the startups that succeeded (achieved the objectives of the founders or leaders) and the startups that failed, was how their leaders resolved the key decisions that make or break a startup business: what products or services to develop, which customers to pursue, which business model to implement, how much to spend on different initiatives or functions in the company, how to fund growth, and so on.

The approach that worked best in my experience with these startups was “test and learn.”

How does a startup test and learn? How do you know that your testing and learning efforts are focused on the right issues, have the right metrics, and yield the right answers? The founders and investors in a startup are in a hurry, so how can startups test and learn really fast?

The most complete answers to these questions that I’ve seen are in Eric Ries’ book, The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. By following the “Build-Measure-Learn” process explained in Ries’ book, startups can get continuous feedback from the market, so that they can decide whether to continue with their current plan or “pivot” into other directions. Build-Measure-Learn becomes a cycle with a built-in feedback loop that improves the quality and “rightness” of decision making.

When I co-authored Building Routes to Customers in 2009, I had been applying the Routes-to-Market (RTM) methodology to help managers and executives make these same decisions more systematically and successfully for over a dozen years in both large companies (IBM, Microsoft, HP, Cisco) and small companies (Knoa, e-Smith, ShoreTel, HaloSource). When first-level managers in these companies were empowered to work as a cross-functional team and use RTM to make these decisions in their businesses, they outperformed the previous results in their businesses, turned around troubled businesses, and/or launched impressively successful new businesses. The RTM and Build-Measure-Learn methodologies have many things in common, but the linchpin is test and learn.

So how do you test and learn your way to success?

  • Step 1 is to develop an hypothesis about how customers will act when you provide a new or changed product or service to them.
  • Step 2 is to measure how the customers really act.
  • Step 3 is to analyze the information from the measurements and determine what the impact was on the customers’ actions. For example, did current customers buy more, or did new customers buy for the first time. Other kinds of change in customers’ behavior might be more important, such as telling their friends or colleagues about your product.

You need to test and learn scientifically, not subjectively.

Testing requires discipline, not the chaotic thinking of the founders of many startups. Learning requires an open mind that’s not wedded to the assumptions of the startups’ leaders. Test and learn is the yin and yang of making the decisions that drive success.

Smart Entrepreneur Takes Start-up from 0 to 200

When my good friend and collaborator Jonny Goldstein of Envizualize introduced me to Mina Mansour in January 2010, I thought that Mina had a brilliant concept for a product family of software plug-ins that add ecommerce capabilities to, the popular sales force automation system. Mina is the founder and president of IdaApps, a software company she started after 2 years of building custom apps for customers.

Mina saw that many of’s 65,000 customers needed inexpensive online product catalogs and shopping carts so they could easily add ecommerce to their operations, instead of spending $100,000 and up for custom-built ecommerce sites. Mina explained to me that she had:

  • built and tested two easy-to-setup ecommerce apps for’s cloud computing platform
  • submitted the apps to’s app certification process
  • identified several consulting and software services firms who could resell and customize her apps for their customers
  • started building a pipeline of leads by networking with people at meetings.

When I talked with Mina, I realized that she had two more assets that would be very valuable:

  • she had 1,000 followers on Twitter (from connecting with people in the ecosystem)
  • she had the “can do” attitude required to overcome the obstacles that all start-ups face.

Over the next 6 months, I worked with Mina to help her negotiate a contract with and close her first customer, sign her first channel partner, build her marketing and sales process, fine tune her marketing messages, fill her pipeline and close more deals. With over 200 customers worldwide today, Mina has proven her concept many times and over, and taken her start-up to the big leagues.

Please click on the image below (opens a YouTube page in a new browser tab) to see a terrific video about the eCatalog and eStore apps from IdaApps, and then check out the IdaApps website for testimonials from her customers and more info.

IdaApps video


Is Conflict Part of Your Go-to-Market Strategy?

As they grow, many technology companies evolve their go-to-market strategy from a direct sales model to a model that includes both direct sales and channel partners. In this evolution, management adds channel partners in order to scale their sales resources more cost-effectively than is possible by hiring only direct salespeople.

Unfortunately, the resulting go-to-market model often includes inherent conflicts that impact the company’s competitive effectiveness and success in the market. In these companies, direct sales and channel partners don’t understand how they are supposed to fit together, don’t have an effective way to resolve conflicts when they occur, and sometimes sabotage each other. Symptoms of these conflicts include:

  • Complaints by the members of the direct salesforce (from individual sales reps up to and including the head of sales) that channel partners “add no value” in specific deals or across the board.
  • Never-ending debates between the CEO and CFO on whether channel partners are really necessary and worth the 30-45% discount.
  • Sales management “looking the other way” when a salesperson takes an order direct instead of through a channel partner in situations where channel partners normally get the order.
  • Channel partners guarding the identities of their prospects and customers from the vendor, to avoid the possibility that vendor personnel will tip off other channel partners or take the deal direct.
  • New product launches that are planned and executed without any partner involvement, even though management expected that partners would start selling the new product as soon as it was announced.

Each of these symptoms is the tip of an iceberg of confusion and mistrust, misdirected or wasted efforts, internal bickering instead of competing with the company’s real competitors for the customer’s business.

If your company has these problems, what can you do to reduce and eliminate them?

The first step to fixing these problems is to clarify which customers should be pursued by direct sales and which should be pursued by channel partners. The ideal is to have a simple, clear and unarguable line that separates these two market segments — the “direct” segment from the “indirect” segment.

One example of such a clear line would be a list of named “house accounts” aka “hard deck” that the company sells to directly, with all other customers “off limits” to the direct salesforce and therefore available to channel partners without competition from the direct salesforce. Another example would be to segment the market by company size, transaction size, or some other attribute that’s easy for everyone to discover early in the sales process. A third approach is to assign specific products to the direct salesforce, and other products to the channel, which works as long as it’s clear to everyone, including potential customers, that the products are so different that no one can easily substitute one product for the other.

The smart way to figure out where the line should be drawn is to model the financials for the different choices. The Route Calculator is a great financial modeling tool that can be used to evaluate alternative boundaries for market segments that can be covered by a direct salesforce or by channel partners, with the output being the “location for the line” that produces the most market share or revenue growth or profit (or any combination of quantifiable goals). The Route Calculator is an integral part of Paramarketing’s approach to optimizing the ROI on marketing and sales.

However the line is drawn, it must be enforced. Poachers who cross the line must be disciplined, and the deal redirected to the correct party, so that others won’t cross the line in the future.

Unfortunately, the management of many companies believe that they cannot implement the ideal solution outlined above. They see their world in shades of gray, not black and white. They may lack the will to draw a clear line between one market segment and another, or between one set of products and another, because they fear it will limit their growth or their ability to change their mind in the future. Companies that are built through acquisition often find themselves full of overlapping products that are not easy to separate for different channels.

The solution for these companies is to take it a step at a time, starting with documenting the “gray area” where deals can go either way (to direct salespeople or to channel partners). Over time, they need to reduce the size “gray area” in order to widen the sizes of the opportunities that clearly belong to direct and indirect channels.

Every company should have a go-to-market strategy that maximizes profitable growth and minimizes wasted efforts. Conflicts between direct sales organizations and channel partners impact both growth and profitability. The problem is not the conflict; it is a lack of rules and poor enforcement of the rules. That’s something that management can and must resolve.

Modern Marketing Matters to Sales


Modern Marketing helps sales leaders make their numbers by getting them in sync with their marketing counterparts. Modern Marketing is the combination of things that a vendor does to build relationships with prospective customers via traditional and new forms of marketing communications such as blogging and social networking.

The Revenue Cycle

In the world of Modern Marketing, sales and marketing are linked in the revenue cycle. Before implementing Modern Marketing (MM for short), sales leaders might say that they have no influence over marketing and that marketing contributes little towards helping them make their number. In contrast, modern marketers care about engagement and lead conversion, which is what builds the sales pipeline.

In a B2B world, sales leaders who closely sync with their marketing counterparts have a much easier time making their number than those who don’t. Laura Mashina at Marketo supports this point in her blog post The Marketing and Sales Handshake. She wrote that, with MM, “the traditional hand off from marketing to sales has become a solid handshake.” As a result, MM can serve as a bridge for sales leaders to get a conversation going with their marketing counterparts. This conversation is essential to ensure that the marketing spend produces strong leads that build the pipeline (or funnel, if you are of that persuasion).

Modern Marketing Savvy

Modern Marketing is how companies communicate and nurture relationships with prospective customers during the early stages of the customers’ buying process. These communication touches have a significant influence on whether or not strong leads result.

Modern Marketing might be a new term to you but its effect is something you are already quite familiar if you have ever been offered a live chat session, registered to attend an online or live seminar, downloaded a paper to learn more, commented on a blog, participated in an online community, or googled to research something you wanted to buy.

You are Modern Marketing savvy if you:

  1. Align your marketing messages to your prospects’ interests and point of view.
  2. Establish trust and relationship with your prospects throughout their buying experience.
  3. Ensure your content and service helps your prospects solve their problems.
  4. Use a mix of technology to programmatically touch your prospects.

Use Sales Insights to Develop Prospects’ Trust

A sales orientation is absolutely essential to the success of Modern Marketing. Because the sales team is closest to the customer, they often have the best input for what the customer will respond to. MM programs infused with effective answers to a prospect’s real concerns will create trust with the prospect. When trust is developed at each stage of the buying process then MM generates engagement and drives pipeline growth. This is why Modern Marketing matters to sales.

Optimize Online and Offline Sales and Marketing with Web Analytics

Information about online customer buying behavior and decision making can now be used to improve offline as well as online marketing. With web analytics, companies can test alternative sales messages, offers and promotions on their e-commerce websites, and then use the most successful message, offer or promotion in offline media (such as print publications or radio) and offline sales processes (such as direct sales, telesales or retail).

Jim Sterne, Chairman of the Web Analytics Association, has been promoting this approach in interviews, webinars and conference presentations since 2006, when he noted that a few companies were just beginning to get “insights coming out of their web analytics tools that were powerful enough to impact decisions the corporation made about offline marketing.”

The approach works like this:

  1. People search online for information on just about every product or service that they are thinking about buying, whether they intend to buy it online or offline. This means that companies can reach people online, even if their purchases will be offline.
  2. People’s responses to online information are very similar to their responses to the same information when it’s provided offline. This means that their responses to online information are excellent predictors of their responses to the same information provided offline.
  3. The information presented to them online can include choices between two or more alternatives, such as a “buy 1 get 2″ offer vs. a “50% off” offer. Even though these offers are mathematically the same, one offer will be selected by customers much more often than the other. Web analytics tools can identify the more popular offer from the customers’ online choices.
  4. For every group of alternatives, the most popular alternative online turns out to be the most compelling offline as well. The company should use that alternative online and offline.
  5. The most popular alternative then becomes the “control.” Other alternatives can be tested against it to find an even more popular and compelling offer.

In addition to testing alternative offers, companies have been using the Internet to test alternatives in marketing strategy, creative approaches, positioning, pricing and promotions. Online testing is quick. Hundreds or thousands of alternatives can be generated automatically and tested thoroughly in 24 hours, if a website has enough traffic. Web analytics tools can identify the most popular alternatives in minutes.

When I discussed this approach with Jim Young, Market Research Manager with BrandSolutions, he suggested that the most efficient and customer-friendly method for determining the optimum set of product features to meet customer preferences and maximize profitability, is conjoint analysis. This analytical technique obviates the need to test every combination of features, which would be daunting for customers to sort through.

With conjoint analysis, customers are asked to make price/feature tradeoff decisions as part of the product selection process on an existing e-commerce website, or via online surveys. In fact, the conjoint analysis solution works for situations where there are many variations in any of the alternatives in marketing strategies, creative approaches, positioning, pricing or promotions.

By using conjoint analysis in conjunction with web analytics tools, the most popular and profitable alternatives can be identified quickly, and they can be put to productive use online and offline immediately.

To optimize your online and offline sales and marketing, please contact Peter Raulerson.