In these annual benchmark industry reports, Key Lime Interactive (KLI) uses a multi-faceted research approach combining: quantitative, qualitative, and behavioral data to provide meaningful consumer insights for a variety of industries.
Cruise Competitive Index Overview
In these annual benchmark industry reports, KLI combines quantitative, qualitative, and behavioral data from hundreds of participants who complete a series of common tasks on each cruise line website. Statistical analysis of this data is employed to rank sites and uncover the top drivers of loyalty and site satisfaction.
Here are the seven Contemporary & Premium Cruise Line websites and mobile sites KLI took a closer look at:
Carnival
Celebrity Cruises
Disney Cruise Line
Holland America
Norwegian Cruise Line
Princess Cruises
Royal Caribbean International
Virgin Voyages
In our 2020 Competitive Benchmark Report on the Cruise industry, we performed a survey of 2,800 people on Mobile and Desktop.
Senior leadership and online marketers alike can make at-a-glance decisions about key areas of the user experience to satisfy their existing and prospective customers. What does your customer want and how can you better serve them?
Our competitive analysis can help address some key strategic marketing questions like:
How do you rank compared to your top competitors?
What are the user’s top pain points using your website?
What are the best practices for cruise research and booking online?
How do you measure up in key segments (such as first-time cruisers)?
Cruise Competitive Analysis Details
Each year the benchmark will examine sites’ top problem areas from the previous year and any site redesigns/changes made subsequently, to answer the question: Have the sites made changes that have improved or worsened the user experience?
This Cruise UX Analysis will include:
Statistical analysis of quantitative data
Multiple regression analysis to yield top predictors of desired outcomes (Loyalty and Site Satisfaction)
Behavioral data
First click analysis
Average time on task
Uncovers problems with the sites’ navigation, nomenclature, layout, clarity or prominence of key features/ functionality/ content
Direct user feedback
Insights from consumers in their own words about what they like, dislike, want, need, and their expectations and frustration
In-depth analysis of report buyers’ website
What does the site do well relative to the competition?
Where are the gaps between the site and its competitors?
Specifically, what is responsible for the gaps? And what specific issues with the online experience drive negative user perceptions of the site?
Success Factors and Best-in-class examples
Best-in-class sites highlighted to illustrate why more users say it’s easy to use makes the best first impression, has the most appealing activities and excursions, why more users are likely to return/recommend/ book on the site or with cruise company in future
Longitudinal tracking and analysis
Additional information about KLI’s methodology is detailed in our detailed report. Request Detailed Report.
The evaluation includes a general discussion about what these companies are doing for their existing and prospective customers via mobile-optimized websites and apps (iOS and Android). Data is captured from actual customers of each issuer and includes screenshots of features from behind-the-login screens of each provider.
In this detailed report, KLI profiles the competitive mobile landscape of eight (8) of the top US retail banks, and ultimately reveals which bank offers its users the best mobile solutions.
Gain insights into typical behaviors and context for the use of your product with year-over-year brand performance ratings across mobile web + mobile app across iOS and Android.
Play-by-Play Screenshot
Get full access to a library of screenshots of your competitor’s latest mobile designs indexed by feature, brand, and release date.
Dynamic Change Tracking Tables
See when top brands in your industry update, remove or add new mobile features.
Features Ranked By Consumers
View results from over 500 active consumers on the mobile features they value most in your industry.
Insights
Chart your course for a better customer experience today
Meet with a KLI Team Member
{
"These rankings acknowledge our continued focus on providing both current and prospective State Farm customers with the same great experience, regardless of how they come to us"
Patty Gaumond / State Farm
"Our C-suite asked us to find an A-Team… Having worked with Key Lime at a previous employer, I knew that there was no better team than them for actionable insights and strategic design recommendations.”
VP of Digital Strategy & Innovation / Miami Heat
“…This methodology provides the details we need in order to focus our development plans in the coming year....”
Home & Renters Insurance Mobile UX Competitive Analysis
(Published Bi-Annual in March and September)
This Mobile UX Competitive Analysis goes far beyond a basic heuristic evaluation or expert review. Additionally, thanks to our innovation arm Key Lime Labs, these insights are now available in a comprehensive platform designed for business intelligence leaders, market analysts, UX/CX strategists, and product managers who want on-demand access to the competition’s mobile customer experience.
Home & Renters Insurance Competitive Index Overview
Key Lime Interactive’s (KLI) Home/Renter’s Insurance Mobile Competitive Index Report reviews eight (8) of the largest home and renter’s insurance companies in the U.S. including:
Allstate
Farmers Insurance
Liberty Mutual
Nationwide
State Farm/span>
The Hartford
Travelers
USAA
KLI’s methodology is unique because it incorporates consumer preferences and goes far beyond a basic heuristic evaluation or expert review. An important part of our analysis is a 500-person survey of smartphone owners in which they are asked to weigh in and identify the features and capabilities that they consider critical for a successful experience with the mobile site or app offered by their home/renter insurance company.
KLI’s intent in creating this third-party syndicated report is to:
Provide consumer-driven data to help guide insurance companies as they prioritize features to implement. This guidance will be based on the results of a current consumer survey.
Summarize how the insurance companies differentiate themselves from their competitors through the capabilities and features that they offer.
To create an overall score, we combine a capabilities assessment with user feature importance ratings. The capabilities assessment is created by examining the feature coverage of insurance sites and applications. User ratings are determined by a consumer survey and card sort.
Our review of the primary mobile properties includes a full verification of the insurance companies’ capabilities. The unmodified score represents feature coverage or the company’s offerings by category. This is a binary evaluation reflecting the possession of certain criteria.
Additional information about KLI’s methodology is detailed in the detailed report. Request Detailed Report.
Incorporating User Feedback
KLI conducted a consumer survey and card sort (n=500) to gather feedback about how customers prioritized features when using a home/renter’s insurance company’s mobile property. Individual feature scores were then weighted by their value to customers. The goal is to provide a metric of relative importance so that the highest-scoring company is also the one providing customers’ desired features.
Summary of Capabilities & Features within UX Analysis
The report features key areas that drive user satisfaction and quickly identifies areas of improvement to satisfy existing and prospective customers. KLI’s research experts highlight opportunities based on a rigorous independent evaluation and that ultimately identifies trends and opportunities that move the entire industry forward.
The evaluation includes a general discussion about what these companies are doing for their existing and prospective customers via mobile-optimized websites and apps (iOS and Android). Data is captured from actual customers of each issuer and includes screenshots of features from behind-the-login screens of each provider.
Each year the benchmark will examine sites’ top problem areas from the previous year and any site redesigns/changes made subsequently, to answer the question: Have the sites made changes that have improved or worsened the user experience?
Gain insights into typical behaviors and context for the use of your product with year-over-year brand performance ratings across mobile web + mobile app across iOS and Android.
Play-by-Play Screenshot
Get full access to a library of screenshots of your competitor’s latest mobile designs indexed by feature, brand, and release date.
Dynamic Change Tracking Tables
See when top brands in your industry update, remove or add new mobile features.
Features Ranked By Consumers
View results from over 500 active consumers on the mobile features they value most in your industry.
Platform
Unbiased Third-party Mobile UX Research
Get the Competitive Intelligence you need to stay ahead in your industry. Competitive Insights is an intuitive competitive intelligence software, incorporates consumer preferences of features and capabilities that they consider critical for a successful experience including:
A complete listing of overall rankings
A detailed scoring breakdown of all capabilities and features reviewed
Screenshots from behind the login screens of all competitors sites & apps
A comparative list of current mobile capabilities available on each mobile app
Detailed descriptions of how users can complete tasks on the mobile device (e.g. pay a bill, look at account history, etc.)
User feedback collected regarding feature preferences
Identifies key success factors, best-in-class features, areas for improvement, and why some companies are lagging behind
Chart your course for a better customer experience today
Meet with a KLI Team Member
{
"These rankings acknowledge our continued focus on providing both current and prospective State Farm customers with the same great experience, regardless of how they come to us"
Patty Gaumond / State Farm
"Our C-suite asked us to find an A-Team… Having worked with Key Lime at a previous employer, I knew that there was no better team than them for actionable insights and strategic design recommendations.”
VP of Digital Strategy & Innovation / Miami Heat
“…This methodology provides the details we need in order to focus our development plans in the coming year....”
The Mobile Credit Card Competitive Analysisby our innovation arm, Key Lime Labs, summarizes these new, successful, and well-received features across the finance industry so that both the industry leaders and institutions, large and small, can make at-a-glance decisions. This analysis features key areas that drive user satisfaction and quickly identifies areas of improvement to satisfy existing and prospective customers. KLI’s research experts highlight opportunities based on a rigorous independent evaluation and that ultimately identifies trends and opportunities that move the entire industry forward.
Mobile Credit Card Competitive Index Report
In this detailed report, Key Lime Interactive (KLI) reviews the competitive mobile landscape of nine (9) of the top US credit card issuers, and ultimately reveals which bank offers its users the best mobile solutions:
American Express
Bank of America
Chase (JP Morgan Chase)
Citibank (Citigroup)
Capital One
Discover
USAA
Wells Fargo & Company
U.S. Bank
This report goes far beyond a basic heuristic evaluation or expert review; KLI’s methodology is unique because it incorporates consumer preferences. An important part of our analysis is a 500-person survey of smartphone owners in which they are asked to weigh in and identify the features and capabilities that they consider critical for a successful experience with the mobile site or app offered by their credit card company. KLI’s intent in creating this third-party syndicated report is to:
Provide consumer-driven data to help guide credit card issuers as they prioritize features to implement. This guidance will be based on the results of a current consumer survey.
To summarize how credit card companies differentiate themselves from their competitors through the capabilities and features that they offer.
To create an overall score, we combine a capabilities assessment with user feature importance ratings. The capabilities assessment is created by examining the feature coverage of credit card sites and applications. User ratings are determined by a consumer survey and card sort.
Our review of the primary mobile properties includes a full verification of the financial institutions’ capabilities. The unmodified score represents feature coverage or the bank’s offerings by category. This is a binary evaluation reflecting the possession of certain criteria.
In this detailed report, KLI profiles the competitive mobile landscape of eight (8) of the top US retail banks, and ultimately reveals which bank offers its users the best mobile solutions.
The evaluation includes a general discussion about what 8 of the largest companies are doing for their existing and prospective customers via mobile-optimized websites and apps (iOS and Android).
Each year the benchmark will examine sites’ top problem areas from the previous year and any site redesigns/changes made subsequently, to answer the question: Have the sites made changes that have improved or worsened the user experience?
Gain insights into typical behaviors and context for the use of your product with year-over-year brand performance ratings across mobile web + mobile app across iOS and Android.
Play-by-Play Screenshot
Get full access to a library of screenshots of your competitor’s latest mobile designs indexed by feature, brand, and release date.
Dynamic Change Tracking Tables
See when top brands in your industry update, remove or add new mobile features.
Features Ranked By Consumers
View results from over 500 active consumers on the mobile features they value most in your industry.
Insights
Unbiased Third-party Mobile UX Research
The Competitive Intelligence you need to stay ahead of your competitors. Competitive Insights, an intuitive competitive intelligence software, incorporates consumer preferences of features and capabilities that they consider critical for a successful experience including:
A complete listing of overall rankings
A detailed scoring breakdown of all capabilities and features reviewed
Screenshots from behind the login screens of all competitors sites & apps
A comparative list of current mobile capabilities available on each mobile app
Detailed descriptions of how users can complete tasks on the mobile device
User feedback collected regarding feature preferences
Identifies key success factors, best-in-class features, areas for improvement, and why some companies are lagging behind
Chart your course for a better customer experience today
Meet with a KLI Team Member
{
"These rankings acknowledge our continued focus on providing both current and prospective State Farm customers with the same great experience, regardless of how they come to us"
Patty Gaumond / State Farm
"Our C-suite asked us to find an A-Team… Having worked with Key Lime at a previous employer, I knew that there was no better team than them for actionable insights and strategic design recommendations.”
VP of Digital Strategy & Innovation / Miami Heat
“…this methodology provides the details we need in order to focus our development plans in the coming year....”
The Mobile Banking Competitive Insightsprovided by our innovation arm, Key Lime Labs, summarizes these new, successful, and well-received features across the banking industry so that both industry leaders and institutions, large and small, can make at-a-glance strategic decisions. The report features key areas that drive user satisfaction and quickly identifies areas of improvement to satisfy existing and prospective customers. KLI’s research experts highlight opportunities based on a rigorous independent evaluation and that ultimately identifies trends and opportunities that move the entire industry forward.
Mobile Banking Competitive Index Overview
In this detailed report, KLI profiles the competitive mobile landscape of eight (8) of the top US retail banks, and ultimately reveals which bank offers its users the best mobile solutions:
State Farm
Bank of America
Chase (JP Morgan Chase)
Citibank (Citigroup)
Pen Fed
U.S. Bank
USAA
Wells Fargo & Company
The evaluation includes a general discussion about what these banks are doing to provide for their existing and prospective retail banking users via mobile-optimized sites and apps (iOS and Android). Data is captured from actual users of each bank and includes screenshots of features from behind-the-login screens of each provider.
Additionally, Key Lime Interactive (KLI) shares the methods employed to rank the banks based on consumers’ self-reported preferences and insights; provides a best-in-class feature list, as well as a comparative list of the available features of the six; and identify trends and opportunities that bridge the gap between consumer expectations and the current banking solutions. This consumer-based solution differs from other similar reports as KLI uses these consumers’ preferences to weigh the importance of features.
An expert capabilities assessment, comparative feature ranking, and user ratings were combined to determine an overall score and ranking.
User feedback is incorporated by gathering responses (n=500), to understand which features and tasks were must-haves when it comes to interacting with the mobile property of a retail bank. From those results, KLI prioritized user’s needs and preferences, and then placed the features into four tiers relative to the importance and these tiers played a part in the weighted score we awarded each feature when tabulating a final score for each mobile property.
Additional information about KLI’s methodology is detailed in the detailed report. Request Detailed Report.
The evaluation includes a general discussion about what these companies are doing for their existing and prospective customers via mobile-optimized websites and apps (iOS and Android). Data is captured from actual customers of each issuer and includes screenshots of features from behind-the-login screens of each provider.
Each year the benchmark will examine sites’ top problem areas from the previous year and any site redesigns/changes made subsequently, to answer the question: Have the sites made changes that have improved or worsened the user experience?
Access Report
Request a Competitive Insights Subscription
Best-in-class Mobile CX Comparisons
Gain insights into typical behaviors and context for the use of your product with year-over-year brand performance ratings across mobile web + mobile app across iOS and Android.
Play-by-Play Screenshot
Get full access to a library of screenshots of your competitor’s latest mobile designs indexed by feature, brand, and release date.
Dynamic Change Tracking Tables
See when top brands in your industry update, remove or add new mobile features.
Features Ranked By Consumers
View results from over 500 active consumers on the mobile features they value most in your industry.
Insights
Unbiased Third-party Mobile UX Research
The Competitive Intelligence you need to stay ahead of your competitors. Competitive Insights, an intuitive competitive intelligence software, incorporates consumer preferences of features and capabilities that they consider critical for a successful experience including:
A complete listing of overall rankings
A detailed scoring breakdown of all capabilities and features reviewed
Screenshots from behind the login screens of all competitors sites & apps
A comparative list of current mobile capabilities available on each mobile app
Detailed descriptions of how users can complete tasks on the mobile device (e.g. pay a bill, look at account history, etc.)
User feedback collected regarding feature preferences
Identifies key success factors, best-in-class features, areas for improvement, and why some companies are lagging behind
Chart your course for a better customer experience today
Meet with a KLI Team Member
{
"These rankings acknowledge our continued focus on providing both current and prospective State Farm customers with the same great experience, regardless of how they come to us"
Patty Gaumond / State Farm
"Our C-suite asked us to find an A-Team… Having worked with Key Lime at a previous employer, I knew that there was no better team than them for actionable insights and strategic design recommendations.”
VP of Digital Strategy & Innovation / Miami Heat
“…This methodology provides the details we need in order to focus our development plans in the coming year....”
(Published Bi-Annual in May and December) The Auto Insurance Mobile UXCompetitive Insightsby our innovation arm, Key Lime Labs, summarizes key features across the industry with the click of a button so that both industry leaders and small institutions can make at-a-glance strategic decisions. This detailed analysis features key areas that drive user satisfaction and quickly identify areas of improvement to satisfy existing and prospective customers. KLI’s research experts highlight opportunities based on a rigorous independent evaluation and that ultimately identifies trends and opportunities that move the entire industry forward.
Auto Insurance Competitive Index Overview
In this detailed report, KLI profiles the competitive mobile landscape of the eight largest US auto insurance providers:
State Farm
Allstate
GEICO
Progressive
Nationwide
Esurance
USAA
Liberty Mutual
The evaluation includes a general discussion about what these companies are doing for their existing and prospective customers via mobile-optimized websites and apps (iOS and Android). Data is captured from actual customers of each issuer and includes screenshots of features from behind-the-login screens of each provider.
KLI shares the methods employed to rank the providers based on consumers’ self-reported preferences and insights; provides a best-in-class feature list, as well as a comparative list of the available features; and identify trends and opportunities that bridge the gap between user expectations and the current mobile auto insurance solutions.
To create an overall score, we combine a capabilities assessment with user feature importance ratings. The capabilities assessment is created by examining the feature coverage of the auto insurance sites and applications. User ratings are determined by a consumer survey and card sort (n=500), to understand which features and capabilities are must-haves when interacting with the mobile property of an auto insurance provider. From those results, we were able to understand what users believed to be most important to them. We use this consumer data to determine the weighted score we give each feature when tabulating a final score for each mobile property.
Additional information about KLI’s methodology is detailed in the full report.
What’s in this Analysis?
In summary, this Competitive Insights report includes:
A complete listing of overall rankings
A detailed scoring breakdown of all capabilities and features reviewed
A complete list of best-in-class features
A comparative list of current mobile capabilities available on each mobile app
Detailed descriptions of how users can complete tasks on the mobile device (e.g. pay a bill, file a claim, etc.)
User feedback collected regarding feature preferences
A comprehensive list of all mobile apps currently available by these eight companies
The evaluation includes a general discussion about what these banks are doing to provide for their existing and prospective retail banking users via mobile-optimized sites and apps (iOS and Android). Data is captured from actual users of each bank and includes screenshots of features from behind-the-login screens of each provider.
Each year the benchmark will examine sites’ top problem areas from the previous year and any site redesigns/changes made subsequently, to answer the question: Have the sites made changes that have improved or worsened the user experience?
Gain insights into typical behaviors and context for the use of your product with year-over-year brand performance ratings across mobile web + mobile app across iOS and Android.
Play-by-Play Screenshot
Get full access to a library of screenshots of your competitor’s latest mobile designs indexed by feature, brand, and release date.
Dynamic Change Tracking Tables
See when top brands in your industry update, remove or add new mobile features.
Features Ranked By Consumers
View results from over 500 active consumers on the mobile features they value most in your industry.
Perks
Unbiased Third-party Research
The Competitive Intelligence you need to stay ahead of your competitors. Competitive Insights, an intuitive competitive intelligence software, incorporates consumer preferences of features and capabilities that they consider critical for a successful experience including:
A complete listing of overall rankings
A complete list of best-in-class features
A detailed scoring breakdown of all capabilities available on each mobile app
Detailed descriptions of how users can complete tasks on the mobile device (e.g. pay a bill, look at account history, etc.)
Identifies key success factors, best-in-class features, areas for improvement, and why some companies are lagging behind.
Chart your course for a better customer experience today
Meet with a KLI Team Member
{
"These rankings acknowledge our continued focus on providing both current and prospective State Farm customers with the same great experience, regardless of how they come to us"
Patty Gaumond / State Farm
"Our C-suite asked us to find an A-Team… Having worked with Key Lime at a previous employer, I knew that there was no better team than them for actionable insights and strategic design recommendations.”
VP of Digital Strategy & Innovation / Miami Heat
“…This methodology provides the details we need in order to focus our development plans in the coming year....”
Are you a User Experience professional who uses online survey tools to deliver insights? If so, you’re in luck! For the last four years, I’ve been working extensively with various online survey tools to deliver everything from simple one-off consumer surveys, to large scale multinational Competitive Benchmarking tests. Throughout that time, I’ve had endless opportunities to experiment with different design methods and survey tools – and to make mistakes, and learn from them – so that you don’t have to. In this article, I would like to share with you some of the potential pitfalls to designing and programming these studies that you can avoid in your next survey. Proper survey design can save you countless hours of frustration when it comes time to analyze the data and deliver your report.
Not Scripting to Your Reporting Tool
Sometimes when you script a survey, you want branching pathways for users who are successful or not, so you create multiple paths. For instance, in UserZoom, you can have a “Success Questionnaire” and an “Error Questionnaire” depending on a particular question. If you only want to look at the success group and the failure group individually, that’s a perfectly sound approach. However, if you want to look at any of those answers cumulatively in the results, you’ll now force yourself to manually compile answers to the same question from those two questionnaires. If you do this across multiple tasks and multiple studies, suddenly you’ll find yourself doing busywork that could have been avoided, had you taken some time to assess how these results would look in the reporting tool. If you’re unsure, run a preview or soft launch with yourself as the participant, and see how the data looks. This could save you hours of time when you get to the analysis phase, trust me!
Not Making the Most of a Tool’s Capabilities
Knowledge of the survey tool you’re using is extremely valuable when scripting. For example, many survey tools allow you to tag your questions with an alphanumeric code, allowing easier identification when you export the results. Taking a moment to label your questions with clear, concise tags will make your analysis phase easier and less prone to errors.
Script Once, QA Twice (or more!)
Okay, the old adage is measure twice, cut once, but you get the picture. It’s important to lock everything down before you go into field. If you make sure that you have everything you need before gathering results, you avoid common pitfalls like leaving out a key question, or any number of logic issues that could tank a survey. Survey software typically makes it difficult to make script changes once the study has launched, so you could end up throwing away data from days of fielding. That’s why I recommend at least two QA’s, one from the panelist or participant perspective, and one from the client perspective. Ideally this QA will be done by another member of your team, not the person who wrote the script. Experienced survey designers know that it’s easy to develop blind spots for your own script. A proper QA should first take in the participant point of view, making sure the instructions make sense to someone with no knowledge of what is being tested. The second QA should both verify logic and setup, but more importantly, map back the study design to the goals of yourself or your client. This added verification can prevent costly mistakes and time lost when the study goes live.
The Kitchen Sink
Finally, the kitchen sink. It’s tempting to shove everything you can into a survey – especially when pressure mounts from your client and stakeholders – but remind them that the most elegant surveys avoid unnecessary questions and focus on what is most important. It’s of paramount importance to minimize survey fatigue, a real problem that lowers response rate and quality. A good rule of thumb for the longest surveys is a length of 20-25 minutes maximum, and that’s stretching it. Even at 20 minutes there is a large drop off on quality comments near the end of the survey. You may end up throwing out results that would have been valid in a 15-20 minute survey. Ask yourself, or your client, “Do we want 50 questions of middling to poor data or 35 questions with high quality responses?”
That’s all for now. I hope you’ve found this educational, or at the very least, entertaining! Subscribe to our newsletter for monthly articles and updates.