Their theme: Make it a priority to plan ahead AND be culturally sensitive/aware.
Infinite perspectives are available around the globe, spanning multiple languages, habits, patterns, cultural priorities, etiquette and more. The requirement to meet the needs of a diverse audience is clear, and with that comes diversity of your recruited panel. In a recent study that spanned four continents, Key Lime’s Eugenio Santiago, VP of User Research, noticed that the stakeholders in the research weren’t aware of the relevant needs of each country when they outlined their vision of a successful project. “That’s my job.” Eugenio stated. “I have to make sure that all of the nuances that organically associate with a culture are considered from the early stages of research design. I need to rely on strong in-country partnerships to bring the surface any minute detail that I may have overlooked.”
Key Lime leverages their UX Fellows global partnership for this. With a mission to make “international user experience and usability testing as easy and professional as domestic studies,” there are checklists in place to ensure success. Simple things ranging from proper power requirements, to considering the time of day you’re asking users to test, to understanding pace punctuality; customary greetings are all to be part of a successful testing experience.
Other panelists stressed the importance of building in time for debriefs and dialogue with all parties involved solidifies strong findings and reporting. The in-country perspective is invaluable when reporting user satisfaction.
When speaking with the audience, it was clear that international work was on the rise. Much of the feedback was that folks were already testing in a global setting, or in the process of preparing themselves for this leap. They happily walked away with what was the start of their own actionable checklist of considerations to be successful testing around the world. A few points to get you started as you build your own list:
Consider Cultural Elements:
This goes beyond language. Customary greetings, pace, punctuality, and etiquette should be considered. For example: One panelist reported that during their first global foray they planned to field a study that investigated the way users interacted with a product in their homes in India. They created a study plan that included six thirty-minute home visits in a day, and presented it to their Indian Research Partner. He took one look and said “Oh no. Not possible.” Why? Because traffic wasn’t being considered. It was assumed that 50% of the day would be spent getting around, so three in a day was pushing it. Adjustments were made in advance vs. sweating it out; rearranging on the fly and failing to meet the schedule.
Consider Technological Capabilities and Be Prepared:
“You may run this exact study 355 days a year in NYC, but when you endeavor to replicate in a different country outside of the US on that one day, you can’t take the ‘little things’ for granted”, says Santiago. Wifi, power, keyboard setup, and a host of other things need to be considered. To overcome unforeseen challenges, the panelists advise to stay on the side of caution. “Backup web connectivity options if the facility can’t provide what you need.” Of course, there are times when that connectivity needs to be considered as PART of the test, but we need to do our best as researchers to isolate the technological variable to best influence design.”
Have a Contingency Plan:
Eugene opened with a funny story about an awkward time when he experienced an in-country moderator and a client bang heads mid study. “My observation, even as a non-native speaker, was that these two had opposing viewpoints of the way this research was to go down, and they were making it known. By the end of the day I had to make significant changes to satisfy the client.” He had done his homework and let two other facilities and two other moderators know that Key Lime was in town testing, and that he may need their help if he runs into a pickle. That simple notification gave him an option to make a moderator change for the remainder of the study.
“Aha moment tips for global studies. Don’t forget your international power converters.”
“Schedule downtime to research when fielding for long 2-week periods.”
“Eugenio shares the importance of multi-tasking and using our partnership with @UX Fellows for global reach.”
“@Google is experimenting with a new approach to shorten the study time since stakeholders want answers now.”
“@eBay speaking about importance of translation and the challenge of emotional measures.”
More from the panel:
Anosha Shokrpour Groupon @anoshas
With 5+ years of experience in the field at eBay Inc. for most of her tenure and now at Groupon, Anosha has a deep understanding of the international e-commerce landscape that spans across developed and emerging markets.
Donna Tedesco
eBay Inc.
Donna Tedesco is a Senior User Experience Researcher at eBay Inc. She has experience using moderated and unmoderated studies for global research while working for in-house research teams over the last 13 years.
Chelsey Glasson Google
Chelsey is a user experience researcher whose skills have impacted a wide variety of enterprise and consumer technologies at diverse companies including Google, Salesforce, Udacity and T-mobile. Motivated to help others avoid her early UX career mistakes, she often writes and gives talks on the topic of UX Careers.
Sin Lee Loh
eBay Inc.
Sin Lee Loh is a user experience researcher at eBay, currently focused on buyer experiences but previously focused on conducting research in Latin America and Russia. Sin most recently conducted ethnographic research in Brazil.
Eugenio Santiago Key Lime Interactive
Eugenio Santiago is a ninja in user experience research helping the world’s most admired brands optimize their digital and product experiences. As Vice President of User Research, he manages the team of both qualitative and quantitative researchers. Eugenio has been recognized by clients for his ability to quickly spot patterns and provide actionable recommendations. Most recently, Eugenio’s focus has been in the areas of Mobile, Finance, and Retail while still maintaining a passion for sports and gaming.
The Gaming Analytics Summit held in San Francisco brought together a nice crowd of headliner video games such as Minecraft, Call of Duty, Destiny, Angry Birds, and Candy Crush. In attendance were the big gaming giants such as Sony PlayStation, Xbox, Activision, and Electronic Arts. Being an avid gamer and data analyst made this conference extremely informative. The topics ranged from in-game analytics to building a company structure that best handles big data. My focus at this conference was to see how the user’s voice was being heard in the video game development pipeline. Qualitative interviews meant very little to this group who focus more on big data and analytics, but some companies set themselves apart by emphasizing the importance of the user in maximizing their earning potential.
With so much data available from in-game selections, purchases, and behaviors; capturing and analyzing data in such volume has to be highly efficient, lightweight, and funneled into a visualization that is simple enough to consume and draw conclusions. Sega’s entire presentation was about the importance of simplicity and consistency in analysis and visualizations. It clearly demonstrated the challenges of presenting huge bar graphs in reports that are difficult to digest. Following Sega’s presentation, I noticed a theme: Big Data, Big Results, Now What? Attention was placed on displaying data, but not on determining the next course of action.
Candy Crush’s presentation also grabbed my attention. The presenter offered one listener a choice between a mobile power pack and a Rubix cube. The listener chose the Rubix and the speaker said, “now that we know what he chose we can determine some things.” I spoke up during Q&A. “My question throughout the presentation was ‘Why did he choose the Rubix? Doesn’t understanding ‘why’ make your content delivery algorithms more relevant?” He was a bit perplexed and said they just try to do their best to analyze the data they have to learn about users. I responded, “Wouldn’t it be easier just to ask?” It seemed that there was little attention paid towards why users behave the way they do. All focus was placed on A/B testing to determine the best conversion rates. While this method may work, it also presented a very wasteful practice of blind A/B testing.
Just when I thought the user was being completely left out, Alex Leavitt from Sony PlayStation emphasized the importance of User Research in his presentation. He mentioned that his focus was on “Game Science”, which is comprised of game analytics, user research, and gaming design. He continued that integrating user research data into the design process is critical to challenging developers’ design intuitions. The slides shown in the picture demonstrate the need for user research to be experience focused, data informed, and player driven; but also that it should be interwoven into the entire development cycle.
The smoking gun to tie everything back came in a case study of Angry Birds development company, Rovio. This study focused on re-activating gamers that have not been playing as frequently. More activity means more chances they will pay for something in-game. The solution was relevant content. Behavioral patterns, feedback, and ratings were used to better personalize the in-game rewards and messaging, which significantly improved conversion rates and reactivation. Minecraft followed suit by emphasizing the importance of building gamer personas to better understand users. The use of gamer personas by Minecraft demonstrated a big data trend for personalization. Personas may not give the details of every type of user, but it does create a personal connection to a type of users that may be a large portion of your customers. Personas can help narrow the gap between advertising and intrusion. Knowing the needs of the gamer and serving them relevant content or preferred in-game rewards makes a game more addicting and more profitable.
Just as smartphones and tablets displaced the once-dominant PC, wearable technology has begun to take over the tech-scene. They have been projected to reach $4.5 billion in revenue this year and $53.2 billion in global retail revenue by 2019 (Juniper Research). These devices are popping up everywhere from smartwatches, fitness bracelets, and even smart-clothing and jewelry, but what if your wearable gadget could potentially save and/or prolong your life?
It is probably safe to say that we have all heard of, worn, or witnessed someone wearing an insulin pump, glucose monitor, hearing aids and/or prescription eye glasses, but we now live in a world where science fiction has turned reality. A world where contact lenses are embedded with microchips that monitor your health while improving your vision (Google and Novartis, “Smart Lens”). A vest that could save your life from sudden cardiac death (Zoll, “LifeVest”). A ‘build your own’ onesie that can detect your infant’s breathing and movement to help reduce SIDS (Mimo, “Smart Infant Monitor”). IBM, Apple, Medtronic, and Johnson & Johnson are even teaming up to develop the HIPAA-enabled “Watson Health Cloud” to collect the estimated million gigabytes of data per individual.
The age of wearable devices is upon us and a revolution in healthcare has emerged. Consumers, Physicians and the medical community, as well as investors, need to know that they can trust the data from the product, so accuracy and availability is essential! Since the risk of life or limb from a fitness bracelet is obviously not as high as the “LifeVest”, safety guidelines have already been drafted to regulate wearable tech devices, as well as the smartphone apps that accompany them. The FDA released its initial draft guidance for General Wellness: Policy for Low Risk Devices on January 20, 2015 and its draft guidance for Medical Device Data Systems, Medical Image Storage Devices, and Medical Image Communications Devices on February 9, 2015. This guidance identifies that they will not regulate “wellness” wearables that encourage a healthy lifestyle or reduce the risk of developing a chronic disease, but wearable Medical Devices that claim to treat, diagnose, or restore a structure or impaired function due to a disease, however, are regulated the same as all other medical devices and do require FDA approval. Basically, your Fitbit isn’t going to be regulated by the FDA as long as its claims are preventative and in a low-risk category. The FDA’s intention is to continue to focus their resources on technology products that pose a higher risk.
These devices offer numerous solutions to patients as well as to physicians and home healthcare professionals, equipping them with tools to track and manage the patients’ health information easily and effectively – ranging from medical and safety critical, to leisure and entertainment. In the case of medical and safety critical, this creates unique concerns and imposes new constraints that existing human factors theories may not fully support.
General Wellness: Policy for Low Risk Devices (Draft Guidance) (January 20, 2015)
Medical Device Accessories: Defining Accessories and Classification Pathway for New Accessory Types (Draft Guidance) (January 20, 2015)
Medical Device Data Systems, Medical Image Storage Devices, and Medical Image Communications Devices (Final Guidance) (February 9, 2015)
Mobile Medical Applications (Final Guidance) (February 9, 2015)
There were two topics in particular that I found most interesting at this year’s NetFinance conference in Miami:
Digital strategies to support the Omni-channel experience
Is Personal Finance Management (PFM) dead?
The first topic was a major focus of this year’s conference (two of three days devoted to Omni-Channel), so I found it rather ironic that the guest speaker, Chris Skinner, who was invited to talk on Day 1 of Omni-Channel has consistently stated “there is no such thing as a channel.” His belief is that in order to be a truly digital bank, which supports the 24*7 real-time world, the core of the bank needs to be a singular digital one.
With this digital core, a bank can deliver the real-time access to information that banking customers now desire. A digital core will also enable banks to provide a consistent experience regardless of how customers choose to access their bank (e.g., mobile, in-person, smart watch, etc.).
Having read Chris’ book: Digital Bank a few months ago, I agree. The outdated Omni-channel structure where legacy sits on top of legacy enables disruptors like Simple and Moven to quickly enter the arena of banking and make huge waves. The transition to a digital core sounds both costly and a tall mountain to climb, and it is, but it’s an evolution that needs to happen for banks to move past becoming a ‘mobile’ bank and towards becoming a digital bank.
The second topic, is PFM dead, was of particular interest to me because it aligned with some research I conducted in 2014. If your definition of PFM is tools by which customers can self-manage their spend and create budget goals themselves, then yes PFM is dead. But if you think about PFM as the usefulness that can be derived from this type of analysis, then no, PFM is NOT dead but very much alive.
Our competitive research has shown that banking consumers (those who are affluent and to a lesser extent all consumers) not only find this type of information useful, but they want their banks to be the provider of such insight. It is not enough for banks to simply present this data to consumers, but rather share this insight in both a contextually relevant and actionable way. Consumers want their bank to be a ‘trusted advisor’.
Banks have a wealth of information, historical and current behavior, to leverage that can enable them to better advise their customers of the behaviors and/or products and services they can benefit from to become more financially secure or achieve their goals. Making sure the message is goal oriented and clear for consumers to see how these actions will benefit them is critically important. However, the push to new products/services needs to be subtle and NOT the goal of PFM 2.x. The value of these types of insights can go a long way to instill loyalty, a more personalized touch, and a greater sense of financial confidence for your customers.
If you are interested in learning more about either of these topics, how to execute PFM 2.x, or are interested in how Key Lime Interactive can assist you in your transition to becoming a digital bank, please feel free to reach out to me.
The transition from Omni-channel to a digital core will happen faster than you think, so stay informed and ahead of the curve.
by Rick Damaso
Let’s face it. We have become a “Google it” society. We can answer almost any question in .00012 seconds (according to Google’s search results page) and get a pretty accurate response. So, before attending the LeanUX15 conference in Brooklyn, NY, I wondered what would Google’s response be for UX’s impact on modern day Systems Development Life Cycles (SDLCs)? Was it even possible to wrap terms like DevOps, Kanban, Lean or Agile into a neat package with a UX bow on top?
More importantly though, does this matter? Or were these terms just the buzzwords from the “The Valley” that larger companies all around were simply talking about emulating? Do new philosophies really only announce themselves when blue chip companies start adapting them?
To say it depends is a boring answer. But, of course, it depends. As a researcher, however, I wanted a straight and narrow answer.
I thought it would be prudent to first set the stage. Here are some quick snippets of what a quick Google Search will give you relating to these new SDLCs and UX. Agile- Developers focus on sustainable development. Sustainability is about good estimation, effective branching strategies for managing code, automated testing to protect quality, and continuous deployment to get fast feedback from users. DevOps- DevOps is a software development method that emphasizes communication, collaboration (information sharing and web service usage), integration, automation, and measurement of cooperation between software developers and other IT professionals. Kanban- Technique for managing a software development process in a highly efficient way. Producing software is a creative activity and therefore different to mass-production (Kanbans’s roots are in auto manufacturing) allowing us to apply the underlying mechanism for managing “production lines”. Lean UX- Lean UX is a set of principles that may be used to guide you to better, more desirable solutions for users. It’s not a process in which each tool is rigidly applied. Instead a group of ideals and principles to guide you in the design process.
So, that makes sense. If I was a designer sitting in a room with other designers, communication and putting this philosophy in practice shouldn’t be that difficult in theory. But, when you take these practices out of Silicon Valley and introduce it to the landscape of companies like Microsoft or Spotify with teams of designers on separate continents, it can make your head spin. How could you package what looks and feels like a startup mentality and scale it up effectively?
LeanUX15 took this challenge head on during a four day event in Brooklyn, NY. This conference was not all that different from any other, unless you consider flamingo colored windbreakers and Paul Bunion beards different. But, all “hipster” jokes aside, the usual laid-back vibes you find in the heart of Brooklyn’s Williamsburg neighborhood was noticeably different. Product managers, Lead Designers, Software Engineers and representatives from some of the world’s most iconic brands were buzzing with excitement on bringing these results driven practices to companies everywhere.
As UX researchers, we encounter organizations along all stages of the UX maturity cycle and work on projects from formative to summative stages, so I was pleased to hear that UX research is taking a prominent role in these SDLCs. As opposed to traditional validation testing, we were now seeing rapid production of software married with UX researchers, architects and designers alike.
So what does this force us to do? To borrow from one of the themes of the conference, it means we are now in the age of Designing for Service, Not Just Software.
Here are some quick hits as to how UX is impacting development and strategic vision:
Visualize your work, in knowledge clusters. Ideas are then disseminated to users as solutions in terms of their problems.
With a UX lens, setting realistic Work In Progress limits for each stage of production is critical. Accounting for time slots within stages for user testing as opposed to piecing it together at the end.
Manage flow to clearly identify bottlenecks and accurate metrics. When infusing UX research into your design process, you are hedging against expensive “revamps” at the tail end of your SDLC.
Make Policies Explicit. Stick to your design, research and implementation policies! However, the #1 policy should always be, “If one of your policies does not work, change it”. By first following your process and analyzing what is wrong, you will be in a much better position to fix it.
Implement Feedback Loops. Communicating accurate measurement with your target market is key. Measurements need to be relevant to the timing of your project, not “at the end of each quarter” or when “you have time for it”.
Empowering yourself and your team to think- you are allowed to think and change processes. These SDLCs are not recipes, instead they are thought of as disciplines. Every question you ask yourself must be phrased as, “is this a driving force to consider design for servicing users or just designing software?”
The message for us as researchers when entering a new frontier of rapid development and testing can be wrapped up with a quote by Prof. Barbara Adam: “The message for research is unambiguously clear: learning is a process with a history and a future; it is thus not containable within observable moments. It entails a joining of life-worlds, a drawing on collective and individual past knowledge as well as projected vision, all of which are brought to bear on the interactive present.”
One of the big themes of nearly every SXSW event we attended was personalization. Even events about the Future of TV had panelists talking about supplemental apps or making sure people could watch on the devices they chose. A news discussion with Dan Rather and Dan Pfeiffer also discussed how people consume news on the platforms of their choice, like Facebook and Twitter. Customers are looking for a more personal, customized experience in the place of their choice.
Predictive technology is making big strides in making these more curated experiences accurate. Facebook’s facial recognition technology is making use of their extensive data on user tagging so they can auto-tag your photographs when you post them. This technology may be more accurate than that of law enforcement. Netflix’s House of Cards was famously made by using data to understand that a political drama starring Kevin Spacey directed by David Fincher would be popular. An important consideration with using predictive data for customer recommendations is providing said data to customers.
Personalization is changing the landscape all over. I went to a talk by Karlie Kloss and Sara Wilson about technology and its role in Fashion Week. Models and editors can now deliver a more personal experience to a massive audience using Instagram and Twitter. They can let people into their lives remotely and enable fan interaction. Vogue recently had a cover featuring nine models with large Instagram followings.
Companies are integrating data into their operations in a variety of ways. Capital One is experimenting with personalized financial recommendations AND personalized offers / rewards recommendations in new apps Ideas and Level Money. Ideas provides recommendations for different types of activities in 4 beta markets: NYC, LA, Richmond, and DC. Level Money lets customers link accounts and program in budgets and receive alerts and content depending on their spending.
For television, companies are looking to make the experience more intimate for viewers. Some companies are experimenting with companion apps, especially in the UK. These might let viewers answer quizzes or play related games while watching. Other companies are trying to make promotions / advertisements more personal. For Game of Thrones Season 5, HBO ran a promotion called The Sight in which people would get text messages with video links that would disappear. The videos would be different for different users but communicate small snippets of information about the upcoming season in the guise of visions / dreams. In Spain, Canal + ran a promotion called 19 Reinos that turned all of Spain into an interactive Game of Thrones-themed game played via multiple different channels: Twitter, Facebook, brand websites, and physical stores. Customers all over the world are looking for targeted, personal experiences. User experience research is one way companies try to identify what kinds of experiences are most valuable to customers. Airbnb mentioned that when they redesigned their website, they made sure to keep the hosts involved in the process since their feedback was critical to its success. There was a fantastic talk by Etsy about how user experience feedback, both from users and from their clickstream data, was extremely valuable to their design process and their feature prioritization. Part of Capital One Labs’ approach to every project is a pilot study with 5-10k end users to understand how they’re using the product.
Stay tuned for more SXSW recaps in the coming months!
Global Research pros from Groupon, eBay, Google and Key Lime Interactive were seated on a panel at the International UXPA Conference event in San Diego in late June. They came together to share their words of wisdom for successful global user testing endeavors.
