The Gaming Analytics Summit held in San Francisco brought together a nice crowd of headliner video games such as Minecraft, Call of Duty, Destiny, Angry Birds, and Candy Crush. In attendance were the big gaming giants such as Sony PlayStation, Xbox, Activision, and Electronic Arts. Being an avid gamer and data analyst made this conference extremely informative. The topics ranged from in-game analytics to building a company structure that best handles big data. My focus at this conference was to see how the user’s voice was being heard in the video game development pipeline. Qualitative interviews meant very little to this group who focus more on big data and analytics, but some companies set themselves apart by emphasizing the importance of the user in maximizing their earning potential.
With so much data available from in-game selections, purchases, and behaviors; capturing and analyzing data in such volume has to be highly efficient, lightweight, and funneled into a visualization that is simple enough to consume and draw conclusions. Sega’s entire presentation was about the importance of simplicity and consistency in analysis and visualizations. It clearly demonstrated the challenges of presenting huge bar graphs in reports that are difficult to digest. Following Sega’s presentation, I noticed a theme: Big Data, Big Results, Now What? Attention was placed on displaying data, but not on determining the next course of action.
Candy Crush’s presentation also grabbed my attention. The presenter offered one listener a choice between a mobile power pack and a Rubix cube. The listener chose the Rubix and the speaker said, “now that we know what he chose we can determine some things.” I spoke up during Q&A. “My question throughout the presentation was ‘Why did he choose the Rubix? Doesn’t understanding ‘why’ make your content delivery algorithms more relevant?” He was a bit perplexed and said they just try to do their best to analyze the data they have to learn about users. I responded, “Wouldn’t it be easier just to ask?” It seemed that there was little attention paid towards why users behave the way they do. All focus was placed on A/B testing to determine the best conversion rates. While this method may work, it also presented a very wasteful practice of blind A/B testing.
Just when I thought the user was being completely left out, Alex Leavitt from Sony PlayStation emphasized the importance of User Research in his presentation. He mentioned that his focus was on “Game Science”, which is comprised of game analytics, user research, and gaming design. He continued that integrating user research data into the design process is critical to challenging developers’ design intuitions. The slides shown in the picture demonstrate the need for user research to be experience focused, data informed, and player driven; but also that it should be interwoven into the entire development cycle.
The smoking gun to tie everything back came in a case study of Angry Birds development company, Rovio. This study focused on re-activating gamers that have not been playing as frequently. More activity means more chances they will pay for something in-game. The solution was relevant content. Behavioral patterns, feedback, and ratings were used to better personalize the in-game rewards and messaging, which significantly improved conversion rates and reactivation. Minecraft followed suit by emphasizing the importance of building gamer personas to better understand users. The use of gamer personas by Minecraft demonstrated a big data trend for personalization. Personas may not give the details of every type of user, but it does create a personal connection to a type of users that may be a large portion of your customers. Personas can help narrow the gap between advertising and intrusion. Knowing the needs of the gamer and serving them relevant content or preferred in-game rewards makes a game more addicting and more profitable.
I received my Apple Watch this past Thursday. I chose the space grey Apple Watch sport with the black band, which was worth the wait. It’s fairly subtle, with one person (okay, a kid!) thinking it could be a real watch. Overall, I am impressed with its performance, especially for a v1 device with limited connectivity options. Powered by my iPhone 6, even on LTE instead of wireless, there is very little lag in most apps. However, the remote app has some issues connecting to iTunes libraries. It’s fantastic as a remote for the Apple TV, but very limited and challenging to sync with my computer’s iTunes library.
Performance at home is fantastic. I was able to leave my phone in my bedroom and wander all over my apartment with the watch. I made calls on it of durations between 30 and 40 minutes with no problems. I will say the speakers could be a bit stronger, though. It’s hard to hear people if they’re speaking quietly, or also on speakerphone. Messages and alerts come through in real-time, though. Pleasantly, if you’re interacting with an app on another device you do not receive an alert on the watch. While this makes sense, it isn’t true for the iPhone/iPad, so it was a great software addition that should come to more devices in the family.
I was deeply impressed with its performance in transit. Using Bluetooth, the watch is still connected to your phone so you can change music or get activity updates while underground with no cell service. Where there is cell service, it will push notifications to you. I was expecting the watch to be fairly useless while traveling, but that is certainly not the case.
It’s useful while at work. Again, the performance over LTE has few noticeable lags for any app, apart from maybe 5-10 seconds sometimes for NYT updates. The calendar alerts are fantastic. They pop up 10 minutes before your meeting and let you scroll through all of the meeting details. There’s even an option to email the meeting creator, which is the only email option I’ve seen on the watch so far. The dictation is good enough that I wish they allowed text responses to emails. It would be a really useful update. My biggest frustration while using it at work was when I went out of range for a meeting in a far conference room. I didn’t bring along my phone because the watch was a great substitute, but it didn’t alert me as I was exiting its range. Some sort of notification would be helpful, as it’s challenging to gauge distances, especially inside buildings.
The messages app is delightful to use. Being able to dictate messages makes it extremely functional. However, the feature could be improved by making it easier to edit these messages. I’ve definitely found myself canceling messages and re-dictating them due to one or two incorrect words in places that would make overall comprehension challenging. I would also like to be able to send the messages without having to touch the watch. There currently isn’t a verbal command that lets you send a message. Despite these usability challenges, I still found myself sending the majority of my text messages this weekend using the watch. It’s the easiest way to send text messages I’ve seen so far, though it would definitely be improved by easier (or any!) editing capabilities and a way to send without touching it.
Email is surprisingly functional on the watch. Initially, I assumed it would be just notifications, but you can scroll through the entire email. Not everything renders on the watch, especially graphics, but you can see the entirety of provided text, which is very useful. My biggest pain point when using the email feature was how difficult it was to delete emails. When I clicked on a notification, I had to scroll through the email to get to the delete option. In your mailbox you can swipe for a trash option, but as a notification that only gives you the option to clear notifications. Being able to delete from the notification without scrolling through the whole thing would be a useful addition.
My largest gripe centers around Apple Pay. Figuring out how to add a card to the watch was NOT intuitive. It kept directing me to my phone, but I assumed it was the regular Passbook section. I tried re-adding my card, but it didn’t let me. I had to Google how to do it to find out it was in the Apple Watch app on my phone. Even then, I had to re-verify my card for the watch by calling my credit card company. When I tried to use it at Whole Foods by tapping the button twice it kept telling me it was ready, but ultimately it was unable to make the payment. Obviously, this was pretty frustrating. I ended up using my phone. Seeing as the watch is likely one of Apple’s best chances at making Apple Pay catch on, it’s a shame this was the least intuitive watch experience I had all weekend. This experience should definitely be improved. The Apple Pay on-boarding would have been easier with a diagram clearly illustrating where to go on the phone. The BEST solution would be letting me choose on the watch whether to add the credit cards from my phone. I don’t see why I need to go through the phone. I’m not sure why it doesn’t work in stores, but that’s definitely a huge issue that needs to get fixed.
The native activity app is interesting. I’ve given it a small amount of information and it’s been making attempts to inspire me to greater efforts. I personally am not a super active person, but what I like about the activity app as it exists currently is that it works with you. It’s not being overly critical or alerting me too frequently, both of which would result in me turning it off. It’s sitting there in the background letting me know when I’ve hit a goal or reminding me when to stand up. I don’t listen to every stand reminder, but I’ve listened to more than I’ve ignored. I’m curious to see if it changes my behavior over time. It’s definitely a much better way to interact with this information than the Health app on the iPhone, which I’ve always found oddly buggy.
Of the 3rd party apps I’ve interacted with so far on the device, I’m most impressed with the New York Times app. They’ve done a wonderful job of creating a new kind of article specifically for the watch. Some articles feature just a headline, some have pictures, and some have 1-2 sentences. It’s a fun surprise to scroll through them a few times a day. I do hope in the future it’s possible to read full shorter articles on the device, but I understand their choice and think it makes a lot of sense for the watch that exists today. 95% of my interaction with the NYT iPhone app is through notifications, so NYT on the watch is an ideal match. Now I actually get more information with the brief articles and images. I prefer the tablet for actual reading, but again I would be interested in having a more email-like experience for the NYT.
While I was initially unimpressed with the battery life, it was fine over the weekend. It does drain my phone battery faster, BUT it means I’m spending significantly less time on my phone so that evens it out for the most part. Like all Apple devices, I would appreciate a longer battery life, but I will say it survives a 12-hour day much better than the iPhone. Having the two devices has made it possible to have weekend days without airplane mode or constant recharging. Speaking of charging, I wish it were possible to wear the watch while charging it. One of the best use cases for me so far has been using the watch to act as an Apple TV remote. I do most of my Apple TV watching at night, so it would be great to be able to plug it in and continue using it. I’m also curious about the watch’s potential as an alarm, given that the taptic feedback might be a more pleasant way to wake up.
At this stage, I would rate the Apple Watch as a ‘nice to have’. If you, like me, own the whole family of devices and upgrade pretty regularly, go for it. It’s an awesome addition to the family, and you’ll find a lot of unexpected uses for it. I think it needs to be able to stand alone, ideally by v2. However, it’s still challenging enough to use that I wouldn’t recommend it to my parents just yet. I do think it will get there, and I will definitely be keeping mine and not returning it. Its best uses for me are: messaging, Apple TV remote, email, and keeping me off my iPhone (supposedly the #1 secret purpose). Those are important enough things in my life that I find value in a device that improves my access to them.
Note to Apple: I would be happy to put a $5 data share plan on it so I could leave my phone behind while at conferences, meetings, bars, parties, etc.
In a recent collaboration with User Experience Magazine I published an article entitled: The Future of UX Research: Uncovering the True Emotions of Our Users. The motivation for writing this article was to expose the UX community to the possibilities of using biometric and neurometric measurements to understand the emotions of our users. Current methods for understanding a user’s emotional response are at best limited, and at worst, entirely inaccurate. As the field of user experience evolves, we need to explore new methods for measuring emotional responses using technologies borrowed and refined from neuroscience and human biology. I also wanted to highlight some of the latest technologies available to user researchers, as well as the challenges of working with these tools.
I was thrilled to receive so many positive comments about the article as well as many questions about using these tools. This recap provides some of the key aspects of using biometrics for user experience research as well as answers to the most frequently asked questions about the topic.
Here are some do’s and don’ts for using biometrics in your user experience research projects:
Do:
Hire specialists on your team with a neuroscience, cognition, or experimental psychology background (preferably a Ph.D.). They will be most helpful during the study design and also during the analysis of your biometric data.
Run lots of pilot tests. Incorporating biometrics can add unforeseen challenges to your study and requires extra practice to ensure that data is collected accurately with actual participants.
Ensure that you have a sufficient sample size. Biometric studies require more participants than a typical usability test to account for the potentially large variability between participants. You will also encounter more situations where data needs to be excluded because due to improper equipment calibration, equipment failures, etc.
Don’t:
Examine biometric data in a vacuum. It is important to triangulate data across a combination of different methods including nonverbal observations, participant retrospectives, surveys & rating scales.
Assume that your data is perfect. Measuring biological and neurological responses, especially with the tools accessible to UX researchers, is not going to be 100% accurate. I would recommend looking at general trends in valence (i.e. positive/negative spectrum of emotions) changes instead of trying to pinpoint exact moments where very specific emotions were felt.
Questions & Answers
There have been a variety of tweets and posts generated as a result of publishing this article. Here are some of the most common questions with my thoughts:
Do participants mind wearing all of that equipment? Does anyone complain about it being uncomfortable?
Before making any purchases, I heavily researched all of the existing hardware currently available to make sure that it was as minimally intrusive to participants as possible. In many cases we are only using one or two measurements at a time. I try to use only one measurement that requires physical contact with the participant, such as wearing a GSR wristband combined with facial analysis that only requires a webcam. At the end of a session I always ask the participant whether wearing any of the gear was uncomfortable for them or if it impacted their experience at all. Most said that once the session began they quickly forgot that they were wearing anything.
Is it [biometrics] worth it? Wouldn’t it just be easier to interview an individual or give them a survey to complete?
We are at the very beginning of an exciting journey to uncover more about how our users are actually feeling. Biometrics is definitely not for everyone, and it is not a useful endeavor for all situations. I actually don’t think that these tools are ready for mainstream UX researchers just yet. The tools and software continue to get better every day, and will likely become more useful to UX researchers within the next few years. However, for those who are very serious about obtaining objective, quantitative measurements about your user’s emotional journey, I would recommend trying out these tools to see if they are useful for your team.
I’m very interested in learning more about biometrics! How did you learn about this topic? Would you recommend any resources?
There really isn’t one place to find out everything you would want to know. Biometrics is still a very new topic for the UX community and most of the resources out there are geared for people in the fields of human biology and neuroscience. I started reading a lot of academic papers to understand how to collect and analyze the data. There are a few worthwhile articles in the ACM Digital Library that provide case studies where biometrics were used in the context of human-computer interaction. I would also recommend looking into each biometric and neurometric measurement (e.g. EEG, GSR, etc.) individually. You can get a basic sense of how these tools work by visiting the websites of the device hardware and software manufacturers. My article provides links to many of the major vendors who make UX researcher-friendly products.
by Rick Damaso
Let’s face it. We have become a “Google it” society. We can answer almost any question in .00012 seconds (according to Google’s search results page) and get a pretty accurate response. So, before attending the LeanUX15 conference in Brooklyn, NY, I wondered what would Google’s response be for UX’s impact on modern day Systems Development Life Cycles (SDLCs)? Was it even possible to wrap terms like DevOps, Kanban, Lean or Agile into a neat package with a UX bow on top?
More importantly though, does this matter? Or were these terms just the buzzwords from the “The Valley” that larger companies all around were simply talking about emulating? Do new philosophies really only announce themselves when blue chip companies start adapting them?
To say it depends is a boring answer. But, of course, it depends. As a researcher, however, I wanted a straight and narrow answer.
I thought it would be prudent to first set the stage. Here are some quick snippets of what a quick Google Search will give you relating to these new SDLCs and UX. Agile- Developers focus on sustainable development. Sustainability is about good estimation, effective branching strategies for managing code, automated testing to protect quality, and continuous deployment to get fast feedback from users. DevOps- DevOps is a software development method that emphasizes communication, collaboration (information sharing and web service usage), integration, automation, and measurement of cooperation between software developers and other IT professionals. Kanban- Technique for managing a software development process in a highly efficient way. Producing software is a creative activity and therefore different to mass-production (Kanbans’s roots are in auto manufacturing) allowing us to apply the underlying mechanism for managing “production lines”. Lean UX- Lean UX is a set of principles that may be used to guide you to better, more desirable solutions for users. It’s not a process in which each tool is rigidly applied. Instead a group of ideals and principles to guide you in the design process.
So, that makes sense. If I was a designer sitting in a room with other designers, communication and putting this philosophy in practice shouldn’t be that difficult in theory. But, when you take these practices out of Silicon Valley and introduce it to the landscape of companies like Microsoft or Spotify with teams of designers on separate continents, it can make your head spin. How could you package what looks and feels like a startup mentality and scale it up effectively?
LeanUX15 took this challenge head on during a four day event in Brooklyn, NY. This conference was not all that different from any other, unless you consider flamingo colored windbreakers and Paul Bunion beards different. But, all “hipster” jokes aside, the usual laid-back vibes you find in the heart of Brooklyn’s Williamsburg neighborhood was noticeably different. Product managers, Lead Designers, Software Engineers and representatives from some of the world’s most iconic brands were buzzing with excitement on bringing these results driven practices to companies everywhere.
As UX researchers, we encounter organizations along all stages of the UX maturity cycle and work on projects from formative to summative stages, so I was pleased to hear that UX research is taking a prominent role in these SDLCs. As opposed to traditional validation testing, we were now seeing rapid production of software married with UX researchers, architects and designers alike.
So what does this force us to do? To borrow from one of the themes of the conference, it means we are now in the age of Designing for Service, Not Just Software.
Here are some quick hits as to how UX is impacting development and strategic vision:
Visualize your work, in knowledge clusters. Ideas are then disseminated to users as solutions in terms of their problems.
With a UX lens, setting realistic Work In Progress limits for each stage of production is critical. Accounting for time slots within stages for user testing as opposed to piecing it together at the end.
Manage flow to clearly identify bottlenecks and accurate metrics. When infusing UX research into your design process, you are hedging against expensive “revamps” at the tail end of your SDLC.
Make Policies Explicit. Stick to your design, research and implementation policies! However, the #1 policy should always be, “If one of your policies does not work, change it”. By first following your process and analyzing what is wrong, you will be in a much better position to fix it.
Implement Feedback Loops. Communicating accurate measurement with your target market is key. Measurements need to be relevant to the timing of your project, not “at the end of each quarter” or when “you have time for it”.
Empowering yourself and your team to think- you are allowed to think and change processes. These SDLCs are not recipes, instead they are thought of as disciplines. Every question you ask yourself must be phrased as, “is this a driving force to consider design for servicing users or just designing software?”
The message for us as researchers when entering a new frontier of rapid development and testing can be wrapped up with a quote by Prof. Barbara Adam: “The message for research is unambiguously clear: learning is a process with a history and a future; it is thus not containable within observable moments. It entails a joining of life-worlds, a drawing on collective and individual past knowledge as well as projected vision, all of which are brought to bear on the interactive present.”
by Eugenio Santiago
For all you sports fans I’m sure you’ve seen and formed an opinion about the new ESPN responsive website. Most of the comments I’ve seen have been negative. In fact, most site visitors who have expressed their dislike for the redesigned site have done so in a very colorful way:
I spent a fair amount of time on Apr 2, 2014 reading through thousands of user comments and was only able to find three that were not negative:
I can’t really say they were positive, just not the common variety of “this is the worst thing EVER!” or “is this an April Fools joke?!”.
Now we all have heard the adages: “If it ain’t broke don’t fix it” or “people don’t like change”, but many of the comments were very specific in detailing WHY they disliked the new design. In short, it is different, but more importantly it forces them to change the way they normally interact with the site. No longer are they able to scan headlines, but now must scroll endlessly through entire stories that ESPN deemed newsworthy*. *This has been changed since the writing of this article. The homepage now features bi-lines of top stories instead of displaying them in full. Here’s an example of how site visitors indicated they typically interacted with the site: I would normally come to ESPN for a quick glance of the top stories then, as time permitted, would dig deeper into the topics / articles that are of highest interest to me.
What I read over and over in the user comments was that site visitors would normally use the homepage of ESPN primarily as a means to scan. Therein lies the problem with the new design. It’s much harder to do that now. The number of stories/headlines that are available, above the fold, has decreased.
Those in charge at ESPN have taken their 6+ months of user testing along with their business objectives and have decided that the homepage interaction should now be different. Did they underestimate or misinterpret their data and come away with a design that didn’t meet the needs of their site visitors? Or did they intentionally redesign the homepage in a way that would force users to interact with it differently?
As of the afternoon of April 3, 2014 the answer is clear, they goofed. Midway through my article I noticed a change on the homepage. The endless scroll of the homepage where you had to view full stories before advancing to the next was replaced to only include snippets of the main stories. Scanning is now much easier than its initial release, however, some may still not enjoy the endless scroll. While many site visitors commented the design was yet another example that ESPN didn’t “listen” to them, I’d have to say this is clear evidence of how responsive ESPN can be.
If this is any indication of things to come, I could see ESPN making continued updates to further refine the interaction, but how many customers did they alienate in the first two days of this new release? Since I started this story, not only have they changed how they display stories on the homepage, they have darkened the background from white to a gray to increase the contrast, which too, was another complaint site visitors had.
There still remains much to do. Some secondary pages still have the endless scroll from story to story (e.g., ESPN Cities blog pages), which is even more difficult to deal with when on your smartphone. Navigating horizontally through the site content once you go 1 or 2 levels deep is a major issue, and not every team page has the same layout (e.g., Boston Red Sox = New/Good, Boston Bruins = Old). These are some of the first enhancements that would go a long way to improving the user experience.
New Team Page Layout
Old Team Page Layout
Conducting user research is important, but what you take away from that research is much more important in our opinion. At KLI, we like to call that ‘being data informed rather than data driven.’ ESPN has shown they can be responsive and make on-the-fly changes that correct/improve the user experience, but are you as flexible?
We at KLI pride ourselves in making sure the insights you learn from research are in alignment with user needs and respect, not only what they do, but why they do it. Whether your user research need is at the concept/ideation stage, pre-launch phase, or post-launch phase, Key Lime Interactive is ideally suited to partner with you.
To learn more about what Key Lime Interactive offers and how we can help you with enhancing your user experience, contact us at: 305-809-0555.
One of the big themes of nearly every SXSW event we attended was personalization. Even events about the Future of TV had panelists talking about supplemental apps or making sure people could watch on the devices they chose. A news discussion with Dan Rather and Dan Pfeiffer also discussed how people consume news on the platforms of their choice, like Facebook and Twitter. Customers are looking for a more personal, customized experience in the place of their choice.
Predictive technology is making big strides in making these more curated experiences accurate. Facebook’s facial recognition technology is making use of their extensive data on user tagging so they can auto-tag your photographs when you post them. This technology may be more accurate than that of law enforcement. Netflix’s House of Cards was famously made by using data to understand that a political drama starring Kevin Spacey directed by David Fincher would be popular. An important consideration with using predictive data for customer recommendations is providing said data to customers.
Personalization is changing the landscape all over. I went to a talk by Karlie Kloss and Sara Wilson about technology and its role in Fashion Week. Models and editors can now deliver a more personal experience to a massive audience using Instagram and Twitter. They can let people into their lives remotely and enable fan interaction. Vogue recently had a cover featuring nine models with large Instagram followings.
Companies are integrating data into their operations in a variety of ways. Capital One is experimenting with personalized financial recommendations AND personalized offers / rewards recommendations in new apps Ideas and Level Money. Ideas provides recommendations for different types of activities in 4 beta markets: NYC, LA, Richmond, and DC. Level Money lets customers link accounts and program in budgets and receive alerts and content depending on their spending.
For television, companies are looking to make the experience more intimate for viewers. Some companies are experimenting with companion apps, especially in the UK. These might let viewers answer quizzes or play related games while watching. Other companies are trying to make promotions / advertisements more personal. For Game of Thrones Season 5, HBO ran a promotion called The Sight in which people would get text messages with video links that would disappear. The videos would be different for different users but communicate small snippets of information about the upcoming season in the guise of visions / dreams. In Spain, Canal + ran a promotion called 19 Reinos that turned all of Spain into an interactive Game of Thrones-themed game played via multiple different channels: Twitter, Facebook, brand websites, and physical stores. Customers all over the world are looking for targeted, personal experiences. User experience research is one way companies try to identify what kinds of experiences are most valuable to customers. Airbnb mentioned that when they redesigned their website, they made sure to keep the hosts involved in the process since their feedback was critical to its success. There was a fantastic talk by Etsy about how user experience feedback, both from users and from their clickstream data, was extremely valuable to their design process and their feature prioritization. Part of Capital One Labs’ approach to every project is a pilot study with 5-10k end users to understand how they’re using the product.
Stay tuned for more SXSW recaps in the coming months!
