Persuasive data: How UX researchers can pitch with integrity


A scenario:

Product team asks a question.  Researcher does a study and hands off a long-form academic-style report, but it’s hard to read and there are no clear arguments for what action to take.  The team ignores the report and goes with their gut, and the researcher is frustrated that the team isn’t addressing the top issues from the study.

Later, a colleague suggests that the researcher should change up his reporting style and try to “sell” the results more.  The researcher demurs, saying that researchers have a responsibility to adhere to certain standards for reporting data.  The cycle repeats.

Maybe you recognize this scenario.  Maybe you’ve been the team member with a report you couldn’t use.  Or maybe you’ve been the researcher, delivering meticulous reports that never seem to land right at your company.  If so, you’re not alone — this scenario crops up over and over again, especially for conscientious researchers who have recently made the transition from academia to industry.

Regardless of where you fit in the scenario, we all benefit when we have common expectations about how data should be communicated.  Most non-researchers are comfortable using data casually to support arguments.  But for academically trained researchers, casual use can be harder.  Researchers are trained to have special respect for data.  We can sometimes feel that by interpreting data too heavily, or by arguing for a particular course of action, we’re committing a violation of our professional code of ethics.

Having a code of ethics is essential to the credibility of the profession, but we shouldn’t adhere blindly to a set of standards without thinking about why.  Is it possible for a researcher to have impact at his or her company while still respecting the data?  To answer this we need to start with the source of our reporting standards: academia. Continue reading

Windows 8 Metro UI: Flat isn’t bad

Windows 8 Metro UI

On Wednesday Microsoft announced the Consumer Preview release of Windows 8, and the flat Metro UI has caught people’s attention.  Gone are the shading and reflections that contribute to a feeling of 3-dimensionality in many current interfaces.  The Metro UI instead embraces a ludic simplicity in the form of basic geometric shapes and solid fields of color.  While there has been a lot of positive buzz about the interface, it has its nay-sayers as well.  Some have decried the interface as a regression which sacrifices the findability of 3D icons in favor of a colorful, trendy design.  And to be honest, doesn’t it look a tiny bit like this?

Clearly, the new display has less visual information than previous versions (in the sense that more detail literally requires more bits).  Knowing this, it can be difficult to avoid the knee-jerk worry that the Windows 8 designers have thrown out something we might need (which is the same reason I never seem to be able to throw out old MIDI cables — you just never know).  Some designers have even expressed concerns that flat elements may not pop out as well as elements with more apparent depth.  But in fact, depth has relatively little to do with the psychological phenomenon of “pop-out.”  Pop-out refers to search involving those features of objects that can be apprehended pre-attentively — that is, almost instantly.  To illustrate, try finding the red T in the illustration below.

Red T among blue Ts

Not terribly difficult, is it?  And this would be true regardless of how many blue Ts appear on the screen.  A red T will always pop out from among a field of blue Ts.

Now try finding the L in the next illustration.

L among Ts

This is considerably harder (and it will continue to get harder and harder still as the number of distractor Ts increases).  The reason why the second task is more difficult than the first is that L shapes do not pop out from T shapes.

Although the word “pop-out” suggests three-dimensionality, depth is actually not a pre-requisite for the phenomenon (and indeed, in the early days of visual search studies, technological limitations would have made it difficult to study scenes with any extensive dimensionality at all).  Prototypical pop-out features tend to be simple, such as color, size, motion, and orientation.

Depth can nevertheless produce pop-out.  This was demonstrated by Dorothy Kleffner and Vilayanur Ramachandran (1992) in a study using computer-generated gradients to simulate concave and convex objects, as shown below.

As in the case of red and blue Ts, the one bump in the field of dents is pretty easy to find.  For the past several years, interface designers have been taking advantage of depth cues like these to enhance the sense of separation between elements (a matter which I’ve studied extensively) and to create easily scannable scenes.  But the failure to include obvious depth cues in much of the Metro UI does not mean that users will be slowed and stymied by unfindable objects.  There are plenty of other ways to accomplish these goals.

Windows 8 Metro UI

Examining the Metro UI again, it’s clear that it incorporates several of these.  Colors are clear and distinct, making it easy to seek out each button from the rest of the tiles.  Although shape is not a reliable pop-out feature, the icon set size of the Metro interface is small (that is, there are not many icons in each grouping), and icons are uniquely and memorably shaped, which should increase the speed of search (although without reading the label I’m not sure what the cotton boll in the lower right hand corner is intended to represent).  Slight luminance differences also help tiles stand out from one another without creating a sense of physical protrusion.  I am curious as to why the designers decided on particular tile sizes (is weather more important than Internet Explorer, for instance?), but overall the visual layout appears to satisfy the basic requirements for a comfortable user experience.  Of course, it remains to be seen whether the functionality and information architecture will satisfy users’ needs as Windows 8 is really put through its paces.

Note: This post was written prior to my employment at Microsoft.  No changes to the content have been made since that time, except to update links and post tags.


Kleffner, D. A., & Ramachandran, V. S. (1992). On the perception of shape from shading. Perception and Psychophysics, 52(1), 18-36.

Researching our communities, part 5: Field trips & recommendations

This is the final in a series of posts analyzing event attendance in the Austin design community. You can jump to any post using the links below.

Also, be sure to read the caveats for this analysis.

Today we turn our attention to a final type of event, field trips.  We then provide a set of recommendations for improving your local community, wherever you are.

Field Trips (57 respondents)

Field Trip research marketing iterationWhile it is not uncommon to hold events at company offices, it is less common to hear from a member of that company about the work that goes on in there.  That being said, there is clearly a subgroup of the community who is interested in seeing how the sausage is made.  Perhaps unsurprisingly, these people show higher than average interest in marketing and research.  Contextual inquiry, anyone?




Our data show that on the whole, there are not a lot of major differences between people who attend various types of events.  But there are enough to suggest that it may be worthwhile to tailor events for the type of attendee you hope to draw.  The Austin design community shows a need for workshops that involve critiquing and research techniques, for well-advertised happy hours, and for company events that discuss logistics.  But moreover, there is a need to continue to learn about our community, to better understand what all of us need professionally and how professional organizations can provide for these goals.

If you are a group organizer, I urge you to start documenting your event attendance and to ask your members what they want.  If you are a community member (and especially if you are not), I encourage you to reach out to your local group organizers and let them know what your needs are.  We’re only as good as our data.

Thanks for sticking with me throughout the week.  If you are interested in reading more, consider picking up a copy of Vitorio’s essay in Distance.

I’ve enjoyed working on this project and feel that it’s helped me to gain a better understanding of the Austin design community.  And I’ve gotten an inkling of how to make it a little better.  If you have any thoughts about this project, or if you’d like to do something similar in your own community, please don’t hesitate to get in touch.

Researching our communities, part 4: Presentations & panels

This is the fourth in a series of posts analyzing event attendance in the Austin design community. You can jump to any post using the links below.

Also, be sure to read the caveats for this analysis.

Yesterday we began to look at the particular interests and characteristics of professionals who prefer lunches, happy hours, and field trips.  Today we continue by investigating trends in community members who seek out various types of presentations and panels.

Peer Presentations (95 respondents)

Peer Presentation UX professional 10+ yearsPeer presentations were the most popular event overall (and anecdotally, we find that these events do tend to be well attended).  These types of events are good bets for satisfying everyone and have strong general draw, even among more senior practitioners.



Company Presentations (60 respondents)

Company Presentation lack research lack variety graphic design marketing not back-end don't keep track estimation offices learn with coworkers deliverables logisticsWhile presentations by peers were the single most popular type of event, presentations by companies showed much less interest (possibly because these events tend to come off as sales pitches).  Graphic designers appear to be more forgiving of this, while back-end developers appear more hostile.  People who like company presentations may be job seekers — they are more likely to feel that their workplaces are lacking in the areas of user research and work variety.  They want to meet in company offices and are also more interested than average in learning many new skills (such as graphic design and marketing, and, to a lesser degree, logistics, estimation, and deliverables).

They are also more likely than average to miss events because they fail to keep track of them.


Peer Panels (74 respondents)

An alternative to a presentation by one member of the community is having multiple members present or discuss a topic.  They’re slightly less popular than peer presentations, but do provide a broader perspective on the subject matter.  They also allow presenters to share the burdens of preparation.

Again, these are good general draw events, although respondents in our sample who favor peer panels do seem to have special interest in research and writing.


Company Panels (42 respondents)

Company Panel lack variety testing metrics critiquing logistics estimation deliverables it's all interestingCompany panels were tied for last place in terms of popularity.  Trends for people interested in company panels are similar to those for company presentations.  They show higher interest in learning specific skills (such as logistics, estimation, and deliverables, testing and metrics, and critiquing), and are more likely to feel that things are missing from their workplaces (namely, proper critiquing and sheer work variety).  For these attendees, the topic is less important — they’re seemingly less likely than others to miss events because they’re simply not interested.



Continue to Part 5: Field trips & recommendations

Researching our communities, part 3: Lunches, happy hours, & workshops

This is the third in a series of posts analyzing event attendance in the Austin design community. You can jump to any post using the links below.

Also, be sure to read the caveats for this analysis.

Yesterday’s post outlined the general profile of the Austin design community.  Today, we begin exploring the draw of certain types of events.  In most respects, event-favoring subgroups resemble others in the community, but some differences do emerge.  These are outlined below.

Lunches (42 respondents)

sole proprietor Thursday Wednesday public recognition independent contractor better graphics founder

Lunches seem to be especially popular among those who are not constrained to a 9-to-5 schedule, such as independent contractors, founders, and sole proprietors.  “Lunch people” may also be more likely than others to pursue professional development as part of a side business (perhaps making them less available for afternoon and evening events?).  They’re more likely than others to yearn for public recognition for their work, and for better visual design to complement their UX work.


Happy Hours (51 respondents)

Happy Hour bars full-time evening don't keep trackThe happy hour crowd appears to be a bit more junior, and seems to view events as a way to socialize and get away from work.  They have a higher-than-average preference for evenings and bars as venues, and a higher aversion to being at offices after hours.  They’re likely to miss events because they don’t keep track of them, which means you want to give plenty of reminders when organizing this type of event.



Workshops (73 respondents)

Workshop research not bars iterative development UX professionals user research critiquing end-usersPeople who are interested in workshops appear to differ most notably from other Austin design professionals on motivational factors.  They show greater interest in learning about research and critiquing and are more likely to report that their workplaces do not provide them with adequate user research, contact with end-users, and iterative development.  In a nutshell, these people are all about interacting with others, be they designers or users.

These types of events are tough to hold in bars, which may be why we see a greater interest in offices and coffee shops.

Continue to Part 4: Presentations & panels

Researching our communities, part 2: Overall community profile

This is the second in a series of posts analyzing event attendance in the Austin design community. You can jump to any post using the links below.

Also, be sure to read the caveats for this analysis.

Yesterday we introduced a project based on a survey conducted by Vitorio Miliano examining the Austin design community. Today we explore the overall make-up of survey respondents.

Overall Profile

designer founder experience designer/consultant/lead user proprietor sole

Most common words in job titles of respondents

Although we are focusing on the differences across event-going subgroups, in most respects they are similar.  They possess certain distinct skill sets, with strong representation of layout/UI and markup skills, closely followed by CSS and information architecture, wireframing, and site mapping.  Less than 30% of respondents are skilled in backend development or SEO.

They also show distinct trends in how they study their craft.  Most do so online or via books, more so than any other format.  Trial and error and conferences follow in popularity, but represent less than half of the total number of respondents.  There’s very little utilization of in-house training or online forums for learning.  In their free time, more of them pursue design as a hobby than as a side business or pro bono.

They also possess clear professional desires.  Learning the fundamentals is extremely popular across the board, significantly higher than every other topic except concepting and user research (which makes fundamentals, concepting, and research the best “general-draw” topics for events).  On the other hand, game mechanics are extremely unpopular with this community, scoring significantly lower in popularity than most other areas.

They want certain things from their workplaces, which they’re not getting.  Mentoring, contact with end-users, proper critiques, customer/market research, documented processes/best practices, and early criticism (in that order) were considered as missing from a third or more of respondents’ workplaces.  However, most respondents’ workplaces do appear to be adequately meeting their need for public and private recognition (slightly less than a fifth of respondent felt they were missing out on these).

And how do they feel about professional events overall?  The group is generally comfortable at any venue.  When they fail to attend events, generally it is due to a lack of time or because they don’t keep track of what’s going on.  But the type of event matters.  Presentations by peers are extremely popular, followed by peer panels and workshops.  The least popular events are lunches and company panels (tied), then happy hours and field trips.  At the same time, you shouldn’t automatically conclude that happy hours are a bad idea, as even the least popular events were endorsed by 41% of respondents.

This summary represents the general makeup of the Austin design community (or at least those who responded to the survey).  Over the next three days, we will be breaking down these results by subgroups in an effort to determine what kind of people are drawn to each type of event.

Continue to Part 3: Lunches, happy hours, & workshops

Researching our communities: Who’s coming to design events (and why)

Several months ago designer Vitorio Miliano asked me to do some statistical analysis for a project he was undertaking profiling the Austin design community.  Vitorio had distributed a survey across various design-related groups in Austin in one of the first community-wide attempts to see who these people are, what they want from their community, and whether those needs are being met.  It culminated in an essay about local design communities that was published in the first issue of Distance.  (We also developed a 2012 version of the survey and event response cards to help others in tracking their own local communities — if you’d like to implement these, please contact us.)

We gained a lot of insights from analyzing the survey, such as the fact that junior practitioners (the people who would seemingly get the most from local events) are more likely to miss events simply because they don’t keep track of what’s going on (whereas senior practitioners are more likely to know what’s happening, but not have time to attend).  But there were several other trends that didn’t quite fit in Vitorio’s essay, such as the statistics of event draw.  Austin’s design organizations have hosted a wide array of events, including presentations, many happy hours, tables for eight, panels, lunches, and workshops.  Each appeals to different types of people who want to accomplish different things at these meetings, and professionally.

Much of this will be most relevant to group organizers, where decisions like venue and event type can literally mean the difference between 5 attendees and 50.  But this information also has value for attendees.  If your primary interest is comparing notes with company founders, lunches may be your best bet.  But if you just want to get away from work and socialize with other professionals, skip lunch and go to a happy hour.

Over the coming week, we’ll be examining the insights we gained about various types of events.  We begin with an overview of the community at large, and then highlight unique characteristics among individuals interested in particular types of events.

Before proceeding, however, do read the caveats below.  This work is highly exploratory and should not be regarded as definitive.  But consider it a first step toward user-centered design for our professional community events.


Several caveats are in order.

1. The current analysis is largely exploratory, meaning that our goal is to extract interesting trends that we can investigate further in future studies.  For the statisticians out there, I have not controlled familywise error in these analyses (it was, however, controlled in the analyses reported in Vitorio’s essay).  This means that some of the trends reported here are likely due to chance.

2. We are primarily examining differences across groups.  We’re just looking to see if people in certain groups are more likely than average to display certain traits or interests.

3. This survey represents only the small subset of the web and design community in Austin who could be reached via the mailing lists of local organizations.  This means that people who are not on one of these mailing lists are most likely excluded from the sample.  For many group organizers, the holy grail is reaching those people who aren’t on any mailing list, and these survey results may not apply to them.

4. There may also be systematic variation within the survey respondents themselves.  For instance, members of the Austin UPA may respond differently from members of the WordPress Meetup, which might influence other observations.

5.  Self-report does not necessarily predict actual behavior.  While these results are probably reasonably good representations of people’s wishes and interests, there is no guarantee that respondents’ reports are accurate.  In 2012, we hope to record actual attendance at various types of meetings around Austin in order to better predict true behavior.

Continue to Part 2: Overall community profile