Evaluating UX: Early Prototypes Evaluation Methods [Assignment #3]

I have examined three early prototype evaluation techniques that can be potentially used in our project Usab.ly.  These are immersion, wizard of oz, and emofaces.

Emofaces technique seems to be a funny – I would even say infantile, in a good sense – approach to evaluate early prototypes by asking participant to represent his or her current emotional state when doing some kind of interaction with the prototype.  Emotional cartoon faces are used to represent the emotional state. The participant also can sort the selected emotion by intensity, which can give evaluators specific insights. The method is very simple and very fast to execute, even though the data interpretation phase afterwards may take some time. In my opinion, this option isn’t the best for our project, since evaluating emotions wouldn’t give us lots of insights with respect to what flaws we have in the system and what features can be added/removed, which should be decided at exactly this moment of product development.

Immersion technique is interesting in a sense that it gives an evaluator himself an opportunity to dive into the system, explore, and discover flaws. The evaluator, in this case, explores the system and “challenges” it, in order to find what’s good, what’s wrong, and what is useful and/or usable. This technique can be mingled together with wizard of oz. This combination of two techniques seems natural and logical to me, since wizard of oz would allow to explore the system and its features, faking back-end.

Wizard of oz, to mention, is not an evaluating technique, but rather a tool for carrying a research itself when having some kind of a prototype. The main idea behind the technique is to fake a real computation, back-end, at the other side of the screen. In a case of paper prototype, it fakes the back-end by means of a “paper computer”, a person who swaps and changes prototype elements in response to a participant’s actions accordingly. Wizard of oz still requires a bit preparation, since a whole set of affordable features should have a UI and appropriate “hardcoded” information when a participant explores the system. The method seems to be a very realistic approach to finding flaws in the system, by deriving all sorts of feedback and insights from the participant. I believe this is the method we should go for, since carrying a wizard of oz technique will generate us a bunch of insights from participants. For example, we will answer very important questions, such as what is truly important in the system, and what is not.

Evaluating UX: Assignment 2 [Comparing Three Methods]

I got acquainted with three more methods for evaluating user experience, which are useful (and usable) during early stages of a product prototyping: contextual laddering, multiple sorting, and object based techniques. I like the way how all these methods elicit information from a participant in their own unique way.

Contextual laddering is an appropriate method to choose if designers do not know yet true reasons why users will use a product, and what true value they see behind it, or more importantly, some values they not even think of, but is more closer to subconscious level of understanding. Asking “why”, then “why”, then “why” again and again can be truly powerful, but it requires good interviewer’s skills of guiding the conversation into the right direction.

Multiple sorting seems to be quite simple method, but results promise to be insightful. The method fits very well to a “lean” approach of developing a product, since the method enables designers to see what features are truly important to users, and what’s not important at all.

Object based user evaluation could fit our context very well, I think. Since our product, Usably, is in very early stages of development and the idea is still prone to changes, generative technique of the method is a very powerful tool to elicit creative insights from participants. Since we still have very vague concept of how data (e.g. charts, graphs, usabiity ratings etc.) is represented in our service, collages would serve a good platform to enable participants to modify layout, choose UI elements, and articulate what interactions they want to be present in the service. The paper is a very productive generative platform, since a subject can provide his own creative ideas into the collage.

Evaluating the UX: Assignment 1 – Evaluating Design Concepts

After reading three methods for evaluating design concepts, I saw how tricky it might be to evaluate qualitative characteristics when having no prototype, or very lo-fi product.

The AXE method deals very well with identifying interviewee’s perception of design concepts by presenting contrast pictures. I like this method because it not only reveals subjective perceptions about concepts, but also elicits suggestions on how to improve a particular concept. Other method, exploration method, is good to learn from competitors’ mistakes and identify usability problems of their products, thus improving usability in your own product. Exploration method is good when having no prototype of the product. Additionally a facilitator can identify what features/characteristics are truly important for the client and if current solutions have these essential features. The last concept, sentence completion, seems to be quite effective to elicit what the product – concept – personally means to the user and how does the user reflect the product on himself.

For our current product, I would choose exploration method since it enables us to see what current solutions provide, what are their pitfalls, and how can we differentiate our product from other products, to truly meet users’ needs.

Foundations of HCI: [M7] Conclusions

An ultimate question posed is whether we are supposed to employ HCI theories, and, if yes, what do we use it for? At first, I do agree with the author of the book that “theory can be very powerful in advancing knowledge in a field”. I see that a nature of theory development in HCI field changes in rather better direction: from lots of criticism of previous methods to re-defining, re-thinking and synthesizing former theories with new frameworks and contemporary models devised by researchers. Tolerant attitude towards other theories and a careful observation of pros/cons of the ones led to a more open discussion in the field. Transdisciplinarity, I think, is the result of an open-mined attitude towards imported theories and closely related fields. Another possible consequence, to my mind, will be a totally new perspective on classical and modern theories, and moreover employing and re-contextualizing these “old” theories to create a synergy with the new ones.

In the past, theories were used as models and tools for analysis and design guidance. Now (and in the future) theories shift towards creating new ways of experiencing, analyzing design/cultural values, and providing new insights. But, it should be mentioned that it’s foolish to think that a theory, no matter how good it is, “do design”, as author notices.

Here is the list of 30 HCI concepts perceived as important by me:

1. cognitive modelling
2. ethnography
3. cognitive system
4. user performance
5. computer-augmented behavior
6. data analysis
7. interaction design
8. human values
9. artifact design
10. ludic & playful design
11. information overload
12. usability
13. “in the wild” research
14. work design principles
15. social and cultural norms
16. user experience
17. system efficacy
18. information propagation
19. learning process
20. aesthetics
21. pervasive technology
22. augmented-reality technology
23. responsibility
24. design for lifestyles
25. ecological rationality
26. social interaction
27. user-centered design
28. tangible user interface
29. sustainability
30. behavioral change

Foundations of HCI: [M6] Contemporary theories (part II)

Details of the assignment can be found here.

In this assignment I extended my previous concept map on contemporary theories in HCI by adding turn to the wild and turn to embodiment parts. This time I tried to analyze book contents in my head, make some notes, and then start sketching the concept map: to avoid the linear structure of the book in my map and put my own understanding of concept linkage of the turns. This time I avoided “to cram all details of the chapter” (as David noticed last time in the class =) ), but I rather tried to concisely depict most important concepts.

To be short, all these turns touch, more or less, human values. In previous periods of HCI history, researchers were preoccupied with efficiency, performance, usability. In contemporary theories, new dimension of a variety of human values takes place; philosophical questions are raised, sense-making aspects are considered, new forms of interaction design appear through the synergy of social interactions & physically shared spaces…

P. S. Seems like a last concept map…hehe =) I enjoyed sometimes making them (cmaps). I think it’s a good way to organize all those concepts in your mind and create strong connections between them.

Foundations of HCI: [M5] Group Assignment: Critical Design Brief

We chose the web site about World Cleanup that offers everyone to join the mass movement to clean up the whole world. Besides the information about the campaign, they also have motivation video, steps to clean up the world and plans to make every country cleaner. But the main purpose of this service is not just to motivate everyone to clean but also to make people think about the problem of littering. Instead of cleaning all the time, people could produce less trash or, at least, do not litter on the streets. “It’s hard to clean, but so easy to mess up”, say people from World Cleanup.

We target the audience of all people 12+, due to the sharp content that could be psychologically harmful for younger children.

We came up with the critical design solution, which would make society think about littering in a provoking way. Inspired by a video, which shows dead birds, whose dead bodies are filled with trash, at a Midway Island, 2000 miles from the continent from where all the trash comes from, our design solution is to create provoking pictures of dead bodies of birds, fish, and other animals who are filled with trash and include a provoking message (e.g. “Look closely. Maybe you can find your trash here?“, “It’s not a photoshop; rather DeadBirdsWithTrashShop” ) below the image. Here are few examples that illustrate the design:

Posters with these pictures can be put in public places, such as bus stations, metro etc. Additionaly, this message can be placed on t-shirts, cups, and other widely used products. We believe that this message is able to trigger people to think more about littering problem. The purpose is that every human will reflect upon himself/herself.

Is it our responsibility to stop creating more trash and take care of it? For sure. Being part of this environment, we should clean and save it. Midway Island is a place where a lot of birds are dying not because of food but by eating poisonous human trash which contains plastic, metal etc etc; it’s not all about Midway Island – it’s happening around us.

Our critical design artifact might serve as one way to create a social impact by challenging the status quo of littering problem.

Authors: Vita Krainik, Khwaja Umair, Vitalii Lakusta

Foundations of HCI: [M5] Contemporary theories (part I)

Details of the assignment can be found here.

Reflection on the concept map

Contemporary theory in HCI has changed even more drastically than former modern theories drifted from the classical ones. Now human values were considered more important, and stood above the efficiency of the system as considered in modern theories. Ethical, social, philosophical, and other human-related questions began to arise in a mindset of designers and researchers in HCI field. On the other hand, HCI community bewares of the totally new humanitarian approach, wanting it to be only interpretative, not prescriptive. In any case, this new even more radical approach enabled new forms of interaction design, which is concerned about user experience and other sensual experiences in the first place.

The concept map reflects turn to the social and culture in contemporary theory in HCI. Here I depicted main concepts that reflected those “turns”. After the challenging concept map from module 4, this one seemed to be easier to create) Meanwhile, I kept in mind David’s notice about creating network, not hierarchy, and making more connections between the concepts.

---

Reflection on the case study papers

1) Developing the Drift Table, by Andrew Boucher and William Gaver

This paper shows the process of having an idea of the drift table to developing a working prototype complete product. The paper shows us what challenges authors faced and what key decisions were chosen, which, in my opinion, reflect the nature of the turn to design, taking place in HCI. First of all, what clearly delineates this article as being separate from modern HCI is the very nature of the Drift Table – the artifact that brings in ludic experience at home, NOT designed for having any specific and robust task-oriented features.

The other thing worth mentioning is the critical engineering decision of pursuing a self-contained approach (far more difficult), which means that the table would handle fetching and processing data by itself, and not by simply streaming data into a display from an external server. The interaction designers pursued the first choice to keep with the idea that the Drift Table somehow contains the world on view. It clearly signifies that user experience prevailed, and efficiency lost the race (it would be more efficient to pursue a data streaming approach rather than a self-contained device, posing many challenges and headache to engineers 🙂 ).

To conclude, it becomes evident that the article fully represents the turn to design, dealing with the interaction design from the user’s perspective, taking into account ludic and playful design (representing novel design values), user experience, aesthetics and other human values. This case study is a good reflection of the theoretical landscape of the turn to design in contemporary theory in HCI.

2) Reflections on Norm-Critical Design Efforts in Online Youth Counselling, by Sofia Lundmark & Maria Normark

The article critically evaluates e-healthcare services in relation to social norms and values and, on the example of the Swedish National Youth Counselling site, illustrates how these norms and values co-relate to the interactions design process. The authors investigate a specific example of the umo.se site (Youth Counselling site mentioned before) project, called The Love Animation, to show some of the issues designers consider and face with while designing a simple to the first glance, but inherently complex when social norms and values are considered, animation.

After reading the article, I pinpointed the most important insights on cultural aspects and interaction design process I would like to share now as a bullet list:

• social norms are embedded in interaction design
• interaction design constructs meaning, norms, and values in design
• interactions design might unintentionally spoil the intended message (as in case with The Love Animation, showing different feelings that might happen when the one falls into love, the first design included a navigation in form of a timeline, that could suggest that these particulars feelings should happen linearly, which is not the case. It could confuse the user and spoil the intended message)
• norm-critical approach in the design enables designers to embed social norms into the interaction design
• design tends to make the norms invisible / implicit
• norms may be embedded in artifacts
• norm-critical perspective questions, challenges, transforms, and creates new norms and standards
• interaction design elements might be experienced as normative structures
• navigation & structure might be viewed as carrier of norms constructed in user and design contexts
• umo.se representations are reflections on existing values and norms
• design might reify and/or reinforce behavioral norms or challenge them
• norms are in the interface

I would say that this case study clearly pinpoints the turn to culture in contemporary HCI. In contrary to the turn to design, where aesthetics, ludic and playful experience etc. is taken into an account, in this case, the most essential factors are cultural norms and values, identities, human disabilities, gender/culture-based issues, considering majorities versus minorities and finding compromises when designing for both of these groups. In this case study, I clearly see the mapping in the theoretical landscape, especially with relation to critical theory, which challenges and critiques interaction design, as it can be seen in this study case.

Foundations of HCI: [M4] Modern theories (part II)

My concept map (which is bigger than I expected) embraces the essential elements of modern theories in HCI, depicting the essence of every theory, and binding them, if needed, with concepts from other theories to create a network which shows the linkage between them. After reading David’s remark on module 3 concept maps, I tried my best (and I hope I succeeded) in creating a network, not a mind map, depicting meaningful links between concepts. It was vigorous and fun and sometimes nervous experience to create this map.

If we draw a line on all of the examined theories, long story short, I can say that all of them “turned to the social”, in contrast to the classical theories, which examined the individual and his/her cognitive characteristics, without taking context and social factors into account. The turn to social is also vivid by the fact, as seen on the concept map, that almost every theory performed ethnographic field study to collect data that embraces different factors in a workplace setting etc.

Foundations of HCI: [M3] Modern theories (distributed cognition)

The process of creating a concept map included reading the chapters on modern theories of HCI. Then I re-read the Distributed Cognition chapter, making notes in parallel and grasping better understanding. Afterwards, and finally, I sketched the concept map, glancing to the notes and the book time to time, which represents my view and understanding of an overall picture of the theory. The map really helped me to see the overall structure of the DC theory and solidify its concepts in my mind.

The concept map itself represents the Distributed Cognition theory, what it described, examined, highlighted and stressed on; also its drawbacks & limitations, how it evolved (DiCoT), and what the theory enabled (better design solutions).

Distributed Cognition describes a cognitive system, and more specifically interconnections between interactions among people, using a variety of technological mediating artifacts. The analysis, through its descriptive nature, points out and enables to identify strengths & limitations of current artifact designs and provides a basis to redesign a work setting & work practices in the cognitive system.

Foundations of HCI: [M2] Classical theories (cognitive modeling)

Here I present the concept map, depicting the main classical theories of HCI and approaches that correspond to each theory (e.g. cognitive modeling -> GOMS).

First I read the first four chapters of the book, then I re-read the 4th chapter on the classical theories, creating in parallel first scratches of my concept map on the paper. Then I adjusted and transferred my scratches to the CmapTools. I tried to create an overview of all phases of classical theory development and their corresponding concepts and/or theories. Also I highlighted  some advantages/disadvantages and main features of each of phases.