Optimism and critique

August 24, 2008 — 13 Comments

When I was at art school the critique was central to the ethos. Every couple of weeks I would have to pin my work up on the wall and defend it from the criticism of fellow students and professors. It was always stressful but as long as I hadn’t made the mistake of staying up all the previous night trying to get the work finished I normally performed OK. The critique is a great way to give someone feedback that they can use to improve the work. People who were good at giving criticism did not let personal feelings get in the way of being honest and inevitably the sessions could become quite tough. It was not unknown for students to fall out with each other over a crit.

Today we are more interested in collaborative teams than we are in the individual. Design challenges are too complex for an individual to tackle alone. We have found out that for teams to work well together there needs to be an essentially optimistic environment. Back in May I talked at the Art Center College conference Serious Play about how designers use aspects of play in their work and one idea that I focused on was permission to take risks. I suggested that the reason many creative companies have playful environments is to encourage risk taking and to help create a culture of optimism.

The dilemma we face today is that in a culture of optimism good honest criticism seems to be dying out. Is it because we are nervous about upsetting each other?

Whatever the reason, I believe we need to find effective methods for criticism within todays optimistic, collaborative design culture. Does anyone have any examples of how this is already being done?

Tim Brown

Posts

13 responses to Optimism and critique

  1. Tim –

    I have enjoyed reading your blog. I just read Randy Pausch’s “The Last Lecture” and he has a chapter on the power of feedback. Here is a quote from his book:

    “Get a feedback loop and listen to it. Your feedback loop can be this dorky spreadsheet thing I did, or it can just be one great man who tells you what you need to hear. The hard part is the listening to it.”

    -Brian

  2. As a follow up to this post, I have just finished reading the article by Pixar founder Ed Catmull in this month’s HBR. He talks about the brain trust of directors that they use for peer review. The article is worth a read.

  3. I’ve had success with the technique described below. I developed it from a combination of methods I’ve both tried and read about.

    Envision a group of designers/innovators sitting around a table (preferably a round one). Everyone is there to present their ideas/designs to the group; even if the ideas have been created collaboratively, one individual takes responsibility for presenting each idea. Before starting, each person has in hand two cards: one says something akin to “Innovation of the Year”; the other says something like “Ho Hum”. As the first person presents, the rest of the group is in listening absorption mode; questions that have to do with fully understanding the concept are encouraged, but no judgmental feedback is permitted. At the end of each mini presentation, the group counts down from three and each participant flips over one of the two cards they’ve been handed. If every card turns up positive, the idea has legs. If a bunch of “ho hum”s come up, the idea is most likely lacking in something. If the designer is still passionate about the idea, he or she is welcome to defend it. At this point, I notice that much healthier conversations ensue; they tend to focus most on the content of the idea. It’s key that everyone flips their cards at the same time to avoid any group think. It also prevents the “boss” in the room from voicing his or her opinions first and unduly influencing the rest of the team. I’ve found that it’s a great mini-version of using the wisdom of the crowds.

  4. Sorry for the late response here TIm – I have only just discovered your blog.

    One of design’s strongest tools is the ‘multiple concepts’ tool (sorry, crappy name for a great thing).

    When we present multiple ideas, we let both others and ourselves be more critical of the ideas before us. It separates ‘the concept’ from ‘me’, in a way that means feedback is not an attack on us, because we have already shown that this is not my one and only and best idea. It is just an idea I came up with.

    I have seen the ‘multiple concepts’ tool used by generals and engineers, but it seems to be designers that get the most from it, because design feels so expressive and linked to our own person.

  5. For all of the (justifiable) discussion of human-centered design, I think the challenge outlined in this post requires a version of human-centered management and organization.

    Below is a brief example that illustrates this idea from my time as a manager at a large Web content site.

    Simply put, we faced a significant challenge: Our content areas contained too many on-screen errors and sub-par experiences. These experiences were frustrating to users and demoralizing to the staff. To make things better, we had to grapple with criticism. So I led a team of people to craft and implement a strategy to deal with the issue. Here’s how it worked:

    Step 1: Assemble a bulletproof set of on-screen examples that show the breadth of the problem to gain critical agreement from management that this is an urgent problem.

    Step 2: Assemble team leaders from across the site to review a broader set of examples to air out the issues. Every section of the site is featured, no one is picked on unfairly, everyone has a chance to discuss what they see. This establishes a culture of inclusiveness, transparency and honesty.

    Step 3: Begin daily dialogue and weekly meetings to formulate standards aimed at improving our on-screen experiences. Everyone is invited to offer *anything* relevant: examples, ideas for standards, etc. Though at times uncomfortable, these discussions are productive because people from all parts of the site are represented (no secrecy/exclusion), concrete examples are offered for all to discuss (no unnecessary subjectivity/abstraction) and everyone buys into the goal (getting better).

    Step 4: Ultimately, involve all staff members in these discussions so that every single person throughout the organization can play as big a role as anyone else.

    In the end, this six-month process created cleaner, better content and a culture focused on quality. Through it all, there was some yelling, some disagreement, etc. And we didn’t accomplish everything we set out to accomplish. But we made great strides in being able to exchange criticism in the pursuit of a better product and our audience and our organization benefited.

    How is this human-centered? To me, this process underscores the importance and effectiveness of simply treating people as human beings. Be candid and honest. Be transparent. Be respectful but not indulgent of poor performance. Be humble. Be inclusive. Be optimistic.

    Thank you for allowing me to share this experience. I look forward to reading more from your blog.

  6. In my portfolio class at Auburn University, Randy Bartlett gave a fellow classmate $20 and told him to go buy some design magazines to research graphic layouts for his portfolio. While I’m not suggesting that professors and design managers start handing out cash, I did find his approach refreshing. He didn’t tell the student what he needed to do to make his work better. Instead, he equipped the student with the tools to figure out on his own how to succeed.

    I have used a similar approach with my design teams in the past. When someone’s work was not up to par I would require them to get hands-on with the problem. I was routinely surprised by the quality of the solutions that came out of this interaction.

  7. Tim,

    In school, I always found that the critiques that were the most useful were given in situations where both people were listening in a setting where people were purposely looking for feedback.

    The best example may have been my desk critiques during my studio days. As students, we were looking for feedback/criticism because the time was designated for such an activity. The critics expectations were established by the professor, so they new they would be coming in to give critical guidance on a work in progress. These two factors, combined with the more informal setting of desks and sketches, seemed to make the dialogue a lot more fluid than pin-ups.

  8. I think the lack of critique is a serious problem we are facing across the societal spectrum. I like the personal responsibility assignment of a team member to a group idea in Brendon’s method above but the process is akin to many “vote” or blind methods of giving small bits of feedback (which are pragmatically necessary) but I think miss the real purpose and value of critique.

    Fundamentally we need to establish a culture of trust that will enable the critique to be grounded in honest feedback. One way to approach this is the use a framework for the critique. Introduce this framework to the team and manage the process. It may feel forced initially but using the framework will baseline everyone and allow for the critique to actually reach the delivery of a judgment statement.

    The framework (an amalgam of numerous un-cited sources) given to the owner of the idea/designer/artist from the person critiquing:
    1) Description – feedback your interpretation of the subject. This lets the designer know what is being heard and diffuses the ladder of inference.
    2) Analysis – focus on the formal aspects of the subject as they relate the goals
    3) Interpretation – express your opinions, use analogy, be personal in the interpretation
    4) Judgment – is the work successful or not and tell why

    Steps 1and 2 are objective assessments that enable 3 & 4 to be expressed without getting personal. Taken alone the value of the critique is diminished (or missed altogether) but as is so often the case this is exactly what we choose to do. Either people attack with the “I like it” or “I don’t like it” with no basis or they safely assess formal aspects without delivering their opinion. Ultimately our fear and lack of trust inhibit us from sharing and allow us to settle with mediocrity.

    Using this framework together with assigned ownership of collaborative elements may be the way to pull the team into a more productive critique session. This of requires outside input to be truly effective.

    Real, honest, difficult critique is a gift. More people need to learn this.

  9. I think a major way to avoid the personalizing of the critique – which may only be doable in interaction design – is to critique teams, not individuals. Especially when the teams are voluntary, and people are working with the people they want to work with. If the criticism is valid, it will help the individual members of the team also critique the team, and in turn their collaboration methods, which will help the team produce better results.

    Also, if you get the students to focus on being members of the same school, and wanting the school to be good – for their future professional reputation – then the critique takes on a new form as well. We do this a lot in the HCI/d program at Indiana University.

  10. One of the best things I took away from the Toy Design program at the Fashion Institute of Technology was how to take criticism and rejection. On the first day, my professor explained how toys are more fashionable than fashion itself. The nature of the toy industry means a toy concept will always have so much more going against it than it will ever have going for it. Take a step back, you can’t take rejection personally. Focus on the few positives and disregard the rejections.

    With that in mind, I rely on three methods of critique to develop viable concepts. The key is to know when to use each type of method.

    Brendan Boyle’s, ‘Wisdom of the Crowd’ technique works really well at an early blue sky stage. This technique is useful when you need to efficiently weed 40-50 ideas down to a handful. Look for other methods when you need more intensive critique. Brendan’s technique enables the team to let their imaginations run wild and still have a safe place to share their craziest ideas. This brings a great opportunity to take risks, keep in mind that your wild but rejected ideas might just spark a viable idea in another team member. This technique can make a stressful process a lot more fun and encourages the group to focus on the positives.

    Once you start investing time and resources into a concept, know when to solicit feedback and do it often. It works best by having a readily available group of people who are passionate about the same goal. Try to keep it as organic as possible; call a few people over on the spot and ask them what they think. If you have the time, repeat with a new set of people and compare. People are more likely to give you constructive criticism if they see you haven’t invested too much into any given decision and know you really want their help. Make people feel empowered to share their opinion and feel like an important part of the decision. Think of every task as a team effort. This way, the project will be viewed as “ours”, not “mine”. Keep an open mind, you might be too personally invested in the concept to see a major weakness. Fresh eyes will often see something huge that was overlooked.

    When not in a familiar and comfortable team setting, I find Andrew Fallshaw’s, ‘multiple concepts tool’ works really well. For example, when user testing I find it best to share 3-5 concepts even if I really just need criticism on one. People generally want to be optimistic and don’t want to put your work down. You are more likely to solicit honest feedback when they can pick and choose which concepts work, and explain why the others are just ‘not as good’. Give them a chance to come off as positive overall so they don’t feel guilty for disclosing some negative aspects.

  11. In response to your question about examples of criticism, and with all the usual risks of self-pimping, I have written extensively on this subject on my blog, Interaction Culture. Many posts explore the relationship between critique and interaction, including related notions of epistemology and judgment. I also wrote a lengthy series of posts on how to practice interaction criticism. Perhaps you or some of your readers will find it interesting:

    http://interactionculture.wordpress.com/2008/06/04/interaction-criticism-how-to-do-it-part-1/

  12. I’ve found that people seem to either be too overly enthusiastic about critique, or very hesitant to experience it.

    This is probably due to the fact that people seem to feel that it’s open-season when they have the opportunity to critique someone–because it’s always easy to find something wrong with an idea/creation/design/etc. As a result, I’ve noticed that many creative people are hesitant to share their ideas and invite critique–because they know the result will often be harsh. This is also what has probably given rise to the culture of optimism you mentioned, with creatives creating a positive environment for ideas so they can avoid being slammed by negativity.

    But, as you mentioned, that also isn’t a perfect environment because ideas don’t receive the honest critique they need.

    What I’ve found works best is also what Jason mentioned above: creating a framework for critiquing ideas. Even doing something so simple as telling respondents that they need to first think of 5 positive things about an idea and 5 possible challenges with an idea can make a huge difference. (It’s important to do positive things first, and every negative thing has to be countered with a positive.)

    The act of asking reviewers for positive aspects first does two things: (1) it makes the creative person happy because their thinking is appreciated, and (2) it signals to the reviewers that they can’t just lay into the idea, they actually have to consider it. Then, when the reviewers get to bring up the challenges of that idea (e.g. not legal, expensive, etc), the creative is more open to hearing the challenges and thinking about how the idea can be improved. Plus, the reviewers see the positive aspects of the idea, and are thus more willing to help figure out ways to fix/surmount the challenges.

    Finally, the benefit of laying the framework of critique out in advance is that everyone knows they will get a chance to share their evaluation in the appropriate time, and so can give all their attention to finding both the positive aspects of the idea, and the challenges.

  13. As an educator I struggled with the critique issue often. Some students were too soft—either they couldn’t take it or they were afraid to say what they really thought. Some critiques felt like dragging a dead horse, with massive gaps of silence between comments. Students in their final years of the program did much better than the newbies, but this feeling that critiques were a necessary evil still bothered me.

    In a classroom of 20+ I encountered a few problems:
    How do you display the work so that everyone can see it up close, enough to discover detail (or lack thereof)?
    How do you encourage ample amounts of feedback without making the critique go on for hours?
    How do you help students be succinct when describing their work and the problems they were solving? (some students felt that if they had spent all week on the project, their reward was to take as much time as possible out of class to talk about it, and when I cut them off resented it)
    How do you get students to articulate concepts and goals and criticism?

    So I made everyone get a Flickr account. Back in the days when my students had never heard of it, or blogging or Facebook. We met in a computer lab, and students had 20 minutes or so to leave a comment on everyone’s images in our Flickr pool. I would highlight areas on the image for the discussion to help teach a concept, and the owners of the work would use the caption to describe their objectives and request specific feedback.

    After a few rounds of this, the feedback and work description really started to improve. Students knew if they said “That’s cool!” with nothing tangible to explain why, their feedback was useless. They also knew if they didn’t quickly explain what their aim was, that the feedback might be ineffective. In addition students made sure that their work was handed in on time. Otherwise they got left behind very quickly.

    But the best thing about the Flickr crit was that there was a certain anonymity, that let students really say what they felt, and students wanted that honest criticism. Students received 20+ comments on their work which would have never been possible in a regular pin-up critique. And it was all written down so they could refer back to it after the critique. Some students even put up work between crits and started their own discussion groups outside of class. This was remarkable for 1st year students.

    So, I used social technology. Of course it required 20+ computers but don’t we all have enough old macs laying around?

    p.s. I’m using your Design Thinking article in my Design PhD dissertation. Thanks for putting into words what’s been floating in my head for awhile now. Enjoying the blog!

Leave a Reply

*

* Copy This Password *

* Type Or Paste Password Here *

Text formatting is available via select HTML. <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>