A Generative-Evaluative Design Meeting Style

What is the right way to run a brainstorming or design ideation meeting? What comes naturally to most people is to sit in a circle and freely shout out ideas until one gets the approval of a few other members, with critiques of each interleaved with their generation. But there are better ways motivated by a desire to solicit input from all members of the team, promote information sharing, and deliberately form and consider ideas against agreed-upon criteria. Here we present a “Generative-Evaluative Design Meeting Style” with justification for each step of the process. Effective meetings don’t happen by chance, and by intentionally structuring them and reflecting on what works, we can achieve better results.

“A Generative-Evaluative Design Meeting Style”

1. Prior to the brainstorming meeting, the group should have a high-level discussion of what the customer or end user values.  This could include things like safety, frugality, convenience, simplicity in user experience, similarity to existing tools or designs, etc.  The purpose at this time is not necessarily to have definite agreement on each of these values, but to prompt everyone to get into the generative mode of thinking by adopting the perspective of the end user.  
2. Ideally before the design meeting, present the question or design challenge to the group so each member can begin thinking of ideas and doing research.  If this is not possible, start the meeting with 10-15 minutes of “quiet time” in which everyone writes down his or her ideas on paper or on sticky notes independently.
3. After that time, everyone selects any number of ideas to share with the group from those written down.  Everyone selects at least one to share.
4. One person is chosen to be the facilitator of the discussion.  This person should ideally be the most junior member of the group, or the role may rotate among members.
5. Going in a circle, each member introduces one idea in a 1-5 sentence explanation.  There is no debate or evaluation of the proposal at this time during the sharing process.  Only the most basic clarifying questions may be asked.  The facilitator writes each idea on the whiteboard with a title and a short summary.  The fact that the idea is written on the whiteboard by the facilitator helps to build distance between the proposer and the idea itself.
   1. It is common and expected that some ideas will overlap in this process, which is no problem. The presenter of the idea that overlaps may decide if it is worth listing separately, as a variant to an existing idea, or not at all.
   2. Everyone goes in a circle (again with each person contributing at least one idea) until everyone has exhausted ideas to share.
6. After all ideas have been shared, the group uses “approval voting” not for ideas to pursue towards design, but for which should be discussed further.  All members vote by raising a hand for each idea they consider to be worthy of discussion, and each person can vote for any number of ideas.  Members should vote for an idea if it seems viable or if it would enrich the group’s discussion.  The number of votes is recorded for each idea.
7. The ideas then are put up for further explanation in the order of the number of votes they have received.   It is not necessary to discuss every idea.  The group may use its judgment for which ideas should be put  up for longer discussion. The presenter of the idea gives more explanation as desired from the group, and answers questions, gives clarification, etc.  Still, at this time, the focus is on exposition of each idea and not evaluation. During this step, others may come up with more ideas, which is to be encouraged, and may add them to the queue to be discussed.
8. Once everyone has had sufficient opportunity to share ideas, the group creates a decision matrix of the ideas that have generated the most interest and seem most plausible with the values of the customer or the design as the criteria for evaluation.  This is ideally done on the whiteboard. At this point, discussion is opened up for full critique of ideas against the evaluation criteria.  More criteria may be added, and the group may collectively try to refine the means of evaluation, e.g., how to judge the simplicity of a user interface.
9. The group does not need to make a decision on the design choice during this meeting.  The appropriate outcome of a brainstorming meeting in this style is a list of action items, such as the creation of experimental plans to compare designs, calculating the cost of each idea, mathematical analysis of system dynamics under the different designs, etc. that would inform the decision process for the top proposals.  Or, the group may decide that none of the ideas would satisfy the needs of the problem, adjourn to conduct further research, and repeat this brainstorming process.

This meeting structure may seem strange or unfamiliar, but has a variety of benefits over a “free style” or unstructured brainstorming meeting.

1. Separating the generative and evaluative phases:
   1. This is an important psychological benefit.  Without wading into questions of why this phenomenon exists, there seems to be two different intellectual modes – one generative, creative, imaginative and another analytical-critical.  Everyone should ideally stay in the “flow state” of idea generation and feel free to propose things that might sound crazy, because those ideas may contain within them pointers to aspects of the problem or user needs that may otherwise be missed.  
   2. Brainstorming meetings often are impeded by either idea introduction and immediate critique, which can make some too hesitant to share, or immediate sharing of whatever pops into someone’s head, which can waste the group’s time.  By requiring everyone to write down and then intentionally share without immediate critique, this process moderates between those who are too eager and those who are too hesitant.
2. Soliciting ideas from every member:
   1. Everyone is required to contribute at least one idea.  This helps to get a wide variety of experience (personal and technical) incorporated into the set of ideas evaluated.  Even if an idea is not selected, it may introduce a new technology, process, or design pattern to the group.
   2. This shared activity in which everyone participates helps to build team cohesion.
   3. Everyone practices design and ideation skills.
      1. Not only the most senior members should do the design.  Team design work should be collaborative.
      2. One of the junior members of the team facilitates the discussion so that he or she remains engaged.  The natural tendency is for the most senior members to have strong ideas from the beginning.  Having the junior members facilitate the meeting improves the vertical transmission of design knowledge by ensuring that the least experienced members can understand what is being proposed.  This also helps make ideas explicit and forces clear communication.  Design meetings are some of the most valuable opportunities for knowledge transmission in terms of time-density.
      3. By requiring everyone to meditate on the problem separately before sharing with the group, no one can be complacent with letting others do the design.  Then everyone can compare their own set of ideas generated with others’, which is valuable for both vertical and horizontal knowledge transmission.
   4. The requirement to write down one’s ideas with a formal process for introducing them to the group also restrains those who might take up too much of the discussion time and gives a structure to those who might be reticent or junior within the group.
3. Approval voting of ideas for further discussion allows everyone to express their high-level evaluation extremely efficiently on every proposal.
   1. Voting for just one may obscure that some believe a 2nd best is still worthy of discussion, and each person coming up with a ranking of all ideas would take too long.
   2. It also allows the group to not waste discussion time on negative evaluation of ideas that no one wants to pursue.
4. Separation of the idea from its creator by having the facilitator write each on the whiteboard with a title that describes the proposal instead of referring to the proposer.
   1. This helps everyone discuss the ideas dispassionately and without ego.
   2. Every idea is put in the same decision matrix, and then everyone looks at the decision matrix instead of the proposer.  This is another important psychological benefit, because idea evaluation is then framed in terms of what the customer needs instead of who is proposing it or a conflict between proposers.
5. Once the idea is on the whiteboard, everyone can understand what has been proposed.  A common meeting pitfall is to verbally describe ideas without recording them, and then go in circles re-explaining what has already been introduced or debated.
   1. In a group of 4-8 people, it is likely that some will lose focus, and it is better that they can refer to the whiteboard rather than ask the basic questions again.
   2. The visual action (writing on the whiteboard) helps to focus attention when paired with the verbal description.
6. Centering the critical discussion on the design criteria that were mostly decided before the meeting.
   1. This is “beginning with the end in mind”.  Everyone knows at the start of the meeting that the goal is to eventually evaluate the ideas based on these criteria.  The goal of the meeting is to decide on how to refine the evaluation of the proposed ideas in terms of experiments, research, and analysis.
   2. There needs to be serious consideration of the user or customer needs before brainstorming begins.  This helps to focus on the ideation itself during this meeting.
   3. It is expected that there might be some user values that were not explicitly acknowledged at the start, but only arise when comparing proposals.  This is a good result to have because the team improves its understanding of the customers by imagining what each proposal would do for them.  The team also moves from private judgments of the user needs to common explicit understandings by requiring framing the evaluation in terms of definite criteria.

In summary, this is not a definitive guide, but a set of suggestions for a better design meeting structure. Other structures are possible. By making this one explicit, new designs can build off of it.