Open card sorting: the protocol
By UXbeam · Information architecture tools since 2021 · Published September 21, 2026
In an open sort, participants get the cards and nothing else. Everything that makes the result trustworthy happens before the first participant arrives: the scope, the cards, the wording, and the pilot. This page is that procedure, ending in a handoff to analysis. The guide covers what an open sort is and how it compares with closed and hybrid.
When an open sort is the right sort
Run one when the sections are genuinely undecided, when you want the organizing rule people reach for rather than the one you assumed, or when you need vocabulary: the names people give to groups of your content. If the sections are fixed and the question is placement, run a closed sort. If most of the structure is settled and you suspect a gap, run a hybrid sort. If the question is which of two organizing principles your audience applies, read the organizing principles guide before writing a card.
Define the scope
One question per study
"How would customers organize the account and billing area" is a study. "How would customers organize the product" is three. The card set is the scope; if it needs more than one sentence to describe, split it.
Decide what a card is
Features, content types, or tasks, not a mix. A set that mixes Invoices (a thing) with Download an invoice (a task) invites two sorts at once.
Leave out what nobody navigates to
Legal pages, error states and settings reached only from inside a feature dilute the sort without informing the structure.
Write the expected answer down, sealed
Write down your expected grouping before the study so you can compare it with what participants produce.
Prepare an unbiased card set
- One concept per card, at one level of grain. A card for Payments next to a card for Change card expiry date tells participants which is the parent.
- No category words in card names. Billing settings answers the question. Payment method asks it.
- User language. The words on the cards are the words participants will reach for in their labels, so internal names leak into the result.
- No ordering cues. Randomize card order per participant. Alphabetical order groups Charging schedule with Charging status before anyone sorts.
- Near-duplicates only on purpose. Keep Start engine and Remote engine start both only if you want to know whether people merge them.
- Descriptions only where the pilot shows a card is unclear. A description after
||steers a sort; every one you add is a decision you made for the participant. - 15 to 60 cards. UXbeam's limits. Sample a large inventory or split the study rather than pushing the top of the range.
Ready-made sets, with recruiting messages, are on the templates page.
Neutral instructions
Inside the sort, UXbeam shows one line: Sort the items into as many categories as needed. The message that carries the link is yours. It should say what the cards are and how long it takes, and nothing about how many groups you expect, what your current sections are called, or what you hope to find. Do not use navigation, menu, validate or our new structure; each one tells participants what kind of answer you want.
Pilot
Run the sort with a few people you can ask questions afterwards before you recruit. Not to collect data, but to find the cards that need fixing.
- Which cards did they ask about, or leave until last? Those need rewording or a description.
- Which cards did everyone leave unsorted? Decide now whether to require all cards to be sorted. Requiring it gives complete data; allowing unsorted cards gives a count of what people could not place.
- Could you read their group names? If pilot labels are single letters or "misc", the message needs to say that names matter.
- How long did it take? Put the real number in the recruiting message.
Handling unclear cards
Four options, in order of preference: reword the card, add a description, remove it, or keep it and treat its scatter as a finding. Make the change before launch. A card changed after the first real participants creates two studies under one name, and the analysis cannot tell them apart.
Preserve participant labels
The category names participants write are direct evidence of the language they use for those groups. Do not rename or merge them during the study. UXbeam keeps every label as typed and consolidates casing and plural variants with a count, so Payments, payments and Payment read as one label used three ways. Read the labels before the groupings; a label you would never have chosen is the point of running an open sort.
Reviewing unusual sorts without erasing disagreement
UXbeam shows every individual sort. Read them before reading anything aggregate.
| What you see | Read it as | Do |
|---|---|---|
| One group holding every card, or one card per group | Not a sort | Exclude, with the reason recorded |
| Finished in a time nobody could read the cards in | Not a sort | Exclude, with the reason recorded |
| Your own or a colleague's test run | Not a participant | Exclude |
| A coherent grouping that disagrees with everyone else | A participant | Keep. This is the sort that may turn out to belong to a second mental model. |
| Group names you find odd but consistent with the contents | Vocabulary | Keep. Odd to you is the finding. |
Excluding a participant changes every downstream view, including which mental models are detected. Exclude for quality, never for disagreement, and write the reason next to the exclusion.
Handoff to analysis
Then start with card sort analysis: what to inspect first, and what each view can settle.
A small annotated setup
The online-store set from the templates page, as it would be entered for an open sort.
Items field (one per line): the fifteen cards, with descriptions on two of them after || because a pilot showed Order tracking and Returns were read as products by some people.
Categories field: empty. This is what makes it an open sort.
Randomize item order: on, so Coffee makers is not always next to Desk lamps.
Require testers to sort all cards: off, because the pilot left Gift cards unsorted twice and that count is wanted.
Recruiting message: the open-sort message from the templates page, with the pilot's real duration filled in.