Kules, B., & Shneiderman, B. (2008). Users can change their web search tactics: Design guidelines for categorized overviews. Information Processing & Management, 44(2), 463-484. doi:10.1016/j.ipm.2007.07.014
Abstract: Categorized overviews of web search results are a promising way to support user exploration, understanding, and discovery. These search interfaces combine a metadata-based overview with the list of search results to enable a rich form of interaction. A study of 24 sophisticated users carrying out complex tasks suggests how searchers may adapt their search tactics when using categorized overviews. This mixed methods study evaluated categorized overviews of web search results organized into thematic, geographic, and government categories. Participants conducted four exploratory searches during a 2-hour session to generate ideas for newspaper articles about specified topics such as “human smuggling.” Results showed that subjects explored deeper while feeling more organized, and that the categorized overview helped subjects better assess their results, although no significant differences were detected in the quality of the article ideas. A qualitative analysis of searcher comments identified seven tactics that participants reported adopting when using categorized overviews. This paper concludes by proposing a set of guidelines for the design of exploratory search interfaces. An understanding of the impact of categorized overviews on search tactics will be useful to web search researchers, search interface designers, information architects and web developers.
Summary: The authors built a “faceted or guided search” interface called SERVICE, to support exploration of search results. Categories for facets were drawn from the open directory project and a database of US Government websites.
(1) How do searchers think differently about their search tactics when categorized overviews are available to augment the result list? (2) What kinds of behaviors do searchers exhibit when categorized overviews are available? (3) In what ways could the presence of categorized overviews affect the quality of the search outcome?
Methods: A 2 x 2 within subjects comparative design with 24 participants, primarily journalism students. They used a baseline system similar to Google first, then the experimental system in a fictional search scenario. Qualitative methods: observation and an interview, and quantitative methods were used to evaluate the experimental system. Codings emerged from the interview:
6 behavioral codes: [Overview helped to filter or narrow list; Issued more general queries; Issued fewer queries; Ping-ponged – alternated between using the overview and the list; Explore – used the overview to explore the results; Used the overview to refine search]
7 tactical codes: [Broad queries; Organize examination by overview; Overview as backup; Preview before narrowing; Assess result set; Probe using categorized overview; Ignore]
11 cognitive and affective codes: [Categorization problems; Helped with task; Thought differently; Complex/overwhelming; Less complex/ overwhelming; Become more comfortable over time; Frustration; Helped when subject had idea in mind; Misleading/distracting; Concerned they might miss something; Cautious]
9 judgment codes: [Problems experienced (of any type) were not a hindrance; Problems experienced (of any type) were a minor hindrance; Saw something that would not have been seen otherwise; Search went faster Search went slower Got more results from a new query; Search was more efficient Found poorer quality information Got side-tracked]
Results: The experimental, faceted system, changed user behaviors. They wxplored results more deeply. Users liked the interface, although no quantitative differences in quality of task outcomes was detected. and provided feedback for suggested faceted search system improvements:
1. Provide overviews of large sets of results.
2. Organize overviews around meaningful categories.
3. Clarify and visualize category structure.
4. Tightly couple category labels to result list.
5. Ensure that the full category information is available.
6. Support multiple types of categories and visual presentations.
7. Use separate facets for each type of category.
8. Arrange text for scanning/skimming.
Figure. The SERVICE Interface