Interactive IR user study design, evaluation, and reporting /
By: Liu, Jiqun [author.].
Contributor(s): Shah, Chirag [author.].
Material type:
BookSeries: Synthesis digital library of engineering and computer science: ; Synthesis lectures on information concepts, retrieval, and services: #67.Publisher: [San Rafael, California] : Morgan & Claypool, 2019.Description: 1 PDF (xvii, 75 pages) : color illustrations.Content type: text Media type: electronic Carrier type: online resourceISBN: 9781681735801.Other title: Interactive information retrieval user study design, evaluation, and reporting.Subject(s): Information storage and retrieval systems -- Use studies | Information services -- Use studies | Research -- Evaluation | interactive information retrieval | user studies | faceted approach | evaluation | controlled laboratory study | field study | human-information interaction | information seeking and retrieval | user study reporting practiceDDC classification: 025.04 Online resources: Abstract with links to full text | Abstract with links to resource Also available in print.| Item type | Current location | Call number | Status | Date due | Barcode | Item holds |
|---|---|---|---|---|---|---|
E books
|
PK Kelkar Library, IIT Kanpur | Available | EBKE917 |
Mode of access: World Wide Web.
System requirements: Adobe Acrobat Reader.
Part of: Synthesis digital library of engineering and computer science.
Includes bibliographical references (pages 63-74).
1. Introduction -- 1.1. Background : user study in IIR research -- 1.2. Aim of the work -- 1.3. Systematic review and user study evaluation -- 1.4. Implications of the faceted framework
2. Interactive information retrieval -- 2.1. Background -- 2.2. Understanding user behavior and experience -- 2.3. System/interface features evaluation -- 2.4. Meta-evaluation of evaluation metrics -- 2.5. Summary
3. Methodology : paper selection and coding scheme -- 3.1. Paper selection : SIGIR, CHIIR, CHI, JASIST, IP&M, and TOIS -- 3.2. Coding scheme
4. Faceted framework of IIR user studies -- 4.1. Structure of the faceted framework -- 4.2. Under-reported facets and factors -- 4.3. Summary
5. Evaluating IIR user studies of different types -- 5.1. Understanding user behavior and experience -- 5.2. System/interface features evaluation -- 5.3. Meta-evaluation of evaluation measures -- 5.4. Summary
6. Implications and limitations of the faceted framework -- 6.1. Implications for interaction IR user study design and evaluation -- 6.2. Implications for IIR user study reporting practices -- 6.3. Limitations and challenges -- 7. Conclusion and future directions.
Abstract freely available; full-text restricted to subscribers or individual document purchasers.
Compendex
INSPEC
Google scholar
Google book search
Since user study design has been widely applied in search interactions and information retrieval (IR) systems evaluation studies, a deep reflection and meta-evaluation of interactive IR (IIR) user studies is critical for sharpening the instruments of IIR research and improving the reliability and validity of the conclusions drawn from IIR user studies. To this end, we developed a faceted framework for supporting user study design, reporting, and evaluation based on a systematic review of the state-of-the-art IIR research papers recently published in several top IR venues (n=462). Within the framework, we identify three major types of research focuses, extract and summarize facet values from specific cases, and highlight the under-reported user study components which may significantly affect the results of research. Then, we employ the faceted framework in evaluating a series of IIR user studies against their respective research questions and explain the roles and impacts of the underlying connections and "collaborations" among different facet values. Through bridging diverse combinations of facet values with the study design decisions made for addressing research problems, the faceted framework can shed light on IIR user study design, reporting, and evaluation practices and help students and young researchers design and assess their own studies.
Also available in print.
Title from PDF title page (viewed on June 26, 2019).
There are no comments for this item.