What happens when decision support systems fail? — the importance of usability on performance in erroneous systems

Philipp Brauner; Ralf Philipsen; André Calero Valdez; Martina Ziefle

doi:10.1080/0144929X.2019.1581258

What happens when decision support systems fail? — the importance of usability on performance in erroneous systems

Philipp Brauner^*, Ralf Philipsen, André Calero Valdez, Martina Ziefle

^*Korrespondierende/r Autor/-in für diese Arbeit

Institut für Menschzentrierte Interaktive Systeme

Rheinisch-Westfälische Technische Hochschule Aachen

39 Zitate (Scopus)

Abstract

With the advent of artificial intelligence (AI) methods, smart decision support systems (DSSs) are becoming ubiquitous. Such systems help reduce complexity for operators by automating data integration tasks and recommending actions. However, these systems are sometimes flawed. It is not sufficiently understood whether, when and why operators comply with such systems in erroneous or correct cases. We empirically investigate compliance with correct and defective DSSs, the influence of correct and erroneous DSS's on performance and subjective factors related to compliance. In the study, a business game was used as an experimental setting in which 40 users took part. The impact of system correctness on user acceptance, trust, compliance and overall performance was investigated. The results show that the defective system reduces trust in automation (−47%), reduces usefulness (−58%), reduces acceptance (−62%) and reduces overall performance (−32%). Overall, the defective system was less user-friendly (−27%). Nevertheless, users who rated the system's usability higher, outperformed users who rated it lower. Usability is therefore an intermediary that compensates for the negative influence of erroneous DSSs.

Originalsprache	Englisch
Zeitschrift	Behaviour and Information Technology
Jahrgang	38
Ausgabenummer	12
Seiten (von - bis)	1225-1242
Seitenumfang	18
ISSN	0144-929X
DOIs	https://doi.org/10.1080/0144929X.2019.1581258
Publikationsstatus	Veröffentlicht - 02.12.2019

Fördermittel

The German Research Foundation (Deutsche Forschungsgemeinschaft–DFG) funded this work within the Clusters of Excellence Integrative Production Technology for High-Wage Countries (EXC 128), the subproject Cognition Enhanced Self-Optimizing Production Networks (Schlick et al. 2017) and Internet of Production (EXC 2023, 390621612). Authors owe gratitude to participants for their commitment and dedication to contribute to this research. Also, thanks are devoted to Sebastian Stiller, Quoc Hao Ngo, Marco Fuhrmann and Robert Schmitt for in-depth discussions and valuable feedback on this work. Further thanks go to Julia Offermann-van Heek, Anne Kathrin Schaar, Patrick Halbach, Fabian Comanns and Sabrina Schulte for their research. Finally, thanks to the anonymous reviewers for their critical and very helpful comments on this article. The dataset is publicly available (Brauner et al. 2018).

UN SDGs

Dieser Output leistet einen Beitrag zu folgendem(n) Ziel(en) für nachhaltige Entwicklung

SDG 3 – Gesundheit und Wohlergehen
SDG 4 – Qualitativ hochwertige Bildung
SDG 8 – Angemessene Arbeitsbedingungen und wirtschaftliches Wachstum
SDG 9 – Industrie, Innovation und Infrastruktur
SDG 11 – Nachhaltige Städte und Gemeinschaften
SDG 13 – Klimaschutz
SDG 16 – Frieden, Gerechtigkeit und verlässliche Institutionen

Dokumente

10.1080/0144929X.2019.1581258

Weitere Links

Link to publication in Scopus

Zitieren

@article{8cef02acaed9433e885ba0bd2d19e291,

title = "What happens when decision support systems fail? — the importance of usability on performance in erroneous systems",

abstract = "With the advent of artificial intelligence (AI) methods, smart decision support systems (DSSs) are becoming ubiquitous. Such systems help reduce complexity for operators by automating data integration tasks and recommending actions. However, these systems are sometimes flawed. It is not sufficiently understood whether, when and why operators comply with such systems in erroneous or correct cases. We empirically investigate compliance with correct and defective DSSs, the influence of correct and erroneous DSS's on performance and subjective factors related to compliance. In the study, a business game was used as an experimental setting in which 40 users took part. The impact of system correctness on user acceptance, trust, compliance and overall performance was investigated. The results show that the defective system reduces trust in automation (−47\%), reduces usefulness (−58\%), reduces acceptance (−62\%) and reduces overall performance (−32\%). Overall, the defective system was less user-friendly (−27\%). Nevertheless, users who rated the system's usability higher, outperformed users who rated it lower. Usability is therefore an intermediary that compensates for the negative influence of erroneous DSSs.",

author = "Philipp Brauner and Ralf Philipsen and \{Calero Valdez\}, Andr{\'e} and Martina Ziefle",

note = "Publisher Copyright: {\textcopyright} 2019, {\textcopyright} 2019 Informa UK Limited, trading as Taylor \& Francis Group.",

year = "2019",

month = dec,

day = "2",

doi = "10.1080/0144929X.2019.1581258",

language = "English",

volume = "38",

pages = "1225--1242",

journal = "Behaviour and Information Technology",

issn = "0144-929X",

publisher = "Taylor and Francis Ltd.",

number = "12",

}

What happens when decision support systems fail? — the importance of usability on performance in erroneous systems. / Brauner, Philipp; Philipsen, Ralf; Calero Valdez, André et al.
in: Behaviour and Information Technology, Jahrgang 38, Nr. 12, 02.12.2019, S. 1225-1242.

Publikation: Beiträge in Fachzeitschriften › Zeitschriftenaufsätze › Forschung › Begutachtung

TY - JOUR

T1 - What happens when decision support systems fail? — the importance of usability on performance in erroneous systems

AU - Brauner, Philipp

AU - Philipsen, Ralf

AU - Calero Valdez, André

AU - Ziefle, Martina

PY - 2019/12/2

Y1 - 2019/12/2

N2 - With the advent of artificial intelligence (AI) methods, smart decision support systems (DSSs) are becoming ubiquitous. Such systems help reduce complexity for operators by automating data integration tasks and recommending actions. However, these systems are sometimes flawed. It is not sufficiently understood whether, when and why operators comply with such systems in erroneous or correct cases. We empirically investigate compliance with correct and defective DSSs, the influence of correct and erroneous DSS's on performance and subjective factors related to compliance. In the study, a business game was used as an experimental setting in which 40 users took part. The impact of system correctness on user acceptance, trust, compliance and overall performance was investigated. The results show that the defective system reduces trust in automation (−47%), reduces usefulness (−58%), reduces acceptance (−62%) and reduces overall performance (−32%). Overall, the defective system was less user-friendly (−27%). Nevertheless, users who rated the system's usability higher, outperformed users who rated it lower. Usability is therefore an intermediary that compensates for the negative influence of erroneous DSSs.

AB - With the advent of artificial intelligence (AI) methods, smart decision support systems (DSSs) are becoming ubiquitous. Such systems help reduce complexity for operators by automating data integration tasks and recommending actions. However, these systems are sometimes flawed. It is not sufficiently understood whether, when and why operators comply with such systems in erroneous or correct cases. We empirically investigate compliance with correct and defective DSSs, the influence of correct and erroneous DSS's on performance and subjective factors related to compliance. In the study, a business game was used as an experimental setting in which 40 users took part. The impact of system correctness on user acceptance, trust, compliance and overall performance was investigated. The results show that the defective system reduces trust in automation (−47%), reduces usefulness (−58%), reduces acceptance (−62%) and reduces overall performance (−32%). Overall, the defective system was less user-friendly (−27%). Nevertheless, users who rated the system's usability higher, outperformed users who rated it lower. Usability is therefore an intermediary that compensates for the negative influence of erroneous DSSs.

UR - https://www.scopus.com/pages/publications/85062364168

U2 - 10.1080/0144929X.2019.1581258

DO - 10.1080/0144929X.2019.1581258

M3 - Journal articles

AN - SCOPUS:85062364168

SN - 0144-929X

VL - 38

SP - 1225

EP - 1242

JO - Behaviour and Information Technology

JF - Behaviour and Information Technology

IS - 12

ER -

What happens when decision support systems fail? — the importance of usability on performance in erroneous systems

Abstract

Fördermittel

UN SDGs

Dokumente

Weitere Links

Fingerprint

Zitieren