Artificial intelligence is increasingly used to support managerial judgment, but effective human-AI decision making depends not simply on whether users follow AI advice, but on whether they rely on it selectively. This thesis examines whether uncertainty transparency improves calibrated reliance on AI recommendations under uncertainty. A mixed 2 × 2 survey experiment was conducted with 117 participants. Advice consistency varied within participants across two managerial scenarios, while uncertainty transparency varied between participants. Behavioral reliance was measured using Weight of Advice, complemented by measures of trust, perceived diagnosticity, confidence, and benchmark-relative judgment quality. The results provide no evidence that uncertainty transparency increased selective reliance: the Quality × Transparency interaction on Weight of Advice was small and imprecisely estimated. Participants nevertheless evaluated evidence-consistent advice more favorably in terms of trust and confidence, without showing correspondingly greater behavioral adjustment toward it. Higher initial confidence predicted lower reliance on AI advice. The findings highlight a distinction between evaluating AI advice and acting on that evaluation, suggesting that uncertainty information alone may be insufficient to produce calibrated behavioral reliance in ambiguous managerial decisions.

Artificial intelligence is increasingly used to support managerial judgment, but effective human-AI decision making depends not simply on whether users follow AI advice, but on whether they rely on it selectively. This thesis examines whether uncertainty transparency improves calibrated reliance on AI recommendations under uncertainty. A mixed 2 × 2 survey experiment was conducted with 117 participants. Advice consistency varied within participants across two managerial scenarios, while uncertainty transparency varied between participants. Behavioral reliance was measured using Weight of Advice, complemented by measures of trust, perceived diagnosticity, confidence, and benchmark-relative judgment quality. The results provide no evidence that uncertainty transparency increased selective reliance: the Quality × Transparency interaction on Weight of Advice was small and imprecisely estimated. Participants nevertheless evaluated evidence-consistent advice more favorably in terms of trust and confidence, without showing correspondingly greater behavioral adjustment toward it. Higher initial confidence predicted lower reliance on AI advice. The findings highlight a distinction between evaluating AI advice and acting on that evaluation, suggesting that uncertainty information alone may be insufficient to produce calibrated behavioral reliance in ambiguous managerial decisions.

AI-Augmented Managerial Decision Making: Uncertainty Transparency and Selective Reliance

PIOMBI, MARCO
2025/2026

Abstract

Artificial intelligence is increasingly used to support managerial judgment, but effective human-AI decision making depends not simply on whether users follow AI advice, but on whether they rely on it selectively. This thesis examines whether uncertainty transparency improves calibrated reliance on AI recommendations under uncertainty. A mixed 2 × 2 survey experiment was conducted with 117 participants. Advice consistency varied within participants across two managerial scenarios, while uncertainty transparency varied between participants. Behavioral reliance was measured using Weight of Advice, complemented by measures of trust, perceived diagnosticity, confidence, and benchmark-relative judgment quality. The results provide no evidence that uncertainty transparency increased selective reliance: the Quality × Transparency interaction on Weight of Advice was small and imprecisely estimated. Participants nevertheless evaluated evidence-consistent advice more favorably in terms of trust and confidence, without showing correspondingly greater behavioral adjustment toward it. Higher initial confidence predicted lower reliance on AI advice. The findings highlight a distinction between evaluating AI advice and acting on that evaluation, suggesting that uncertainty information alone may be insufficient to produce calibrated behavioral reliance in ambiguous managerial decisions.
2025
Artificial intelligence is increasingly used to support managerial judgment, but effective human-AI decision making depends not simply on whether users follow AI advice, but on whether they rely on it selectively. This thesis examines whether uncertainty transparency improves calibrated reliance on AI recommendations under uncertainty. A mixed 2 × 2 survey experiment was conducted with 117 participants. Advice consistency varied within participants across two managerial scenarios, while uncertainty transparency varied between participants. Behavioral reliance was measured using Weight of Advice, complemented by measures of trust, perceived diagnosticity, confidence, and benchmark-relative judgment quality. The results provide no evidence that uncertainty transparency increased selective reliance: the Quality × Transparency interaction on Weight of Advice was small and imprecisely estimated. Participants nevertheless evaluated evidence-consistent advice more favorably in terms of trust and confidence, without showing correspondingly greater behavioral adjustment toward it. Higher initial confidence predicted lower reliance on AI advice. The findings highlight a distinction between evaluating AI advice and acting on that evaluation, suggesting that uncertainty information alone may be insufficient to produce calibrated behavioral reliance in ambiguous managerial decisions.
File in questo prodotto:
File Dimensione Formato  
Piombi_Marco_888841.pdf

non disponibili

Dimensione 3.25 MB
Formato Adobe PDF
3.25 MB Adobe PDF

I documenti in UNITESI sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.14247/31026