What is algorithm appreciation?

Algorithm appreciation is the tendency to take more of a piece of advice when you are told a machine produced it. Jennifer Logg, Julia Minson and Don Moore established it in 2019 by showing the same numbers to different people under two labels and measuring how far each person moved. The effect held across estimation and forecasting tasks, it weakened when the alternative on offer was the participant's own judgement, and in the one study that tested professionals it reversed.

The same advice, two labels#

The design is plainer than the result. A participant makes an estimate, receives a piece of advice, and makes the estimate again. The measure is Weight on Advice: the distance moved, divided by the distance between the first estimate and the advice. Nought means the advice was ignored. One means the participant abandoned their own view entirely. The advice itself was identical in every condition. Only the label changed.

In the first study, 202 participants estimated the weight of a person in a photograph and then saw a figure of 163 pounds, which was one pound under the truth. Told it came from an algorithm, they moved 0.45 of the way towards it. Told it came from other people, they moved 0.30. The gap was reliable at F(1, 200) = 8.86, p = .003, d = 0.42, and it survived controls for gender, numeracy and initial confidence.

It repeated on Billboard chart positions with 215 participants and on forecasts of whether a stranger would enjoy a date with 286 participants, where the split was 0.38 against 0.26. In a fifth study, 154 university participants who saw only one advisor relied more on the algorithm, and 75 per cent of those offered a choice between the two picked the algorithm.

Three details about how the work was done belong beside the numbers. All the sample sizes were set in advance by power analysis. Six of the seven studies were pre-registered, the exception being the weight-estimate study, which the authors say predates pre-registration becoming their standard practice, and they say so on the page. Materials, data and the pre-registrations sit on the Open Science Framework.

Researchers predicted the opposite#

The most useful study in the paper measures the field rather than the public. The authors circulated the date-forecasting materials to academic researchers through the Society for Judgment and Decision Making list and asked them to predict what participants would do. The researchers predicted algorithm aversion, at a mean of 0.14 in the wrong direction, while the participants had shown appreciation at minus 0.11. The prediction was wrong by more than a standard deviation, t(118) = 14.03, p < .001, d = 1.25, and graduate students were no better calibrated than senior researchers.

That result is the reason the paper matters more than its effect size suggests. A published literature had been read as saying people reject algorithms, and the people who had read it most closely were the ones most confidently wrong about what a participant would do.

The forecasters who would not listen#

The fourth study put the same design to 70 United States national security professionals whose work involves geopolitical forecasting, alongside 301 participants recruited online, on four questions including electric vehicle deliveries and the triggering of Article 50. The professionals forecast far more often at work than the lay sample, and they were slightly more familiar with algorithms.

They discounted everything. The interaction was reliable at F(1, 338) = 5.05, p = .025, and the professionals took less advice than the roughly 30 per cent that advice-taking research usually finds. Their accuracy suffered: on Brier scores the interaction ran F(1, 366) = 4.16, p = .042, and the lay participants who accepted algorithmic advice ended up more accurate than the experts who refused it. The authors' summary is exact: algorithmic advice "falls on deaf expert ears, with a cost to their accuracy".

Two cautions travel with that finding and are easy to drop. Sixty-one experts entered the main analysis, and 67 of the 70 were men. The authors state in their own text that the two samples "likely differ in many aspects beyond just expertise in forecasting". The result is a signal about a population, not a measurement of what expertise does.

Appreciation shrinks when the rival is you#

One study borrowed Dietvorst's own materials and asked 403 participants to stake a bonus on either an algorithm's estimate or a person's. Against another participant, 88 per cent chose the algorithm. Against their own estimate, 66 per cent did. The drop is large and reliable, z = 6.62, p < .001, and the mechanism the authors propose is not distrust of the machine but attachment to oneself.

That reading changes what the wider literature is about. If much of the apparent hostility to algorithms is confidence in one's own judgement wearing a different coat, then interventions aimed at building trust in the system are aimed at the wrong variable.

How this sits beside algorithm aversion#

Two famous findings appear to contradict each other and do not. Algorithm aversion describes what happens once somebody has watched a system err. Algorithm appreciation describes the weighting before any performance information has arrived. Logg and colleagues make the point against their own interest: in the control conditions of the aversion studies, before any errors were shown, participants chose the algorithm more often than they chose themselves.

So the line between these two pages, written here so a later reading does not merge them: aversion is measured after an error and appreciation is measured before one. The aversion page holds the response to visible failure. This page holds the default weighting, the population that does not share it, and the size of the gap between what people did and what the researchers expected.

Where this changes how a team is set up#

The finding organisations act on is usually the first one: people over-weight machine advice, so train them to be sceptical. The fourth study points the other way for the people whose judgement the organisation is actually paying for, and it puts a cost on their scepticism.

Deploy one assistant across a mixed workforce and the same tool can produce over-reliance among people who cannot yet check it and refusal among people who can, with the first group's errors invisible and the second group's losses never recorded at all. A blanket instruction about how much to trust the system is wrong for one of the two groups whichever way it is written. What travels is a rule about when an override is required and when it must be justified in writing, which is the argument made at when should I override AI.

What a laboratory weight-guess cannot settle#

Every task in the paper is a numerical estimate or a forecast with an answer that arrives quickly and unambiguously. Most professional judgement has neither property. The outcome comes late, arrives mixed with other causes, or never comes at all, and nobody learns whether an override was correct. Whether the labelling effect behaves the same way without feedback is untested.

The human comparison is also narrower than the headline implies. The advice attributed to people came from other participants or an average of them, never from a human expert, and the authors say directly that people may prefer a human expert to a peer strongly enough to swamp any appreciation of algorithms.

Then there is the word itself. Asked to define an algorithm, participants in the 2018 studies described mathematics, an equation or a calculation 42 per cent of the time and a step-by-step procedure 26 per cent of the time. A system that answers in fluent first-person prose is a different object, and nothing here tests it. Anyone citing this paper about a large language model is extending it.

One printed inconsistency belongs on the record. The paper's summary table gives the researcher-prediction study a sample of 199 while its own method section describes 119 participants from 120 completed surveys, and every test statistic reported for that study carries 117 or 118 degrees of freedom. The 119 is the figure the analysis supports. The abstract also says six experiments where seven numbered studies and two further samples are reported.

Finally, a result the popular reading of this paper drops. In a benchmark study of 671 participants, the algorithmic advice was an average of 314 people's estimates, so the correct move was to abandon one's own guess completely. Participants weighted algorithmic advice above human advice and still fell far short, F(1, 669) = 275.08, p < .001, d = 1.30. Both labels were underweighted. Appreciation is a difference between two conditions and not a statement that anybody relies on machines too much.

Related SuperSkills research#

On the mirror finding, algorithm aversion. On over-acceptance once a system is in use, automation bias and automation complacency. On what happens to a specialist's own judgement, AI and expert judgement. On the taxonomy that holds all of these, use, misuse, disuse and abuse. On the boundary nobody can see from the output, the jagged frontier.

Key sources

• Logg, J. M., Minson, J. A. and Moore, D. A. (2019). Algorithm appreciation: people prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151, 90-103.
• Dietvorst, B. J., Simmons, J. P. and Massey, C. (2015). Algorithm aversion: people erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1).
• Parasuraman, R. and Riley, V. (1997). Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors, 39(2), 230-253.

About this definition#

Algorithm appreciation is Logg, Minson and Moore's term and is not a SuperSkills coinage. The paper was read in full at source on 17 September 2026 in the copy the first author hosts, which carries the journal pagination, the digital object identifier and the dates of receipt and acceptance. Rahim Hirji is the author of SuperSkills (Kogan Page, 2026).