奴役算法：从“解释权”到“更好决定权”？-研究论文资源-CSDN文库

需积分: 3 33 浏览量 2021-05-20 07:47:31 上传评论收藏 115KB PDF 举报

资源详情

资源评论

资源推荐

Enslaving the Algorithm: From a “Right to an

Explanation” to a “Right to Better Decisions”?

Lilian Edwards, University of Strathclyde [l.edwards@strath.ac.uk]

Michael Veale, University College London [m.veale@ucl.ac.uk]

Published in

IEEE Security & Privacy (2018) 16(3), 46–54, doi:10.1109/MSP.2018.2701152

As concerns about unfairness and discrimination in “black box” machine learn-

ing systems rise, a legal “right to an explanation” has emerged as a compellingly

attractive approach for challenge and redress. We outline recent debates on the

limited provisions in European data protection law, and introduce and analyze

newer explanation rights in French administrative law and the draft modernized

Council of Europe Convention 108. While individual rights can be useful, in pri-

vacy law they have historically unreasonably burdened the average data subject.

“Meaningful information” about algorithmic logics is more technically possible

than commonly thought, but this exacerbates a new “transparency fallacy”—an

illusion of remedy rather than anything substantively helpful. While rights-based

approaches deserve a ﬁrm place in the toolbox, other forms of governance, such

as impact assessments, “soft law,” judicial review, and model repositories de-

serve more attention, alongside catalyzing agencies acting for users to control

algorithmic system design.

1 Introduction

Businesses and governments are increasingly deploying machine learning (ML) systems to

make and support decisions that have a crucial impact on everyday life: decisions about

(inter alia) criminal sentencing and release on bail, medical treatment, eligibility for welfare

beneﬁts, what entertainment we see and can access, the price and availability of goods and

services delivered online, and the political information to which we are exposed. These ML

systems—colloquially entering public consciousness as just algorithms, or even just AI—have

been extensively criticized in the past few years as a result of a number of well-known “war

stories” that have revealed patterns of discrimination embedded but invisible to casual users

in such systems.

Because algorithms are trained on historical data, they risk replicating unwanted historical

patterns of unfairness and/or discrimination. For example, in hiring systems, a lack of women

being hired in the past may mean the systems fail to recognize the worth of female applicants,

Electronic copy available at: https://ssrn.com/abstract=3052831

or even outright discriminate against them. Luxury goods may be advertised to people with

certain proﬁles on social media and not to others, creating a consumer “under class.”

A severe obstacle to challenging such systems is that outputs, which translate with or

without human intervention to decisions, are made not by humans or even human-legible

rules, but by less scrutable mathematical techniques. A loan applicant denied credit by a

credit-scoring ML algorithm cannot easily understand if her data was wrongly entered, or

what she can do to have a greater chance of acceptance in the future, let alone prove the

system is illegally discriminating against her (perhaps based on race, sex, or age). This opacity

has been described as creating a “black box” society.

2 Enter the Right to an Explanation

Since the 1990s, the law in Europe has been concerned with this kind of opaque and difﬁcult-

to-challenge decision making by automated systems. In consequence, the Data Protection

Directive (DPD), a measure that harmonized relevant law across EU member states in 1995,

provided that a “signiﬁcant” decision could not be on based solely on automated data pro-

cessing (article 15). Some EU members interpreted this as a strict prohibition, others as giving

citizens a right to challenge such a decision and ask for a “human in the loop.” A second

right, embedded within article 12, which generally gives users rights to obtain information

about whether and how their particular personal data was processed, gave users the speciﬁc

right to obtain “knowledge of the logic involved in any automatic processing” of their data.

Both these provisions, but especially the latter, were not much noticed, even by lawyers,

and scarcely ever litigated, but have revived in signiﬁcance in the latest iteration of EU data

protection (DP) law within the General Data Protection Regulation (GDPR), which passed in

2016 and will come into operation across Europe in 2018.

In the GDPR, article 15 has been transformed into Article 22 and has arguably created

what the media and some technical press have portrayed as a new “right to an explanation”

of algorithms. The former article 12 has also been revamped to a new article 15 and now

includes a right to access to “meaningful information about the logic involved, as well as

the signiﬁcance and the envisaged consequences of such processing” (article 15(1)(h)). This

provision, notably, applies only in the context of “automated decision making in the context

of” Article 22. This leaves it unclear if all the constraints on Article 22 (discussed below) are

ported into article 15 (though our view is that it does not). Sadly, all this adds up to a reality

considerably foggier than the media portrayal.

Several factors undermine the idea that Article 22 contains a right to an explanation.

Primarily, Article 22 does not in its main thrust even contain a right to an explanation, but

is merely a right to stop processing unless a human is introduced to review the decision on

challenge. However, Article 22 does refer at points to a requirement of “safeguards,” both

where the right to prevent processing (paradoxically) does not operate, and where it does

but sensitive personal data is processed. In relation to the ﬁrst case, safeguards are partly

listed in Article 22(3), but in the second case, the only guidance is in Recital 71. (“Sensitive”

personal data in DP law refers to a restricted list of factors regarded as particularly important

such as health, race, sex, sexuality, and religious beliefs.)

Electronic copy available at: https://ssrn.com/abstract=3052831

It is important to note that, in European legislation, the articles in the main text are binding

on member states but are accompanied by “recitals,” which are designed to help states

interpret the articles and understand their purpose. Recitals are usually regarded as helpful

rather than binding, but this is contested and differs among states. Unfortunately, in relation

to Article 22, Recital 71 mentions some key matters not included in the main text. Article

22(3) mandates that safeguards include “at least the right to obtain human intervention on

the part of the controller, to express his or her point of view and to contest the decision,” but

the safeguards listed in Recital 71 “should include speciﬁc information to the data subject

and the right to obtain human intervention, to express his or her point of view, to obtain an

explanation of the decision reached after such assessment and to challenge the decision” (italics

added).

This strange mishmash of texts thus cannot ﬁrmly be said to mandate a right to explanation

in all or indeed any circumstances and may not be interpreted the same way from state to

state.

This is a serious, but not the only, problem with Article 22.

•

Article 22 applies only to systems where decisions are made in a “solely” automated

way—that is, there is no human in the loop—and there are very few of these and fewer

that are “signiﬁcant” (see below). How meaningful this input has to be is subject to

recent regulatory guidance,

but remains unclear and untested.

•

What is a “decision”? The GDPR gives us no help with this at all other than that it

includes a “measure” (Recital 71). Is sending a targeted ad to a user using an algorithmic

system a decision? It produces no binding effect; the advert may be ignored; and in

many cases, it is hard to see what action causally ﬂows from it. Yet as in the well-

publicized Latanya Sweeney example,

sending adverts promoting help with criminal

arrests solely to “black-sounding” names was worrying and offensive—and potentially

dangerous if these characterizations were inherited by systems selecting individuals for

stop and search or airport screening. Although a single advert delivery decision might

not have a signiﬁcant effect on an individual’s life, the cumulative effect on an entire

group or class may be worrying. Such group privacy impacts are not dealt with well

by DP law—an area based on individualistic human rights—and are exacerbated by a

continuing lack of provision for class actions in EU states.

•

Article 22 applies only to a decision that produces legal or other “signiﬁcant” effects.

This is vague in the extreme. Some would argue this could only apply to systems that

make important, binding decisions on things like criminal justice, risk assessment, credit

scoring, education applications, or employment. Yet such systems are rarely if ever

entirely automated, even if the human’s involvement is often nominal. Furthermore,

some commercial decisions may seem trivial as a one-off, but are signiﬁcant in aggreg-

ate. Mendoza and Bygrave argue that advertising decisions can never be signiﬁcant,

while European regulators recently produced guidance indicating the opposite.

Might

systems recommending buying choices or targeting adverts not limit a user’s worldview

or choices, or disseminate “fake news” via algorithmic ﬁlter bubbles? Arguably, such

phenomena are becoming deeply and signiﬁcantly destructive to our democracy. We

Electronic copy available at: https://ssrn.com/abstract=3052831

剩余13页未读，继续阅读

评论收藏

内容反馈

weixin_38538585

粉丝: 3
资源: 956

奴役算法：从“解释权”到“更好决定权”？-研究论文

评论0

最新资源

奴役算法：从“解释权”到“更好决定权”？-研究论文

评论0

多尺度能力：一种更好的算法交易绩效衡量方法-研究论文

论文研究 - 两台机器上相似大小的作业有序在线计划的更好算法

欧盟成员国的自动化决策：欧盟国家立法中算法决策的解释权和其他“适当的保障措施”-研究论文

论文研究 - 统一机器上订单在线调度的更好算法

好算法，更好的规范：算法工具如何重振披露监管-研究论文

遗传算法：评论、实现和应用-研究论文

论文研究 - 根据国际法和尼日利亚反贩运法对人口贩运中的运输，奴役和剥削的要素进行审查

被奴役的本体论：历史悠久的奴隶贸易民族-研究论文

国际移民的大数据：国际人权法下国家的机遇与挑战-研究论文

苹果公司：2020 年打击人口贩卖和奴役现象

initscripts-veth:用于将主机 veth 对奴役到主机桥的 Red Hat initscripts

奴役和自由的家庭伦理.doc

《通往奴役之路》读书笔记.doc

初中语文文摘人生被奢侈品奴役的弱魂灵

当人工智能来袭 未来我们的职业都将被机器奴役？

katyusha:for用于奴役Windows的邪恶和智能机器人

js13k-2019：xx142-b2.exe。 js13kgames 2019的条目

人工智能发展存在的问题.docx

人工智能发展存在的问题(1).docx

SDPlus：SDPlus

投资机构内部资料：FY23最新版苹果供应链清单Apple-Supplier-List

mochadick_enslave:Enslave 是 Minecraft 1.7.10 的村民奴隶制模组。 Enslave 是更大的 Mocha Dick 宇宙的子模组

Angry Aliens for Chrome-crx插件

Logiciels temps réel：有关信号处理、反馈和调节领域的实时软件-matlab开发

Qt 5实现串口调试助手 （源工程文件、0积分下载）

【SystemVerilog】路科验证V2学习笔记（全600页）.pdf

AutoSAR标准协议4.2.2

光伏-储能并网系统仿真.rar

NPPJSONViewer.zip

最新资源

当人工智能来袭未来我们的职业都将被机器奴役？

Qt 5实现串口调试助手（源工程文件、0积分下载）