Not intentionally evil!

The ‘critical data studies’ approach is underpinned by concerns with the detrimental and harmful outcomes arising from digital data in contemporary society. These include various inequalities, injustices and discriminations that belie the promise of data bringing an essentially objective, value-free precision to any decision-making process. Rather than seeing digital data as essentially neutral, critical data studies is interested in unpacking the politics of data.

It is important to stress that acknowledging the persistence of these harms and injustices does not imply that computer engineers and data scientists are necessarily to ‘blame’. Indeed, very few people involved in the construction, development and implementation of these systems are actively aware of (let alone actively aiming for) unfair, unjust or discriminatory outcomes. Instead, these are harms that arise primarily from the wider structural issues and inequalities that shape the social contexts in which these data systems are being designed, produced and implemented. As Hintz et al. (2019, p.58-59) put it:

“The issue is not that harms are necessarily intentionally programmed into data-driven systems, but that the development, design and uses of algorithmic processes across social life are embedded in certain historical and institutional contexts, values and agenda that shape the multiple and differential ways in which citizens are implicated in them”.

Of course, it can be argued that those involved in the construction, development and implementation of these systems need to be more mindful of these issues and the likely outcomes of what they are doing. In this sense, all aspects of data-work need to be seen along problematic lines – i.e. as ‘deeply political decisions’ rather than a set of depoliticised actions (Hintz et al 2019). Rather than assuming that data-work is somehow ‘objective’, ‘neutral’, and that data has an inherently explanatory capacity, we all need to make sense of data and data-work in more nuanced and critical reflexive terms.

Thus, regardless of one’s best intentions, data-work (and computational science in general) is not an area where one can simply remain faithful to the maxim of ‘Don’t Be Evil’ without engaging with the wider contexts one is working in. In the context of our own research project, for example, this might include issues such as the commercial marketplace for personal data, or the deep educational inequalities that pervade school systems around the world. Just as critical social scientists certainly need to engage more with the technical aspects of data sciences, even a little inter-disciplinary understanding is likely to go a long way.

 

[notes from Hintz et al.  2019.   Digital Citizenship in a Datafied Society.]