NewIn: Orestis Papakyriakopoulos

The way to fair digital technologies

Can you recall an outstanding example of discriminatory technology?

A very clear example was the facial recognition function of a phone model for unlocking the phone. Because the technology was not as effective in distinguishing between people with Asian background, unauthorized individuals were sometimes able to access phones. When those who train AI models are unaware of potential discrimination, the training data will replicate the social asymmetries of the real world. For that reason, technologies are more likely to discriminate against women and also against people with darker skin, for example when the image optimization of a camera is geared to lighter-skinned people.

Are there areas where there are greater risks of encountering discriminatory technologies?

We can run into problems of this kind anywhere. But of course the consequences are more serious in some areas. For example, when non-inclusive algorithms are used in medicine or when AI is used to filter out applications or approve credit requests.

What do you want to achieve with your research?

First, we want to uncover and document such problems as discrimination, hate speech and bad decision-making. Second, we want to show policymakers how technologies could be regulated in order to prevent these problems. And third, we want to study how the collection of data can be more inclusive and how models can be developed that are fair by design – in other words, have fairness taken into account from the very first development stages – independently of the big tech companies.

Research has already identified many problems in recent years. Have companies been giving more consideration to these issues when developing the latest technologies such as generative AI?

Yes, we are already seeing a lot more thought being given to these matters, not only in generative AI, but also with conventional algorithms. That is because it has also become a topic in the training of developers in the meantime. And because non-discriminatory models are now seen as more accurate and generally better.

Is that why you describe yourself as a creative optimist?

The problems are not there because most people do not care about discrimination, but rather because we are dealing with complex social and socio-technical problems, often with unexpected effects. Most people in companies and in the world of politics want to fix mistakes. For that reason, we want to enter into a dialog. To that end, we have established the Civic Machines Lab at the TUM Think Tank. We organize workshops to bring together researchers, companies, NGOs and policymakers to work on solutions.

And what role does creativity play in your work?

I’m always trying to be more creative in my teaching. I want students in my courses to address their own questions and not only learn what I think they should learn. With doctoral candidates I also try to see what they want to achieve themselves and provide the necessary resources. This is in contrast to the German academic culture, which has traditionally tended to be very hierarchical – as reflected in the term “doctor father” for thesis supervisor.

Your own career path has also been somewhat unconventional, starting with civil engineering before studying philosophy of technology and ending up in informatics.

The jumps between the subjects were not quite as big as they might sound. When I studied civil engineering, I did a lot of mathematical modelling. I was always interested in ways of incorporating societal aspects. The former master’s degree program in philosophy of science and technology at TUM – like today’s master’s in science and technology studies – offered considerable freedom to combine social and technological aspects. That coincided with the first Big Data wave, and I have continued to work in that area since doing my doctorate.

What is the biggest challenge today?

Today a student came to me and said: The model that we are using for our experiments is actually too old. But we only developed it six months ago. There is enormous pressure to be very fast if we want to investigate the latest technologies, which are already being used by people. But of course that is also the exciting thing about it.