How to better understand value change? Use the ValueMonitor, a data driven tool developed within the TU Delft!

Tristan de Wildt is a postdoctoral researcher at Delft University of Technology with a background in computer sciences. As part of his postdoc, he contributes to the ERC project Value Change and develops a tool that helps trace and understand value change using large text corpora. Tristan completed his bachelor’s, master’s, and PhD at TU Delft.

https://delftdesignforvalues.nl/wp-content/uploads/2023/12/tristan_de_wildt-e1703087862387.webp

INTERVIEW

Authors: Beatriz Lafuente Alcazar & Helma Dokkum

Designers, academia, policymakers, and communication agencies have a lot of data at their disposal seeking insight into the values associated with technologies, or when understanding shifts in values among the general public. However, time is often a limiting factor when designing for values. Tristan is working on the tool ‘Value Monitor’ to help them in this matter. Such a tool uses a Natural Language Processing (NLP) approach – specifically, topic modelling – to capture specific values through the distribution of words. The topic model makes searching for values way more effective as opposed to searching based on keywords. Tristan is currently refining the tool’s interface and can use your help to further enhance its relevance, also for academics!

Tristan will join us in our next Values Cafe on February 15^th to talk about his tool and give a short demo to the attendees.

Try it out: Try it yourself – Value Monitor!

Where did this idea start?

It started when I was doing a literature review on scientific articles on values. I was using keywords. However, values are often discussed in a latent manner. People sometimes explicitly name the value, but oftentimes they don’t. Instead, people use a wide range of words that refer to the idea of the value. For example, when talking about privacy we may be using the words private, encryption, etc. When doing a literature review, academics keep adding these keywords to their search entries. That increases the risk of finding less relevant articles because the keywords can also refer to other scenarios.

Can you explain what this ValueMonitor is about?

To responsibly develop and introduce new technologies into society, organizations need to address moral and societal values proactively. Governmental organizations might want to evaluate if policy programs had a significant impact on the consideration of privacy within target groups. Research councils might want to assess whether values voiced in society for AI and Blockchain Technology such as democracy and environmental sustainability are adequately reflected in research programs. The ValueMonitor can support this.

More information: ValueMonitor – Monitoring values for technology governance

The current version of ValueMonitor is based on news items. We are planning to extend this to scientific articles and policy documents as well.

Who can use it?

Different stakeholders can eventually benefit from it. We started with a use case focusing on reviews being done by policymakers and researchers who want to get a better understanding of the moral problems and values associated with specific technologies within the public debate.

Later on, we will also focus on communication agencies. ValueMonitor can help to reveal values emphasized by different societal groups.

How can it help in designing for values?

Designers oftentimes face the challenge of a lack of time to research the product they are making, and the values that are at stake. ValueMonitor will hopefully accelerate this. It can also help draw attention to values mentioned in the public debate but not by the direct stakeholders involved in a design project, or it might help to prepare for interviews with stakeholders.

https://delftdesignforvalues.nl/wp-content/uploads/2023/12/ValueMonitor-1-500×267.pnghttps://delftdesignforvalues.nl/wp-content/uploads/2023/12/ValueMonitor-3-500×276.pnghttps://delftdesignforvalues.nl/wp-content/uploads/2023/12/ValueMonitor-2-500×331.png

But how did you do the topic modelling?

We use a semi-supervised form of topic modelling. We feed a set of keywords referring to a value to the model, which then finds other words that refer to this value as well. In this way, we can guide the tool in the creation of topics that refer to values. So instead of keywords, we look for a distribution of words.

And why not use chat GPT?

A problem that comes with using ChatGPT for this is that the costs and energy used are higher. This is because the datasets we use are large and need to be updated with new articles every day. The time that chat GPT takes to evaluate each article is also relatively slow. We now have a model that has been trained by ourselves.

What insights did you get from the tool and data that strikes you the most?

One thing that I noticed is that connotations for the same values depend on where the text is coming from. Different datasets (e.g. newspaper articles, scientific papers, policy documents) have different functions within the societal debate, and therefore mention values for different reasons. Let’s take privacy concerns as an example. In scientific documents, privacy will be discussed in a more positive perspective than in the news. When doing sentiment analysis, you see a huge difference between text from scientific literature, news articles, and social media. A striking question for me is to what extent Twitter (or X) can represent what society thinks. Perhaps Twitter is mainly about the clash of values as opposed to being representative of common values.

What is the next step in taking this tool to another level?

My ambition is to make the design of values ‘philosophy’ more broadly available to the outside world. The more I talk to people in academia, the more I realize that seeing the world in terms of values is very useful. It would be great if I could make the tool available for policymakers, for the public and private sector, so that people have the means to do as much good as they can, which starts by discussing the relevant values. A lot of people want to do so, but they lack the proper tools to do so.

The practical next step is to introduce it to academia, policy makers and I also want to refine the interface on the website. Then, I want to work on the data availability and create a business case around the costs of further developing the tool. However, for that, I first need some feedback on the current prototype!

If users want to provide feedback, where can they do that?

In this pilot iteration, the ValueMonitor is only using Dutch news articles for a relatively short period. A possibility to upload and analyze your own datasets will be available soon. Please keep in mind that we envision it to be based on a more solid database after we explored further use cases and drafted the business case. You can request a login and password to access the tool on the ValueMonitor website. The tool also has a feedback form to provide suggestions.

The ERC project is almost over. What is next for you?

Starting in February, I will be working for Yaghma where I will continue to understand the needs better for users in the public and private sector and develop computational tools accordingly. I hope to still collaborate with- and connect to the TU Delft.