Designing for Good Social Systems
[no description provided]There's a long story in the New York Times, "Where Countries Are Tinderboxes and Facebook Is a Match:"
A reconstruction of Sri Lanka’s descent into violence, based on interviews with officials, victims and ordinary users caught up in online anger, found that Facebook’s newsfeed played a central role in nearly every step from rumor to killing. Facebook officials, they say, ignored repeated warnings of the potential for violence, resisting pressure to hire moderators or establish emergency points of contact.
I've written previously about the drama triangle, how social media drives engagement through dopamine and hatred, and a tool to help you breathe through such feelings.
These social media tools are dangerous, not just to our mental health, but to the health of our societies. They are actively being used to fragment, radicalize and undermine legitimacy. The techniques to drive outrage are developed and deployed at rates that are nearly impossible for normal people to understand or engage with. We, and these platforms, need to learn to create tools that preserve the good things we get from social media, while inhibiting the bad. And in that sense, I'm excited to read about "20 Projects Will Address The Spread Of Misinformation Through Knight Prototype Fund."
We can usefully think of this as a type of threat modeling.
- What are we working on? Social technology.
- What can go wrong? Many things, including threats, defamation, and the spread of fake news. Each new system context brings with it new types of fail. We have to extend our existing models and create new ones to address those.
- What are we going to do about it? The Knight prototypes are an interesting exploration of possible answers.
- Did we do a good job? Not yet.
These emergent properties of the systems are not inherent. Different systems have different problems, and that means we can discover how design choices interact with these downsides. I would love to hear about other useful efforts to understand and respond to these emergent types of threats. How do we characterize the attacks? How do we think about defenses? What's worked to minimize the attacks or their impacts on other systems? What "obvious" defenses, such as "real names," tend to fail?
Image: Washington Post