First Semester Recap

My grand plans of keeping the blog regularly updated have fallen by the wayside. An initial target to write something every week quickly became once every two weeks, then once a month, then we’re suddenly at the end of the semester and there have been no updates since August, aside from a new headshot photo. It’s a very nice headshot, I have to admit.

The silence on the blog front belies what has been a whirlwind semester. There were times it was intense, there were times I felt like I wasn’t doing enough, but looking back, I think I can be proud with the progress I’ve made in three months. To tally it up:

• Completed my first semester of coursework
• Used that coursework to develop and field an experiment on community governance
• Used that experiment to apply for a grant (which, alas, I did not get, but it was a great foundational experience) and present to my research center
• Used coursework in my other classes to develop and validate a quality measurement scale, setting me up for productive research next semester
• Read a bunch of papers – this will be important to doing my PhD, I’ve discovered
• Worked with the Prosocial Design Network, and have been privy to some amazing discussions with academics and practitioners on how we can best translate research into practice in prosocial design – this felt like attending a ‘super seminar’ given the expertise in the room
• Had the privilege of being flown out to Roblox headquarters to facilitate a discussion on fostering research for prosocial design in gaming – bonuses included being given a super comfy blanket by Roblox, seeing the best of San Francisco, and catching up with a couple of dear friends
• Started to build a wonderful community around amateur musical theater performance
• Made friends! I have found so much community in Philly. I went from knowing only one person in this city in August (who I am immensely grateful for introducing me to my awesome roommates!) to knowing about 50

A Bridge from the View

I came into graduate school with a semi-clear idea of what I wanted my research agenda to be, but also with an awareness that initial research agendas change dramatically in the first year or two as people realize “what it actually means to do research.” And although I’m definitely learning fast “what it actually means to do research”, I’m happy to say that – for now – I’m sticking to the agenda I came in with (see figure below).

Conceptual framework for Rehan’s research (or more ambitiously, a “Theory of Change for Digital Platforms”)

The community governance experiment I developed maps well onto all three elements of this framework. Respondents played the role of community administrators on a forum and evaluated whether user-generated content aligned with established community guidelines. I tested whether participants’ personal viewpoints influenced their judgment consistency, and initial findings suggest alignment affected evaluation patterns. Mapping this onto my framework, (a) community governance mechanisms fall under the umbrella of “platform governance and norms”, (b) my theory relied on motivated reasoning from cognitive psychology, and (c) the outcome hopes to address potential inconsistencies in community guideline enforcement.

The quality scale I developed not only measures discourse quality outcomes, but will be used to examine the ‘delta reward system’ on r/ChangeMyView, a community design feature which I interpret to positively influence discourse quality through Mercier and Sperber (2011)’s argumentative theory of reasoning, which covers both individual cognition and social interaction.

So “what does it actually mean to do research”? Some reflections:

Concept explication and measurement approaches are crucial. When I initially drafted my experiment’s study design and grant proposal, it got ripped apart. The main reason was the lack of a solid theory behind what I was proposing. Instinctively I had the right idea – alignment influenced judgments on content – but I had little justification from what is a vast body of literature on this topic to support my study design. This flowed through to construct validity – that we are properly measuring what we intend to measure. I initially confused my outcome, whether respondents thought content broke community rules, with one of the study manipulations. I essentially said, “We will manipulate the extent to which content breaks community rules, and we want to measure how that affects the extent to which people think the content breaks community rules”. Doesn’t make much sense! I realized I needed another dimension – incivility – and explicit forum rules stating that incivility violated community standards.

The other big reflection is to be humble in knowing what you don’t know, but hopefully will in the future. The answer to addressing that is simple in concept, if difficult in practice. Read. Then read some more. My advisor starts and ends every semester asking people in our center to share one piece of research that has blown our mind. This demonstrates the importance of reading regularly, to keep up to date with a constantly evolving literature, and one you’re playing catch up with simultaneously, as you’re consolidating foundational knowledge on topics you’re trying to become an expert on.

Technical foundations are important, and that leads nicely into plans for next semester. I’m taking an intro to stats class for the fourth time in my higher education next semester, but I’m excited to do so because every time I learn something new. Between that class, Modern Data Mining, and ongoing research, I hope to one day be proficient in R. I’d like to continue to build my knowledge of the mechanisms literature, build out a database of emerging interventions that could move the dial in tech industry, and continue to develop the two studies I started this semester into publishable work that provides tangible contributions to both literature and practice.

That’s all for now folks. I’ll see you at the end of next semester, if not before then!