Love of the Fact Check

People who know me well are not surprised when I tell them that I enjoy fact checking.

People who don’t know me very well look some combination of shocked, horrified, or judgmental when I tell them that I enjoy fact checking.

What they are missing is that fact checking is not about proving people wrong — I don’t like having to flag notes or facts and then follow up for a “can I see your data set” conversation. It’s not about making anyone look bad or asking them to do more work and more research… it’s about ensuring that what is produced is the best piece we can build together.

For me, the fact check is like being a sleuth. I need to figure out what kinds of questions the initial data collectors were asking and whether or not the data is being interpreted accurately when it’s allowed to roam free and be interpreted by other people outside the initial collection team. This is sort of like sniffing out the initial environment of the “scene of the crime” where the data was collected.

Then I get to check out the arguments the opponents or other groups are making and see where there are interpretation issues. Sometimes data sets are very consistent.. sometimes it’s an entirely different story.

I get to look up conversations between brilliant thinkers and try to track down the initial inception point of a thought or idea that became a major piece of intellectual capital.

I get to have really cool conversations about strengthening arguments and narratives with facts. I LOVE this part. I want to make my speakers be the best they can be, because the talk should be able to stand on its own at the point that it’s given… and years later or when its viewed online by people in other contexts.

I hope next time people wont be so horrified when I tell them that I love fact checking. I know I always appreciate the feedback on my own work.

#ScholarFest

Last week I attended the ScholarFest at the Kluge Center in the Library of Congress.

Perhaps the best moment was being asked by the hostess if I would like to visit “the past, present or future?” and then, upon selecting the future, being led down a beautiful hallway in the Thomas Jefferson Building of the Library and settled into the beginning of a discussion about life on other planets.

The event initially came into my periphery when someone that I work with at TED sent a description of the event, highlighting their program for the “Lightening Conversations” and asked if I had time to go. I said, absolutely.

[Quick update: videos from Scholarfest are now on youtube. Here is the session cut for the Future]

First, what were the “Lightening Conversations?”

The first part of the ScholarFest program used scholars paired based on mutual research interests, tangentially related research interests, or directly opposing research interests.

Each pair was given 10 minutes to start a dialogue intersecting their research and/or engaging with each other’s work. Speakers were not directly introduced by the initial introduction to the event, instead weaving in a quick line or two about their work in the first few minutes of each session. There were five sessions in the first piece of the program and some time set aside for town hall style Q&A. The total program ran for an hour and 20 minutes before it transitioned into a new room with a new theme.

The structure of each pairing depended on what the two speakers decided they wanted to do. Participants were informed of their pairing and introduced to the other speaker the night before the panel. For some, it seemed they had found new collaborators and conspirators, even though their topics and opinions on various subjects varied so greatly. For others, the mix could be abrasive, but also ended quickly.

Some of the structures that evolved during these 10 minute Lightening Conversations:

  • each person introduced a few key points and themes from their research,

  • each person introduced their work and then asked their partner about their specific research work,

  • they started with a thematic question that applied to both of their areas of interest,

  • they presented a question directly to the other person

  • a science historian moderated/interviewed the scientist

  • the critiqued each other’s theories/work and had a lively discussion

  • they discussed and wove themselves from each other’s work into the same discussion

I was intrigued by the Lightening Conversation format for a few reasons. First, it seems like a great way to breathe life back into Academia. It was a wonderful treat for me, as a researcher, to watch experts in their fields have an open conversation and ask each other questions. It was a chance to see how their minds worked outside of purely academic contexts and formats. The informality and speed of the conversations meant that each person had to think on their toes.

Second, the interdisciplinary themes of the Future (and some of the other conversations, in particular Freedom of Speech) meant that these experts were asked to step outside of their fields of expertise and engage in new thought experiments. It made academia feel more human. Challenging. Like a continued experiment that the audience was invited to watch and engage with… not a typical experience when attending a university lecture. A lot of ground was covered quickly.

Finally, the audience was offered a wide range of perspectives before they were offered the opportunity to reengage with the entire cast of speakers from that session. Rather than pull from a single thread of opinions or thoughts, there was a tapestry of conversation to pull from and multiple experts could respond to our questions. This was nothing short of delightful. I felt very spoiled.

I would really like to see more of this take place at Yale (and others, but I can only speak from my experience). It brought rich life back into the research I’ve seen only in very long and dense academic texts. Looking forward to ScholarFest next year!

Love Your Data. Can I have some context with that?

You know what is sexy? Presentations where the data and algorithms presented by researchers come with a healthy does of real life context. [Also, other researchers who read applied statistics textbooks in coffee shops early in the morning. I have been doing this a lot recently and just made friends with someone who was reading a different book by the same statistician I was reading.]

I constantly complain that we lose a lot of information when we work with big data analytics. Part of it is that many researchers are encouraged to work with data from their desks in offices tucked away inside of universities or office buildings in major cities, far away from the ecosystems they are trying to describe through numbers and algorithms.

Nate Silver spends a lot of time talking about the weakness of prediction models in his book The Signal and the Noise: Why so many predictions fail — but some don’t. He points out that economists have trouble identifying relevant variables to make predictions. This is fair… economies are constantly changing in structure and dynamic. It would be really hard to collect appropriate data on the formal economy as it shifts, and even harder to keep track of informal economic activity in a way that would lend itself well to predicting output for the future.

I’ve found the only way that I truly understand the pulse of an economic ecosystem is by living and breathing the structure and community of it. After all, economies depend on communities and trust for transactions to take place at all. But this is for another post.

But I did find someone trying to add context to big data!

I watched this talk by Anna Rosling Rönnland from TEDxStockholm yesterday, and while the introduction is a little confusing, the center of the talk is important. The best way to watch this talk, in my opinion, is to consider the implications of using photographs to describe the spread of the distribution.

In non-jargon speak, this means, consider how your perspective on wealth disparity changes when you see how people in the richest 25% versus the middle versus the lowest 25% brush their teeth. This hits home a lot harder than quoting per capita numbers at someone would, because it also takes into account differences in pricing/living costs within the country. We can see where wages fall short and what that means in the day to day life of workers around the world. We gain perspective on data. And that’s sexy.

 

Could you walk away from your work?

In The Signal and the Noise: Why so many predictions fail — but some don’t, Nate Silver states that “one sign you have made a good forecast is that you are equally at peace with however things turn out — not all of which is in your immediate control.” [130]

I think this concept applies to far more than just predicting weather forecasts, stocks, or how well particular baseball players will perform over the course of their careers. This applies to decisions that we make and how well we do our work.

I know that I’ve done a good job with my research, or really, my work in general, when I am comfortable presenting it and leaving it there to speak for itself after I present the work. When I have truly done my best, I am comfortable walking away from the work. It can exist independently, without me.

The ideal for any organizer is that your program will continue running without you, even if you quietly disappeared. The ideal for any researcher is that the work has merit and value, even when you are not there to carefully re-explain it.

So that is what I strive for. When I complete work, am I at peace with it? Does it have the legs it needs to stand on its own. Am I able to grant the work its independence?