Qualitative research as public scholarship

Originally posted at www.21stcenturyscholar.org

At this year’s AERA conference, Bill Tierney and I presented a paper, “The Role of Ethnography as Ethical and Policy-Relevant Public Scholarship.” We had a great panel, including Rob Rhoads, Jessica Lester, Laurence Parker, and Yvonna Lincoln. Fellow blogger Antar chaired. Michelle Fine acted as discussant, providing great commentary.

The idea for the symposium developed after Bill sent a link to an article in The Chronicle about Goffman’s On the Run. If you remember, last year, I blogged about the book. Rather than focusing narrowly on Goffman’s research—many people have already critiqued her work—the session focused broadly on concerns of conducting ethnography as public scholarship.

What is public scholarship? Stated simply, it is a scholar’s engagement with multiple publics in order to inform social issues, provoke civic participation, and promote social justice. Typical examples include writing nonfiction books, appearing on NPR, and creating policy reports. This blog is a form of public scholarship. Unconventional acts—although, certainly not rare—include teaching courses, participating in local political movements, and conducting participatory action research. The difference between the first and second categories depends on the scholar’s level of engagement. A nonfiction book creates a one-way conversation from researcher to reader. Organizing a neighborhood-based planning committee or providing a summer outreach program involves collaborative engagement.

Qualitative research is particularly well-suited for more participatory examples of public scholarship. For years, the Pullias Center, based on research, provided mentoring services for students in Los Angeles. They even created and shared an infographic (another form of public scholarship). And, among other examples, they have developed apps and games to reach more students.

Public scholarship, as an abstract concept, seems noble and harmless. Why wouldn’t scholars want to advocate for social justice and facilitate deliberative democracy? In practice, it is considerably more complicated. While some (particularly those who believe in positive science) may disagree, all research is a political act. To engage with multiple publics draws attention to its political nature and makes researchers vulnerable to critique. That makes some uncomfortable.

For qualitative researchers, whose work is often misunderstood or dismissed, public scholarship poses even more risks. Think about two examples:

• A group of university researchers conduct a large-scale experimental study. Education Week writes about it. The study gains national attention. Policymakers use the findings to argue for reforms. Later, statisticians at a think tank contest the validity of the findings. Debates ensue about the researchers’ methodological decisions.
• A researcher conducts a five-year ethnography. The scholar publishes a book that becomes a NY Times best-seller. She performs a popular TED talk and appears on national news outlets. Journalists and scholars begin to critique her work. Discussions transition from the topic of the book to the qualifications of the researcher.

How do the two scenarios differ? There is often a degree of separation between quantitative researchers and their studies. Critics may read the methods section and think, “That’s a terrible design.” But, they typically don’t denigrate the researchers. The same separation does not exist for qualitative scholars. If critics perceive a problem, they often target the methods and the researchers. At a certain level, this makes sense. The researcher is the instrument. However, having witnessed enough controversies, the discussions often become personal, not professional. Instead of discussing flaws in the methods, critics target defects in the researcher.

Still, qualitative research as public scholarship is important and necessary. It contributes unique and grounded perspectives and contests deleterious stereotypes. It also has the ability to incite change.

The question then becomes, how might qualitative researchers improve the utility of their research as public scholarship and, relatedly, establish standards and techniques to enhance the quality of their work. These are a few of the issues Bill and I address in our paper. We plan to revise it this summer and invite feedback. If you have suggestions, let me know via comments, email, or Twitter.

On issues of trustworthiness in qualitative research

Originally published on May 03, 2011.

Trustworthiness–frequently referred to as validity and reliability–in qualitative research involves two intertwined parts: process and product. What are the strategies necessary for a researcher to conduct rigorous research? And, how does a researcher present data in order to maximize trustworthiness?

Reflexivity performs a central task to both process and product. In other words, where is the researcher situated in relation to the study, subjects, presentation, and readers? Even more, what are the researcher’s own beliefs and experiences in relation to the topic of study? For instance, if I was adopted as a child and am now studying foster care youth, should I reveal that to the youth? Should I mention it in the final text? There are few steadfast rules. The answer may be yes or no, but the point is that the researcher is constantly engaged in thought about these issues.

Because strategies to improve trustworthiness during data collection–triangulation, member checks, multiple researchers, prolonged engagement, audit trails, multiple coders, and multiple and varied interviews and observations–are so well-known, I am going to focus today on trustworthiness in writing, acknowledging that most of my points also apply to the process of research.

The author’s presence in a text varies and depends on two factors–both critical to trustworthiness. First, presentation is slave to paradigm. What you have studied and what you like explains much of your stance on writing. I was raised in the humanities and grew up in the qualitative side of social science. My mentor and fellow blogger Bill is an accomplished life historian. As such, I’ve received exposure to and training with the method. I am also particularly fond of ethnography. For me, the lived experiences of marginalized individuals are a central concern; my influences inform my views on writing and how I view the world and my place in it.

The second factor is an author’s personal style. While style certainly relates to an individual’s training and chosen discipline, the voice of an author in either a life history or ethnography can differ considerably. The narrative strategies I employ are a matter of choice, depending on the style, voice, and tone I hope to achieve.

Why are paradigm and personal style critical to issues of trustworthiness? To be rigorous, qualitative researchers have to be transparent. Where do they stand in relation to the research? What did they do during the project? And, why should the reader believe him or her? A reader should always feel as though the researcher has given him or her the time and also taken apart the watch to show the gears inside and the process involved. Research is subjective, situated, and dated. It is the researchers job to grapple with these issues during the process and presentation of research.

Using social Media to improve learning

Last week, I discussed the use of social media to collect data and improve trustworthiness. In this blog, I talk about the benefits and pitfalls of social media to improve learning. 

I want to begin with a few underlying assumptions: First, standing still is not an appropriate strategy to improve underperforming schools and districts. The world is moving faster than ever. A trademark of successful schools is not only the ability to manage the massive challenges of day-to-day operations but also anticipate and embrace educational innovation, including technology. 

Second, schools in general and learning in particular are changing. Brick and mortar schools become less important every day. This is both good and bad news. The open source movement has the potential to democratize knowledge. Even students attending the worst schools will have access. That’s the good news. The bad news is that the most underperforming schools will most likely be the last to adopt digital learning and teach the skills necessary to capitalize on digital learning opportunities. Just as we have seen with charter schools, unequal implementation will lead to uneven opportunity. Counter to the hope of the movement, the participation gap may exacerbate the achievement gap. 

Education is changing, and that leads to uncertainty. School administrators have responded by prohibiting social media use in schools and establishing guidelines for use outside of schools. Reasons include the need to govern access to inappropriate content and guard against inappropriate interactions between teachers and student. I view these responses as shortsighted. Rather than dealing with social media, administrators are ignoring it. S. Craig Watkins recently discussed the problems with current technology use policies. Most importantly, the hardline stances illustrate the growing divide between how people interact and learn outside of school and what they do inside of school. 

Education is changing. Schools that incorporate blended learning like USC’s Hybrid High, School of the Future, and High Tech High are setting the standard for educational innovation. Informed uses of technology have the ability to improve access to and facilitate learning for all students. However, school administrators and teachers must first have the courage to embrace change. The transition will not go smoothly and will include failures. Sometimes, though, the reward is worth the risk. Besides, what’s the alternative? Stand pat? Defend the status quo? As far as I can tell, change is the only option.

Using social media to collect data and improve trustworthiness

This is the first of a two-part blog where I discuss the use of social media in research and practice. Today’s blog emphasizes methodological concerns. Next week, I will discuss social media in schools.

As regular readers of the blog know, I am conducting an ethnography that focuses on the lives of seventeen- and eighteen-year-old male teenagers in a low-income neighborhood. My sample includes a range of participants organized into three categories: low, middle, and high academic achievers. A small percentage of adolescents do not consistently use social media. Those teens— illustrating that opportunity aggregates discriminately in economically disadvantaged neighborhoods—most often live in low-income households, have the worst grades, and uneven access to technology. The majority of students, however, regularly use social media, especially Facebook, Twitter, and YouTube. It is a critical part of their lives.

How does a researcher incorporate social media data into a study? Few ethnographers blend real world and digital data; danah boyd is the most obvious exception. Even fewer methodologists have provided practical guidance. That should be of no surprise considering the sluggishness of the publication process and the rapidity of technological innovation and adaptation.

Researchers cannot ignore social media; however, students are adopting it in numerous ways, which makes data collection difficult. To make things even more complex, a divide still exists between the physical and digital worlds. In the future, apps like Path will potentially blend the two and provide innovative data collection tools and access previously unavailable. But, for now, we are in a sort of technological and methodological hinterland. 

During my study, I have approached social media data as part of document analysis. I friended and/or followed my participants and created lists in Facebook and Twitter. Everyday, I have checked the lists. For notable posts, I have done screen captures and uploaded and coded them using Atlas.ti—an alternative, as my colleague June Ahn did for a recent study, would have been to create a program to collect the posts.

Last week, I sat in a 12th grade class. The 40 students listened attentively to the teacher, or so I thought. As I checked Facebook, I saw a post from one of the students: “LMS  [like my status] if you want to date me.” I also saw a series of tweets from another student who was chatting with a friend. Both appeared to be listening in class. This example illustrates the value of collecting social media data. The scenario also highlights the value of social media data for trustworthiness. How does a researcher know his observations and interpretations are accurate? Social media data presents an option for triangulation. 

Educational settings are complex. Social media and technology only adds another layer to the complexity. Just as technology is changing the ways in which teenagers interact, it will also reshape every aspect of qualitative research, from collection to presentation. The challenge facing all qualitative researchers moving forward will be to integrate technology into their own methodological thinking.

Getting to the truth: Doing research with teenagers

Credibility is the first (and most important) criteria for establishing trustworthiness in qualitative research. Credibility, like it’s step-sibling validity, is often the subject of much debate; scholars argue about what it can and cannot do and what strategies researchers should and should not use to ensure rigor in research (see “Varieties of Validity,” an article Yvonna Lincoln penned, or “Qualitative Research and Public Policy”, an article Bill and I wrote). Plainly speaking, however, credibility is truthfulness. How does a researcher know his or her data and interpretations approximate the truth? And, what strategies did he or she use? When it comes to research with teenagers, however, credibility is anything but a straightforward idea. Permit me to elaborate with an example. 

The Smallwood Recreation Complex, located in the northwest corner of a neighborhood in South Los Angeles, spans nine acres. The main feature of Smallwood is a multipurpose brick building, which houses a gymnasium, boxing ring, weight room, and dance studio. A playground, soccer field, baseball diamond, and tennis and basketball courts dot the landscape behind the building. Large eucalyptus trees line the outskirts of the park.

During the spring prior to my year-long ethnography, I conducted a pilot study of the neighborhood. I visited parks, schools, and businesses and interviewed residents, teachers, and workers. I conducted participant observations at Smallwood five times. During each visit, the setting was often the same. Cars filled the parking lot. Adults sat at benches watching children on the playground. Young men played basketball. Three or four men in their 20s and 30s congregated at the building’s main entrance. Children and teenagers walked in and out of the building. After each visit, I often left feeling upbeat. Los Angeles is one of the most park-poor metropolitan areas in the United States; however, the young residents of this neighborhood had a nice place to play and exercise.

Last month, I asked Matthew, one of my informants, to go to Smallwood with him. I knew the 18-year old often boxed and played basketball at the park. On the morning of the scheduled visit, he said, “Hey Randy, we can’t go. Not today.” Two days later, while driving Matt home, I reminded him that we still needed to visit the park. “What?” he responded, “You want to go at the worst time. Three gangs is battling.” A Latino gang member shot and killed someone from a rival gang. The park, he said, was the hotspot. Matt continued, “They shot up right where I live. I was pissed and grieving.” Later, I discussed Smallwood with Matt and his peers. To them, the park and the surrounding neighborhood represented a contested territory, a place where violence could occur suddenly. 

How does a researcher make sense of such divergent experiences? What is the truth? If, as Paul Rabinow says, fieldwork is a “cultural activity,” my experience highlights the dual process of interpretation. I was both making sense of the experiences of my participants as well as myself in the field. My understanding of the park diverged significantly from that of my subjects. 

So, which is it? Is Smallwood a family-friendly park or gang-controlled territory? I have embedded in my study multiple strategies to ensure credibility. In this instance, different methods provided different interpretations. The challenge of credibility is not to eliminate different interpretations like crossing items off of a grocery list. The challenge is to acknowledge that multiple truths exist, often simultaneously, and to understand what that means for the lives of those involved.