In Part Two of the history of Dedoose, we learned how Drs. Weisner and Lieber tried to better understand what any numerical scale response, like a ‘3,’ means in a qualitative data analysis context.  

Once they were able to gather and summarize the contents of qualitative data collected from a range of families, they discovered related topics that appeared most often. In this installment, we return how our co-founders used qualitative and quantitative research challenges as a basis for building a QDAS tool that would help to compile and assess these data sets within a single platform.

Dr. Lieber:

The EFI Manual, which we can make available on our website and in our documents already, is a thorough description of how you get there. Because that’s part of the challenge when we think about measurement.  

That’s part of one of the most frequent topics that we hear about is inter-rater reliability. How do you do that? We have these quantitative approaches from Cohen’s Kappa Coefficient and then you have the problem of somehow putting reliability around these ratings.  

What I learned from - I am a quant nerd, I lean that way, my brain still works that way as much as my work in Dedoose is much more qualitative. My brain still works in very quantitative ways, but to think about it from my perspective we must have a way of describing that, and Tom has many presentations I have listened to and have slides from talking about how you produce these ratings.  

The EFI manual, I was glancing at some of the pages the other day, how do you start? Describing this especially important versus less important? Its richness must be based on the data. So, people come to Dedoose to say, how do I accomplish inter-rater reliability in a qualitative approach with these ratings, and it is hard. It takes some work. But boy, when you can accomplish it, it is tremendous the data integrative possibilities that it allowed us to accomplish. That is where our technology I think has been able to boom.  

Dr. Weisner: One of the great advantages of Dedoose and it was important in our development process, is to be sure we captured reliability as traditionally measured, as Eli was saying. Also, validity is matched to the narrative experience or behavior of the family or medical establishment, or whatever you are trying to measure. One of the great advantages of Dedoose, (which) was important in our development process, is (being) sure we captured reliability as traditionally measured, as well as validity that is actually matched to the narrative experience.

Then there’s veridicality, give me an example of what someone who scores 5 on a rating would be like, would do, say, feel. If you can do that, now you have something that is truly interpretable and meaningful. So, we always try to include those three things in assessment. That is why the EFI has a little interpretation or story of what someone with a “5” or a “7” or a “2” would think, feel, would do.  

Dr. Lieber: Yeah, I am compelled to say, what part of what we are talking about is, the methodological foundation of where Dedoose came from. As a tool, under Jason’s leadership, it is a fantastic tool. We have accomplished marvelous things with the technology itself to serve this goal. That is what we were always after in building this, we just wanted to make our service better.  

What I am compelled to say here, as we talk about how we do it, how do you accomplish these things, you need the tools, yes Dedoose can be it or not, but you also must have a framework for thinking about it and the knowledge base.  

When we conceived of the IMMR (Institute for Mixed Methods Research) I do not know how many years ago, that is what we were thinking about in addition to all the other things that we, ambitions that we had for what kind of services could a research or academic branch of the Dedoose operation provide.  

Dr. Weisner: Eli mentioned the EFI, or Ecocultural Family Interview, which he emerged from this perspective if you want to be able to integrate many ways of trying to understand interpretive significance. One example of this is a mother who had kids with disabilities who was participating in a round table. One of the quantitative researchers in the audience asked her to talk about the family assessment scale, which is one of the rating scales we use.  

She paused for a second and said, “Let me just tell you what I do every day to take care of my son. I am just going to start in the morning, and I will walk through the day. Here is what we must do to get him up for school and deal with temper tantrums,” and all the things this mother had to do. That leads to the EFI part of all of this, which is “How do you capture this type of information? What we decided to do based on ecocultural theory and activity theory was to ask parents and kids and other people who were participating in our various research projects to walk me through their day from the time they start from the time they go to bed.”  

What are the activities you do? Who is there? How are you feeling about those activities? Do you have the resources you need or are you struggling to get them? Do you feel this is sustainable? Can you keep doing this or is it always a margin on the edge and you are struggling all the time? Can you persist and do it on a routine basis? Which of course makes it easier. If you could change something about your daily activities and your routine, what would you like to change?

These were very revealing about the context, the circumstances, the cultural ecology around the kids, parents, and schools and all the institutions involved in many of our research studies. The interviewing technique we developed which is part of the EFI which Eli developed, the interview starts not with asking someone “So on a 7-point scale how happy are you with how things are going in your parenting these days?” It does not start with that. It starts with taking me through your day, tell me about it. Tell me about your morning, tell me about good and bad experiences.

‍

The Birth of EFI

The Ecocultural Family Interview (EFI) story is a reminder that innovations often emerge from the intersection of compassion and scientific rigor.

The idea for EFI did not come from thin air. Rather it was born out of genuine interactions and experiences with families and parents. One pivotal moment was when a lady participating in a round table expressed her frustration with traditional scales and instead urged them to simply listen to her subjective experience. This demand reinforced the importance of capturing both the subjective and contextual aspects of family interactions.  

Methodological Shift

To bring their vision to life, our founders pondered the methodology of the times with an eye for capturing the individual’s subjective interpretations and perspective withing the broader context of family interactions. This required consideration of the behavior, routines, settings, values, and relationships within the family.  In the context of the quantitative and qualitative data gathering strategies of the time, the EFI’s approach stood out as more holistic and balanced in ways that facilitate more multidisciplinary and mixed methods research.  

A Diverse Social Science Group - Putting Focus Group Data to Work

The conception and development of the EFI took place within a multidisciplinary social science research group at UCLA. Comprised of individuals from backgrounds including anthropology, psychology, and sociology, this collaborative environment provided a fertile ground for innovation and transformative thinking.

Using Empathetic Listening and Multidisciplinary Collaboration to Design a Research Tool  

The origins of EFI and its connection to the Dedoose design and feature set, are a testament to the power of empathetic listening and multidisciplinary collaboration.  In essence, Dedoose was inspired from real-life experiences that combined diverse perspectives. Our co-founders used these perspective to shape a platform that seeks to capture the complexity of family interactions that span the qualitative and quantitative continuum in revolutionary ways.

‍

Dr. Lieber: One item that jumped out at me is that the EFI approach helped us find new ways of hearing the participants. I think we are missing the connection between how you got the idea to include the EFI, it was really the other way around. Were there moments that you can recall that shifted your and your colleagues' conversations? Maybe when you were developing this stuff...

Dr. Weisner: I think it came from ethnography. The idea came from listening to families and parents who had kids with disabilities. A lady who said in this round table, 'You know- stop giving me these scales let me just tell you what it's like. So, it started there. Then the question was: methodologically, how could we capture both the subjective experience and the context, the ecology around the family? Further, how to capture this around the unit of analysis versus the activity, the setting, the interactions. You have with your kids, and your siblings, and your husband, and all the people involved not only the individual. Too many measures are implicitly individualistic. You ask a single parent or a single child for an assessment or a rating rather, than always including the context. Then also asking about the individual rating.  

Dr. Lieber: So, that's how it started. Tell me, do you think this is accurate: my understanding, because I came into the group to serve in a particular role, and this is a mixed method social science group, is that this wasn't an ethnography group. It was an anthropology group. There were psychologists and people who were coming from different disciplines from a more quantitative side.  

Then you and others from a more qualitative side. For me, I know that was a very cool place to be exploring. This was because there were people you could talk to, and so I guess, maybe that's what I felt. To me, at the lab, it was a very special place and an opportunity for me to change my own thinking. Would you say it's fair that everybody benefited from that- everybody who spent time in that group?

Dr. Weisner: Hell yes! Then there would be those that we must agree to disagree. But I've been there for over 20 years, people. It had an influence and, you know, that was also intentional.

‍