Mixed Methods in Disease Prevention and Health Promotion Research (MtG)


good afternoon my name is Dr. Ranell Myles and I want to welcome you to the NIH Office of Disease Prevention Medicine: Mind the Gap Webinar Series this webinar series explores research design measurement intervention data analysis and other methods of interest to prevention science our goal is to engage the prevention research community and thought-provoking discussions to promote the use of the best available methods in prevention research and to support the development of better method before I begin I have some housekeeping items to submit questions during the webinar there are two options first you may submit questions via WebEx by clicking on the question mark in the WebEx toolbar and direct questions to all as all panelists second you may participate by Twitter and submit questions using the hashtag in #NIHMtG at the conclusion of today’s talk we will open the floor to questions that have been submitted via WebEx and Twitter lastly we would appreciate your feedback about today’s weather upon closing the WebEx meeting you will be directed to a website to conceal attending our evaluation and also send an email with a link to the evaluation your insights will help us to continue to improve this webinar series at this time I’d like to turn things over to Dr. David M. Murray Associate Director for Prevention and Director of the Office of Disease Prevention thank you Ranell it’s my pleasure to introduce today’s speaker Dr. Leonard Jason was currently a professor of psychology the director of the center for community research at DePaul University Dr. Jason is a former president of the division of community psychology at the American Psychological Association he has served as a vice president of the International Association Chronic Fatigue syndrome Myalgic Encephalomyelitis and as the chairperson of the research subcommittee of the U.S. Chronic Fatigue Syndrome Advisory Committee Dr. Jason has edited written 27 books has published over 750 articles and chapters is served on the editorial boards of 10 psychological journals in addition to serving as review committees of the National Institutes of Health in 2013 Dr. Jason was presented with DePaul University College of Science and Health’s excellence in research award in 2015 he was presented the American Psychological Associations Award for distinguished professional contributions to applied research it’s my pleasure to welcome Dr. Leonard Jason and turn the session over to you thank you very much for this invitation to present mixed methods and disease prevention health promotion research this lecture today draws on material that was in a book that Dave Glenwick and I edited in 2016 called the handbook of methodological approaches to community-based research qualitative quantitative and mixed methods Oxford University Press was the publisher in this handbook Dave and I wanted to contribute something about the community oriented approach which utilizes a wide variety of content contemporary qualitative quantitative and mixed methods approaches that were theoretically sound and perfectly valid and creative we wanted to address an oppression and innovative manner questions of input for communities in which they work first let’s talk about definitions mixed methods approaches combine qualitative and quantitative methods within the same study or project quantitative research often is based on positivism or the belief in a single reality accessible through scientific procedures in contrast qualitative studies are grounded and constructivist erudite – rather than a single universally shared reality each participant has his or her own reality let’s first talk about qualitative approaches and these there’s an emphasis on the meaning of the phenomena under consideration to those who are experiencing it data which typically consists of words provides the description of the participants experiences there is an active collaboration between the researchers and the participants throughout the research intervention process and some examples of these types of qualitative methods are participant observation qualitative interviews focus groups and case studies now quantitative approaches have an emphasis on trying to establish cause and effect relationships they typically consist of numbers and they’re obtained by use of standardized measures in this approach they try to produce generalizable findings as opposed to qualitative approaches which focus on specific contexts as examples of quantitative methods we have quantitative description randomized field experiments non-equivalent comparison group designs and interrupted time series designs in one of the chapters of review mixed methods in the handbook that Dave and I edited Valerie Anderson mentioned some of the limitations of research methods and what she said was that mixed method studies which used both quantitative and qualitative methods can offset each other’s strengths and weaknesses in a sense mixed methods can provide a more nuanced understanding of research questions then a single method can accomplish now there are different types of mixed methods approaches often one is hierarchical so one method is usually dominant or more central the other is more secondary or supportive but that doesn’t always occur sometimes you have both approaches are in a sense equivalent also mixed methods may also be sequential by first using an exploratory method for discovery and later using a confirmatory method for justification but they can also be simultaneous they can be curring at the same time as opposed to one occurring and then the second one occurring now there are actually many different types of mixed method designs and many different ways of thinking about them Morse developed one of the first type typologies and I’m going to talk a little bit about that one the dominant method is usually represented by a cantilever letters like quantitative and qualitative and the complementary method were the what’s called secondary method is represented using on lower case letters like quantum equality usually there’s an arrow used to denote a sequential design and a plus sign is used to connote a concurrent design here you see nine different combinations equivalent simultaneous designs which is qualitative as quantitative equivalent sequential designs quantitative and then followed by qualitative or qualitative followed by quantitative and you can also see dominant dominant simultaneous designs where the primary method is quantitative secondary qualitative or the primary qualitative secondary quantitative and you also have dominant sequential designs and you can see some examples of that at the bottom these are just a few of the examples and there can be circuitous routes well so what can actually have more than two different approaches in a particular study or area Wiggins has outlined different ways of despite the mixing and it could be triangulating which allows for converging findings to use multiple methods the increase to studies validity it can be demarcation referring to how methods are related quantitative as a dominant and qualitative as a secondary for example and reclassification referring to how both qualitative and quantitative methods can be used in exploratory and confirmatory ways campbell and book chapter published in 2012 talked about so realistic studies which were also known as mixed methods mixed methods research can occur within the combined confines of a single study or can exist at multiple levels let’s explode that means in one study a mixed methods approach can incorporate both quantitative and qualitative data collection and analysis in contrast a pluralistic research program can reflect pluralism across a program of studies conducted by the same team on one topic finally pluralistic research disciplines are fields of science through scholarship in which a balance of study designs analysis techniques is used in publications for example community psychology pluralism also can accrue occur across programs of studies conducted by a team and I’m going to give today two examples of two different types of research and how these types of mixed methods we use first I’m going to talk about the DePaul University team that has been studying recovery homes called Oxford houses for over 25 years real briefly on street houses have no professional staff in our completely self-run they’re the largest self-help recovery residential program in the u.s. is over 2000 homes and a person can live in these homes as long as they want but they need to stay accident pay their share of rent and follow house rules so we developed a collaboration with this organization with a phone call to the founder probably and for over a year we had conversations building trust between the community organization Oxford house and the progress urge group the Oxford House representative was eventually sent to Illinois to set up group of recovery homes in the early 1990s and for the first year we collected no formal data but spent time attending Oxford House business meetings and building up a relationship for mutual trust so in our early steps our Oxford House partners attended our research meetings and tape-recorded them so that their members could learn what we were thinking about doing and so that everyone can have input into this process that they wish we did not just start collecting data we focused on building a trusting relationship with our community partners we determined together what questions should be addressed what kind of data to be collected as you collect the data so our first study was really a mixed study it used the equivalent simultaneous design which used both qualitative and quantitative methods the qualitative methods were really trying to understand why did people want to live in these Oxford houses why did they join them and we really found there was a place that was safe it was a place that people can have fellowship with others but we also had some quantitative questions we wanted to find out the sociodemographics of these houses which turned out to be very comparable to other treatment types settings and also we found that the longer people lived in these houses it seemed like the better outcomes in terms of abstinence so those were some more quantitative methods that we approached in our first study of this particular organization during the 1990s our research groups submitted a number of grant proposals to collect longitudinal data from the Oxford House members but NIH reviewers asked us to conduct a randomized study to determine whether or not these homes could lead to reductions in substance abuse we inform the grant reviewers that a randomized design was not feasible as each Oxford House voted on whether to allow new members to live in their house so we finally approached a founder of Oxford House Paul Molloy with this predicament and he said he would work with us toward a randomized study so the willingness of Oxford House leadership was likely a direct result of years of mutual trust building using both quantitative and qualitative methods this mixed method approach really throwing us closer to this particular organization and allow them to want to help us with the research so we did finally get funding from NIH in a in a quantitative study it was a two-year longitudinal study founded by funded by the National Institute of Alcoholism and Alcohol Abuse included 150 participants recruited from a variety of inpatient treatment centers throughout Chicago and participants were randomly assigned to either an Oxford House or usual after care after finishing treatment those in the Oxford House condition over a two-year period has significantly less substance use higher employment lower criminal justice related outcomes and these are examples of two years later the Oxford house at about 69% abstinent versus 35% for the usual care control incarceration rates were significantly lower in the Oxford House and monthly income were significantly higher in the Oxford House two years later so over the next number of years we instituted a multitude of quantitative qualitative and mixed methods designs and we’ve actually published in the Journal of Community Psychology first author is May a summary of this research and here’s again some examples of some of the work that we did and there is a prostitue of research on the role of natural mentoring relationships and substance abuse recovery so our first quantitative study was really gave us a glimpse that something else was going on we found that if you had a best friend in the Oxford House that was the best predictor of long-term outcomes if you could just find one person who you could find to be a friend if you stayed in the Oxford House longer and you had better outcomes a couple years later this led to a series of equivalent simultaneous and ultimately sequential studies which I’d like to report on briefly Mahler for example use focus groups paired with the series of empirically validated quantitative measures to better understand natural mentoring relationships in the qualitative section of the study we had focus group questions were created with the input of Oxford House residents and the nature of these mentoring relationships was described during focus groups as an example of the richness of this process one male participant explained I usually go to any of the guys that’s why I put five of them on there on the survey regarding the number of mentors in the house I didn’t just put one because we all have different personalities so there’s a certain thing that you can speak to certain individuals that would help you out and lead you the right way the quantitative findings resulted showing that the mentoring characteristics and activities actually predicted social support and helping behaviors so the use of both quantitative and qualitative methods in this study provide an enriched and fearful support for the characteristics of natural mentoring relationships and their importance for our pronounced read presidents ultimately John Light sociologists the Oregon Research Institute and Abe Dukan a person at Ohio State began working with us as well I’m trying to understand some of the dynamics using more quantitative methods of looking at for example trust in Oxford houses over time and we found that trust relationships tended to be a bit more symmetrical which this graph shows but competent relationships the element of role specialization where someone’s a listener it not generally confide in the other person we didn’t see that type of reciprocal nature as we did with trust so in a sense the methods and results of we’re not treehouse study often shed light on the results of another study or inspired to design the new study so the dominant sequential from quantitative to qualitative to quantitative actually has occurred in a number of our studies so prefer while conducting a nationwide longitudinal study of an Oxford house through qualitative final findings and discussions with members of the Oxford house organization we found that Latinos were underrepresented on Oxford House residents this awareness stirred curiosity about Latinos perceptions of Oxford houses and recovery and potential barriers that Latinos encounter when entering Oxford House to address these questions us as researchers helped with the Oxford House Organization and we conducted a series of qualitative studies that revealed that Latinos had a positive recovery experience in Oxford House and that more Latinos could potentially benefit from participating so the quantitative study was really the quantitative study was an NIH grant where we actually did the findings of the qualitative studies helped us basically do a quantitative study which was a Kwanza experiment in which Latino residents of traditional recovery homes or compared to Latino residents of culturally modified homes with funding provided again by NIH so and I would like now switch to another area which is chronic fatigue syndrome CFS or ME Myalgic Encephalomyelitis which as some of you might know is as debilitating as type-2 diabetes congestive heart failure MS or end-stage renal disease in real briefly stigma over the past three decades there’s a series of key decisions that were made concerning methods for gathering prevalence data the name and treatment approaches for these illnesses most of these decisions are formulated within a societal and political context which since illness was assumed to be psychologically generated problem physicians have also regarded fatigue is one of the least important presenting symptoms mixed methods were especially suitable for research in pursuit of social justice for merchants marginalized populations like those with envy and CFS because it’s more sensitive to context it’s more likely to uncover power differentials and to give voice to participants in such controversial community context the employment of a purposeful multi-phase mixed methods program coupled with the values of community psychology can be a tool especially suited to provide patients health care workers scientists and government officials less stigmatizing ways of understanding this illness so let me give an example of some work that we did in this mecfs area the CDC in the late 1980s had conducted studies primarily with ME and CFS and they found that it was a relatively rare disorder primarily affecting Caucasian women we listened to patient groups to challenge these findings in our group launched a more community-based study rather than physician based study that randomly selected individuals from the community and then brought them in for work ups our estimates suggested that 20,000 people were affected and that’s what the CDC had suggested and rather than 20,000 our findings in 1999 suggest the death it was closer to a million so we use sequential nested sampling procedure so for that particular study which was done from 1995 to 1997 10 years later we basically did an equivalent simultaneous design where we use both quantitative and qualitative methods we wanted to find out 10 years later after finding the prevalence rates of 0.42 which suggested that there were a million people with this illness we wanted to then look 10 years later those who developed this illness over this period of time and those whose diagnosis remitted over this past decade so our quantitative outcomes involved recontacting the participants and giving them a complete medical and psychological evaluation our quantitative data suggested that a prevalence rates had remained somewhat comparable over this decade this helped answer an important question we’re illustrates increasing decreasing your statement saying our qualitative methods were used to provide us with a deeper understanding of the interaction of the multiple systems involved in this chronic illness as long as to hear the voice of the patients a coding system was developed to a grounded theory framework in order to focus on the context dependent and structural processes within the data so qualitative themes the community response to the illness included these types of things from the medical community as well as related networks such as support groups an example many patients mentioned that they experienced negative attitudes such as physician the minimization of their illness furthermore many respondents told their obstacles to securing disability in a climate where the illness encounters disbelief also there was the theme of identifying the community response to the illness was an essential component understanding systems involved with the experiences of its sufferers so what are some benefits in this MECSF arena of using mixed methods approaches the qualitative methods allowed us to better understand the experience of the patients who had been identified in the prior epidemiological study now we had a 10 year follow-up it provided us an opportunity to look at the connection of illness to the distribution of power and privilege within the medical community as well as the social response to chronic illness and healthcare thus we found as Ponterotto suggests that mixed method approaches can provide multiple windows into the lives of the less empowered and historically silenced within our society so what are some of the distinctive benefits well what qualitative methods we use to help us develop better quantitative methods to study epidemiology using community based samples the sample generated was subsequently used to probe qualitative features of the illness experiences of patients quantitative research provided to state on the magnitude of this illness qualitative methods allowed us to better understand the unique challenges in stigma that the patients and experience the families friends and treatment professionals these things these data provided us data that allowed us to provide better appreciation of the magnitude of the effects of the Selma’s on patients we use these data ultimately to estimate the economic cost of this illness to our nation so let’s talk about mixed method approaches in general this approach more accurately represents the illness to family friends and healthcare professionals and thereby as possibility of improving services to those effected these types of integrated and value-based approaches are useful for the purpose of advocacy research that works toward the understanding and empowerment of other similar groups who may experience negative social perceptions and dismissive treatment but there are certain challenges to doing these types of mixed method approaches and we’d like to kind of mention just some of them first thing there is no definitive guidelines for how to conduct a mixed method study also integrating multiple data sets is a complex task especially when they come from different methodological traditions there’s also a lack of in-depth training by scholars in both methodologies often people are trained in qualitative methods other people are trained in quantitative methods but very frequently people don’t get training in both of these methods finally it’s challenging and publishing doing two-page and word limits in journals often when you present the mixed method approach it could be a little bit longer in page level and as we know many journals prefer having you send papers at our 25 to 30 pages long and sometimes it becomes a problem with mixed methods approaches but ultimately there are many benefits of this type of work mixing methods can enhance the validity or trustworthiness of inferences and assertions by providing mutual confirmation of bindings this occurred both in our Oxford Health Studies as well as in our ME and CFS studies mixed method designs can provide deeper exploration of causal mechanisms interpretation of variables and contextual factors that may mediate or moderate the topic of study and again we saw this as a quantitative study found that friendship patterns were important and that led to qualitative work with focus groups to try to understand what was it about friendship patterns in these recovery homes that made a difference in those studies ultimately in a sequential way led us to working collaboratively with people from different disciplines on social networks research so again you can sort of see a program of research borrowing from quantitative the qualitative in our Oxford Health Studies mixed methods research can facilitate the development of culturally appropriate instruments and foster a deep understanding of the phenomena of interest clearly our epidemiology study help us understand the magnitude and the distribution of this illness amount of community-based sample is unbiased by health seeking behavior but they really understand some of the themes and the issues of the individuals who have this illness the qualitative interviews gave us themes that resonated with us so that we could understand the experience of the illness not just the magnitude of the disability of those individuals were experiencing in conclusion quantitative procedures have the most power to appeal to collaborators in funding and policy so quantitative procedures seem to certainly be ones that different funding agencies seem to be a little bit more interested in and often policy individuals like to have the hard numbers of saying yes this is the economic costs of this particular illness or yes for the Knoxville house these are the benefits over a usual aftercare that you can have a randomized study and you can basically find for example that these recovery homes actually do make a difference over those who get released from treatment for substance use disorders on the other hand qualitative studies are more likely to empower community members and gain insights into how to identify and work with participants so in a sense the qualitative studies really are attempts to really get to appreciate the characteristics and the feelings of the people that are involved in these projects and I can just say that by going to Oxford House meetings and going to Oxford House conventions and by sitting down and talking with individuals and getting the individuals from the Oxford House movement to tell us what they think are important to study and to involve them in thinking about how to study the topics we actually end up with richer types of research that ultimately could lead to different types of funding sources as was indicated in the opportunity health studies as noted by myself and Dave Glenwick work in a 2012 publication another book on research methods formative mixed-method studies can be instrumental in learning how to access and develop trusting relationships with different sectors of the community combining these methods can be most effective when undertaking community based issues and again a lot of the studies that I’ve been talking about during this presentation involve very close relationships with community members this is participant community research this is very much within the framework of community psychology I am a clinical community psychologist so that’s the type work that I do these are some of the references that I refer to in the presentation and of course certainly they’re available for follow-up during a brief 35 minute presentation I’m not able to go into lots of the details of these methods but for those of you who have interests I certainly would encourage you to look at some of the journals that are mentioned here as well as a couple of books that have come out that give much more details I might add that the handbook of methodological approaches to community based research as over 35 chapters in it and the chapters are both on quantitative qualitative methods as well as mixed methods and they’re all give examples of how they were used by researchers not only in the United States but in other countries okay at this point you can submit your questions by WebEx or Twitter and I will be available to answer your questions and once again I want to thank David for the invitation and I want to thank you all for listening on to me over the last stop few minutes thank you Dr. Jason very much I enjoyed your presentation we have some questions and others will come in as we participate the discussion so let’s just start by asking some of the questions that we’ve received so far I you spoke a number of times during your presentation about challenges in getting projects approved or grants funded in NIH Dr. Myles and I are sitting here at NIH today hosting this webinar so we’re always sensitive to those issues do you have advice for others in the audience who are thinking about putting the grant proposal together it would use both qualitative methods and quantitative methods are there lessons that you have learned in your years of approaches to avoid or to use that you think may increase the likelihood of success in the funding process that’s the general question for example have you found that reviewers expect that a good bit of the qualitative work should already have been done and be described in preliminary studies and then the grant proposal may build on that but the key feature of the grant proposal itself is a quantitative investigation or have you had success in getting support to do what is largely qualitative work in an IH grant proposal that might ultimately lead to quantitative work but the quantitative isn’t actually included in that application so let me pause and see what comments you have thank you data that’s an excellent question and I do agree with you that I do think having some preliminary data potentially qualitative or quantitative certainly helps with or grant proposals and please show that one has kind of some familiarity and comfort going from both qualitative and quantitative I think can actually be a strength of a research proposal one thing that you might want to consider as you’re thinking about using a mixed method approach and I do think that mixed method approaches can be funded is that there might be important issue of talking to the program official at NIH and basically finding out their comfort level with mixed methods and if your project officer and NIH feels that it’s very is something that they’re interested in then you might want to look at the review permit the review committee you actually have some influence instead of getting your proposal sent to because you can listen which review committees you think would be appropriate or your grant proposal to go to so you might go and actually look at the review committee members which is listed on the web and you can see who the people who are reviewing the grants do mixed-methods work so for example if you have a plan that sent to a particular review committee and all the people in the review review committee you just quantitative or you through just qualitative research well that will probably tell we do something about the particular review committee you might want to have a conversation with the SRO about whether it’s possible for having someone that’s maybe more of a mixed method researcher that’s involved in it but certainly I would start with going to the project officer because ultimately they’re the ones who would be funding you and asking them about what’s the possibility of having a mixed method approach now sometimes there is an RFA that might be focusing on saying yeah we’d like to have both approaches Iten that if you have two equivalents aside in other words they’re both equally important you might have more difficulties in the space to actually present all the information you need to justify them both so that if you clicks we put a proposal that’s dominant with one approach whether it’s qualitative or quantitative and you have more time to really rushing to the details and then for example having a piece of it that’s maybe for example qualitative then I think they can get their flavor of one approach in a little bit more detail remember you only have 12 pages to really make your argument that’s not a lot of space so I do think that probably focusing on one approach really going into detail going through all the methods and then maybe having a piece on at the end and secondary that’s a different approach is actually something that I have done in several grant proposals successfully and I think the reviewers and actually appreciated the fact that I had some flexibility and I could for example say once grading was a quantitative study where we really were trying to in a sense understand the epidemiology the distribution of cases of Emmy and CFS and in urban areas Chicago but we were also interested in finding out people’s experiences and handless illness that could generate other potential hypotheses in the future so that we could do some qualitative work with some of the individuals that we identified as having a silmas I think that that was something that was a beneficial to the reviewers who thought that that was innovative so when you’re writing a proposal for NIH you always have to think about what is innovator what is differentiate you from other people because that’s an important piece the significance of the project’s important certainly but innovative section of it is very important and if you can show that you’re using mixed methods and very creative ways that could get people excited and give you a fundable score alright thank you a follow-up question that’s very much related so in many areas it’s difficult for the p i– the principal investigator to be an expert in all of the issues involved in the grant application and as an epidemiologist I’m used to working with teams where we have people that have skills in different areas that are helping with a proposal together that seems even more likely to be important in a project that’s using mixed methods where one person may be an expert on quantitative methods but not qualitative or vice-versa have you found that you need to have a team approach where you can have individuals with expertise in these various areas but you’ve been doing this yourself a long time thinking about all the other people out there that are thinking about doing that they have strengths in one area is it better for them to try to become an expert in everything where is it better for them to partner with people who are already experts in the areas that they don’t cover again thank you David another excellent question yes I do think that it is important to partner with different people with different expertise so if you have strengths and quantitative methods you might have some familiarity with qualitative methods but that might not be your script so to the extent that you can bring onto your team someone who has similiarity with that an area that really improves your team and I might add that you know when we do research at the Center for community research at DePaul University we’re constantly looking for the best of people who have topic areas so for example we reach out to an economist to help us work on economic issues we work out with a sociologist who basically understands networks we work with the simulator who helps us understand simulations as we develop models so and we also then reach out for people who have some of the more qualitative experiences of understanding and of the individuals experiences so so first ed so yes in our oxford house research right now we do have multiple disciplines and actually reaching out to people is easier than you think because there to give a topic that’s grabbing and they say this is something that hasn’t been done that is really interesting come together and let’s join ourselves with the team they actually don’t often you know you don’t have to have money proven funded to get a team going if you have money that’s great if you have something that you can entice them to work with you but they might just be interested in the topic or in the collaboration of the multi disciplinary group that actually will get them excited now there are some risks involved in doing the site work too so for example I’ve seen collaborations where basically they’ve brought on two or three different people in an area so let’s say qualitative and there are many different qualitative methods and I’ve seen for example where you had three people who basically leave in three different types of qualitative methods and ultimately there was some difficulties and trying to figure out how to analyze the data because each of them had their preference on a particular way so I think early on trying to set the boundaries trying to find out the comfort level with the collaborators trying to basically see how you’re thinking about approaching this from which particular methods just like with the complication when you’re going to write something up you should really have discussions about authorship very early on and I take the same applies here and when you bring a multidisciplinary group together there should be early the sky about what the roles are and how different strategies and tactics and techniques and methods will be used and then when the data actually comes in where the grant is written and the funding occurs you really can try to minimize some of the potential negative sites of miscommunications and conflict that can occur if you haven’t had those discussions really early on thank you thank you you made a very good suggestion earlier about contacting a project officer at NIH early on as you’re considering a grant application idea we’ve always endorsed that kind of advice very important to speak to someone at the agency that might fund the project early on described the project describe the rationale describe the methods if feedback project officers can be very helpful they can also offer advice on what would be a good study section to review the application and you mentioned that you try to look at the membership of potential study sections and make your own judgment about whether it would be put a group to review your project I I want to ask a question related to that aspect sometimes the the program officer is going to steer you in one direction you look at the panel and say well they’re all in this direction or they’re all in that direction they’re missing certain kinds of expertise that would be important for my project and you’ve indicated that sometimes you contact an SRO but to suggest that depending on the timing it could be too late for those suggestions to be useful if you’ve already submitted your application for example the SROs may have already pretty well booked the panel and it’s too late to act on anything have you tried suggesting reviewers that that might complement the existing membership when you submit your application is that a strategy that you take okay this is a very complicated question because um basically of scientific we do officer who oversees the review process you cannot basically ask them to put a particular member on that review committee if you mention a person’s name whether it’s a qualitative or quantitative researcher they will not then use that person so you can’t basically indicate a name now you can say to a scientific review officer who oversees the review process you have a great review team you’ve got the topic comfort substance use or chronic illness but I’m doing some stuff that’s mixed methods and is it possible to find someone maybe as an ad hoc reviewer to sort of compliment your only rich group of reviewers that’s something that’s appropriate to do you can also talk to your project officer even before then and basically say listen I have a proposal that I have that basically will basically be using both approaches what do you think of it what types of review groups do you think might be interested in it remember those project officers often sit in on review committee meetings so as they sit in those meetings they basically get a chance of sort of what types of reviewers are there so they can give you great information so yes there are ways of influencing the process if it’s a special emphasis panel usually you’re not going to know who’s going to be on it until very right right before the panel maybe 30 days before but if it’s a regular review committee that all the members are listed and they all have like four-year terms so you can actually see who’s on the review committee and you can read research that and you can actually do and we do that we do literature searches of the review committee members and we find out what types of work not just the topic but also their style and and how they feel about particular research methods and if you don’t feel that there’s any competence on that review committee to basically understand your proposal then you probably don’t want to kind of target that review committee to submit it to thank you we have a few questions you want to get to so one person asks did you publish your studies as a single mixed method study or separate qualitative and quantitative research day so yeah so that’s a really good question and this again comes up to one of the barriers that occurs with page page limitations we have one study that were were one of my colleagues is doing in Norway and it’s a qualitative study from people who got Giardia and Bella Bergen and what happens to them a few years later and in that study if you know again because it was qualitative we couldn’t even put it in one study there’s actually three studies that are coming out taking three different aspects of it and we have another study going on now about recovery and and we’ve interviewed physicians about how they define recovery for mecfs we’ve interviewed patients and in qualitative studies you know to really capture what’s going on we actually have had to divide them up into several papers so in our epidemiology study or longitudinal study and I mentioned we publish some of our results in quantitative journals and some of our results and qualitative journals and the reason that we did this was because to get through the you know to get that 25 to 30 pages was just very difficult to basically do one and do them both together so I would say we often split them apart but not always sometimes we have basically had a journal that would give us the space and we were just very careful about how much time we put describing the methodology of the quantitative and the qualitative so that we could get the richness of both the data quantitative and qualitative so I would say that often we do split them apart because of space issues with journals if you have a monograph or a book certainly you’ve got you know more ability to put them together but there are some journals that with are very interested in multi method research and you can submit your articles to those journals and you just have to be a little bit careful with how much space you have to describe the different methods and as well as the results okay um the next person writes I teach occupational therapy master’s level students and mentor research how do you recommend better preparing them to embrace and mixed-methods approach well you know I do think if it’s possible if you have a course in quantitative methods which is usually occurs in most of these schools but maybe having the course in quantitative and qualitative methods so if you can have courses in both areas or at least one course in both areas that’s really important and then even better if you can actually have a course in mixed methods as maybe an advanced seminar where you have a chance to actually review studies that have useful so so I know I recognize that you know lots of colleges and universities and graduate programs don’t have the time and space to have all these types of methodology courses but I would at minimum recognize for competence and quantitative confidence and qualitative so at least one gets some experience in graduate programs in both areas and then if you have the luxury of being able to have a course in mixed methods that’s wonderful and you can also tell your students that they can potentially for example attend workshops this type of webinar or even kind of readings and again there’s a number of books out there besides my own and Dave gran works that you know cover these areas so individuals who have particular interest and I must admit lots of students are really interested in getting confidence in both areas so I find that the students at DePaul in our clinical community and our community program once they do quantitative methods they want to then experience qualitative methods and sometimes they want to actually do both which is really exciting so I find having some exposure to both areas is really very important even though one might be dominantly in one area one can certainly have a secondary area that one has experience with enough experience so that when one wants to do collaborative research one can then bring in an expert in that area that maybe a person is not that strong can you quickly provide the reference for Tasha Cory at all 2003 oh I can maybe I can do that by email to you oh I’ll make sure it’s posted on our website I’ll get that to you yep okay the next question is I wonder if dr. Jason might comment on multi methods and the use for a multi method versus mixed method approach for example using different methodologies from the same research paradigm like a qualitative content analysis plus phenomenological study yeah I think that’s really I think that’s really interesting that within a particular method often using several different strategies can be extremely helpful to understanding the phenomena and whether it’s qualitative or quantitative there’s so many different rich ways of understanding stuff in a sense Israel just like toolkits these are tools that we use to understand the reality of nature and nature is complicated nature is not all often linear it’s often kind of has feedback loops it’s often like systems that have to be understood and really to understand those systems and the dynamics in those systems we need to have the richest ways of understanding that and I basically think that having the mixed method approach gives you the richness of that dynamic system and we’re multum utley trying to understand gets the gets us beyond simple a to be linear logic which unfortunately still represents lots of the type work that occurs in our field okay and another person writes one of the reasons i can think about why the qualitative research is relatively difficult to get funded is because the results of qualitative research might be difficult to be replicated and verified so how do you deal with this concern yeah I mean certainly um you know vonage people who don’t have training in some of the qualitative methods are kind of concerned about generalizable information but certainly if you understand that different context might draw for different types of phenomena and behaviors and attitudes beliefs that we can basically have a richness of a contextual way of understanding people in relationship to their environments and how that basically at transaction really occurs and I think these types of methods that help us understand these phenomena are really incredibly important and if some people dismiss one method or another method you know I think that’s just a limitation of their own training because the reality each has which is to give us and I think that you know while some quantitative research might be more generalizable qualitative research as it strands to of understanding the depth of the knowledge of what’s going on within that particular setting so together they really provide a story that often isn’t told by one or the other do you ever mix funding streams so that you get the quantitative aspects funded by one source and the qualitative aspects funded by a different source so um that’s an interesting question um I think that you know we primarily have focused on one source of funding to do our research and it would be like a particular grant and we often don’t have like multiple grants supporting a particular area of research but I think that is possible there are some for example foundations that would be particularly more interested in one particular method than the other our group at the Paul generally doesn’t go after a foundation research we just happen to go after more federal sources of support but I think those units that are working with for example counties working with States working with at for-profit organizations working with foundations there’s probably multiple ways local sources of funding that might be particularly more interested maybe in qualitative methods and outcomes than even quantitative so yes I think one can easily have multiple sources of funding at different types of the research to be funded our group generally has not gone that direction but I think some groups do pink well thank you so much for all of your useful information and thank you everyone who participated in today’s webinar on the medicine mind the gap website which is again prevention.nih.gov/mindthegap you will find several resources for this talk including the slides references and a link to complete and evaluation your feedback is very important to us as we plan future sessions thank you again for your time please join us next next month for a session with Dr. Maria Fernandez from the University of Texas School of Public Health about the challenges and opportunities associated with emanation and implementation research thanks again you

Leave a Reply

Your email address will not be published. Required fields are marked *