Abstracts and links for Oral Papers, 2021
Stream A: Advancing mixed methods
Doing better, doing more: Provocations for mixed methods research in the 21st century
Katrina McChesney, University of Waikato – Tauranga, k.mcchesney@waikato.ac.nz
Winner of MAXQDA prize for best paper in Stream A
Mixed methods research is more than simply adding “any other comments?” on the end of a quantitative survey or running two separate stages or studies (one qualitative and one quantitative) on a common topic. While these strategies may have their place, advances in mixed methods theory in the 21st century invite us to do better and to do more. We have new understandings of the power and potential of mixed methods research, new insights into what high-quality mixed methods research really looks like, and a well-established field of mixed methods research theory. It is important that our practice as mixed methods researchers – including not only our own research projects but also our work as supervisors, examiners, or reviewers of mixed methods research – is not static but remains informed by current developments in the field.
This presentation highlights some current thinking about the practice and potential of mixed methods research. The presentation centres on six key provocations. The first three provocations relate to mixed methods’ potential role in addressing “wicked problems”, in theory generation, and in culturally responsive research. The next two provocations highlight important considerations for doing mixed methods research well: ensuring paradigmatic coherence and robust qualitative-quantitative integration. The final provocation considers the scope for researchers at all levels, including postgraduate students, to contribute to further advancing our understanding of what it means to “do more and do better” with mixed methods research.
The presentation is aimed at researchers at all levels, including postgraduate students and those new to mixed methods research. Examples and useful resources will be noted in relation to each of the six provocations, equipping attendees to grow their learning further in areas of interest. Overall, the aim of the presentation is to support attendees to engage with some current thinking around mixed methods research and consider how this might be of use for “doing better and doing more” in mixed methods research.
McChesney - extended outline McChesney - link to presentation
Researching catechetical ministry (CM): Implementation strategies, challenges, and learnings from a mixed methods study amidst the COVID-19 pandemic
Clarence M. Batan, Tisha Isabelle M. de Vergara, Florence C. Navidad, Keith Aaron T. Joven, Vincent Reuben E. Valientes, Ruth DL. Andaya
University of Santo Tomas (UST) Research Center for Social Sciences and Education (RCSSED)
Corresponding author: Prof. Clarence M. Batan, PhD, cmbatan@ust.edu.ph
This study examines the process of using a mixed-methods design in researching the state and dynamics of catechetical ministry in the Philippines amidst the COVID-19 pandemic. It interrogates a) the research contexts of why mixed methods best fit in doing a national study; b) the utilization of research technologies; and c) the salience of mixed methods in implementing pastoral and intervention initiatives towards effective catechetical ministry. This study argues that while the COVID-19 pandemic affected the original research plan of this study, the mixed methods approach was found to have a sense of “resilience” in threading alternative paths in recovering data through the use of research methodologies and available online strategies. In particular, as survey, interview, archival, and visual data were altogether designed to be collated using provided “smartphones” with programs (for example, Kobo Toolbox, Cam Scanner) and applications (for example, WPS Office, Recorder, Camera, Notes, Albums), the retrieval process although delayed by the COVID quarantine restrictions, was made possible by transitioning into online-based research. With the help of the locally-trained, Katekistang Kalakbay sa Pananaliksik (KKP) (catechists research assistants), all data were cautiously organized by the KKPs through the online mentoring of Manila-based National Catechetical Study (NCS) researchers. As such, a relatively rich amount of mixed methods data in smartphones were sent through the Internet, using both emails and G-Drives, before being sent through couriers from various islands in the Philippines. This allowed accessible shared online data management, analysis, and validation between the researchers and respondents. In so doing, mixed methods data analyses provided sound bases in the formulation and implementation of Research-based Intervention Outcomes (RIO) aimed at improving the delivery and pedagogy of catechesis in the Philippine Catholic Church.
Batan et al - extended outline Batan et al - link to presentation
Teaching mixed methods research: Effective course design elements and impactful teaching strategies
Su-I Hou, su-i.hou@ucf.edu
(a) School of Global Health Management & Informatics, University of Central Florida (UCF), Orlando, FL, USA; (b) Doctoral Program in Public Affairs, College of Community Innovation & Education, UCF.
There is a critical need to conduct empirical studies on teaching mixed methods research (MMR). This study used a mixed methods process evaluation to identify effective design elements and impactful teaching approaches among five cohort groups of students (2016-2020) enrolled in a doctoral MMR course. Process evaluation data were collected via mid- and end-course feedback with both qualitative and quantitative measures. Visual joint-displays were created for the integrated findings analyses. Top five most frequently mentioned key course design elements were after-class assignments (19%), followed by instructor-design-environment-pace (17%), the overall MMR team project (15%), assigned readings (13%), and class lectures (10%). Four most impactful teaching strategies identified were: 1) in-class discussions and activities (29%), which incorporated three course design elements: class lecture, discussion, and case studies; (2) outside class homework assignments and readings (31%), which incorporated three design elements: assigned readings, homework, and article review; (3) team project hands-on applications (29%), which incorporated three design elements: progress report, peer review, and overall team project; and (4) instructor effects (11%), which the student emphasized related to the instructor’s lecture guidance and project consultations. The current study provides practical guidance on effective teaching MMR.
Hou - extended outline Hou - link to presentation
Stream B: Using mixed methods for intervention and evaluation
Application of mixed methods research in the onshore oil and gas drilling environment
Mark Stacey, Dev Raj Paudyal, Peter Gibbings; University of Southern Queensland, Australia
Corresponding author: Mark Stacey, Mark.Stacey@usq.edu.au
Winner of MAXQDA prize for best paper in Stream B
There is significant variance in safety related behaviours between co-workers in a multinational oil and gas drilling operations functioning within South East Asia. Manifestations of this issue within this process driven high risk workplace included extensive variation in safe work performance stemming from individual and collective preparedness to engage in risk taking (Russo 2015).
Understanding the identified phenomenon involved the application of a sequential exploratory mixed method study incorporating both a qualitative element in the form of a semi-structured interview, and a quantitative element involving a widely distributed survey within the broader onshore oil and gas industry. Analysis of variances in risk tolerance achieved through the conduct of both methods, permitted an in depth exploration of antecedents, variables and factors and their manifestation in desirable or undesirable workplace behaviours.
Because this research is concerned with longitudinal influences that on safe work performance within onshore oil and gas drilling operations, it is principally concerned with factors involving human behaviour. A multi-faceted data collection and analysis approach was supported by the selection of a pragmatic constructivism paradigm. Mixed methods are useful for practical minded researchers investigating the experience of a population in order to comprehend the situational realities within a professional environment. A sequential exploratory mixed methods research methodology was found to be appropriate in all of these circumstances.
Stacey - extended outline Stacey - link to presentation
Collective knowledge exchange enhances Australian farmer climate change adaptation: Integrating methodological approaches
David McKenzie, Charles Darwin University, davidkeith.mckenzie@cdu.edu.au
Background: Australian farmers face the wicked problem of landscape protection and sustaining viable farming systems whilst minimizing risks of evolving climate change impacts. Learning and knowledge exchange culture is a quality found in exemplar farmer communities of practice that are strengthening farmer group efficacy and collective adaptive responses.
The purpose of this paper is to explore integration in a multi-phase mixed methods grounded theory (MM-GT) study. The study investigates knowledge making and functioning processes in a NSW Landcare farmer group, and incorporates an intervention aimed at strengthening group cooperation by developing future-purpose strategies.
Methods that contribute to resolving wicked problems: The three-phase sequential-concurrent design in this MM-GT study incorporates development (QUAL), multi-level expansion (quant + QUAL), and intervention-process oriented (quant-QUAL-quant) phases. GT analysis of 16 farmer interviews with theoretical sampling of positive cases enables criteria building and case exemplar selection. In the expansion (case study) phase, group and participant well-being are assessed with a web-based survey instrument from a purposive sample of group members and a control group selected from the farmer population. Group stakeholder interviews and artifacts data are also collected for GT and statistical analysis. The concurrent intervention (action research) phase incorporates a pre-post workshop web-survey quantitative assessment of group functioning, and two-day workshop. Thirty-five participants are allocated into 3 sub-groups for generating worksheet plans and audio-transcript data of future strategic group purposes. Importantly, collected mixed methods data is integrated and reflected through the research purpose and questions (design phase), sampling processes (data collection), analytical procedures and in the drawing of meta-inferences about efficacy of the farmer group in enabling collective adaptation.
Implications: Farmer groups have efficacy in resolving wicked climate change problems that is strengthened by collective learning, knowledge exchange, and adaptive group functioning. This study contributes to methodology with insights derived from the integration of MM-GT with action research. Farmer groups benefit from research that is co-created and has the potential for giving them meaningful answers for meeting future challenges and strengthening collective cooperation.
McKenzie - extended outline McKenzie - link to presentation
Designing and delivering an online learning program for female entrepreneurs in regional Indonesia using a mixed methods research design
Mandias, G.F., Kinanti, S. A., Karamina, K., Tjokro, S. P., ANGIN (Angel Investment Network Indonesia); Cameron, R., Torrens University Australia, Dhakal, S., University of New England, Burgess, J., Torrens University Australia,
Corresponding author: Prof John Burgess, john.burgess@torrens.edu.au
This paper reports on the exploratory sequential mixed methods research design employed in a funded project titled: Capacity Building of Women Entrepreneurs in Rural Indonesia. The purpose of the research is to design and deliver practical online modules to support the training needs of the female small business owners; to support the sustainability of the businesses; to support mass delivery of the program and to develop an evaluation and ongoing development process to ensure the sustainability of the program. The development of the program was confined to female owned businesses in the provinces of Mandalika Lombok (Lombok Island) and in Labuan Bajo (Flores Island).
An exploratory sequential mixed methods research design was chosen so as to better inform the development of the online modules. This involved multiple methods that are integrated and linked towards realising the objectives of the project across three phases. Within the era of COVID, all research processes were remote, and IT mediated. The methods employed were as follows, in sequential order, with each phase informing the subsequent phase of the research:
1. A stakeholder analysis in the targeted regions to identify the population of potential participants in the project.
2. A survey of female led small businesses to establish the demographic profiles of owners; their education levels; their training needs and their access to online training infrastructure.
3. Engagement with key stakeholders who are responsible for delivering the training needs of female entrepreneurs in regional Indonesia. This phase of the research involved focus groups that are guided by open ended questions that are designed to identify the training needs of the female entrepreneurs.
4. The development of training modules to address the specific training needs of the female business owners. This involves the participation of consultants in curriculum development and online training delivery.
5. The delivery of the training modules online within a region of Eastern Indonesia.
6. The evaluation of the training modules effectiveness to inform further development and roll-out of the program.
In this paper the justification, sequencing, the relationship between the mixed methods research process is discussed.
Acknowledgement: Capacity Building of Women Entrepreneurs in Rural Indonesia’ is supported by the Australian Government through the Australia-Indonesia Institute of the Department of Foreign Affairs and Trade.
Mandias...Burgess - extended outline Mandias et al (ANGIN) - link to presentation