Ontology, Epistemology, Paradigms,
Methodology and methods
Introduction
Educational research has been conducted in various forms and
settings throughout the decades, with a constant shift between popular research
designs favouring a positivist, constructionist, transformationist or pragmatic
approach to which paradigm, ontology, epistemology, methodology and method is
the ‘true’ method of educational research. The influence and importance of
gender interactions have been widely discussed and debated within a variety of
disciplines, especially in educational contexts over this time, however little
research has been conducted into investigating the effects
of gender interactions between physical education teachers and their
students (Davis& Nicase, 2011; Nicaise, Bois, Fairclough, Amorose &
Cogérino, 2007)
For
the purposes of analysis, two articles addressing the same research area were
chosen; Davis & Nicase (2011): Teacher student interactions : Four case
studies of gender in physical education, with the aim to better understand
gender interactions between teachers and their students in secondary level
physical education utilising a mix of qualitative and quantitative means; and
Nicaise, Bois, Fairclough, Amorose & Cogérino (2007):
Girls'
and boys' perceptions of physical education teachers' feedback: Effects on
performance and psychological responses,
which
analyses the perceptions of physical education teachers’ feedback patterns with
male and female students through quantitative means.
By analysing these articles on gender interactions between
physical education teachers’ and their students I was able to gain insights into
the utilisation of different research methodologies and methods, the importance
of differing ontological and epistemological views regarding teacher practices
and the systemic approach to conducting effective, ethical, valid and creditable
research in educational contexts. This paper begins with the analysis of various
research approaches, ontology, epistemologies and paradigms used in research,
including approaches used in the research samples; discussion surrounding
various methods and methodologies, discussing the ethical considerations and the
influence of power in educational research and concluding with a discussion on
the major outcomes of the sample research processes.
Research
approach
There
is a substantial plethora of differing (however equally important) approaches to
research in social sciences, especially education; however to understand its
development all prospective researchers must consider the overarching concepts
of which ontological, epistemological and paradigm in which they belong to.
Before doing so, understanding of broad approaches needs to be established, with
O’
Toole & Beckett (2010) suggesting 3 approaches to research;
Ontology
forms the basis of all research inquiry as it forms the basis for the
construction of reality or as “claims and assumptions that are made about the
nature of social reality, claims about what exists, what it looks like, what
units make it up and how these units interact with each other.” (Blaikie, 2000,
p. 8, as cited in
Grix,
2002, p. 177) Epistemology is concerned with the theory of knowledge, how it is
obtained, how is it validated and new ways of knowing which are more effective
than other knowledge gathering models and theories (Grix, 2002). Paradigm then
results from the combination of the researchers’ ontological and
epistemological approach which informs the approach to research methodology and
research methods (Grix, 2002)
A
positivist paradigm can be viewed as a ‘scientific method’ approach to
educational research which prioritises a quantitative research approach.
O’Toole
& Beckett (2010) describes positivism as human understanding of behaviours
and learning should be positive, tangible and demonstrable.
Whereas
Darlaston-Jones
(2007) ascertain that‘positivism views reality as universal, objective, and
quantifiable’,suggesting that reality is measurably equally and remains the same
regardless of the individual experiencing it. A
constructionist paradigm however surrounds the idea that humans create their own
realities, scaffolding their learning in the process (O’Toole & Beckett,
2010) Progressing this, Darlaston-Jones (2007, pp.19) suggests that “reality is
shaped by the cultural, historical, political, and social norms”, that being the
accumulation of experiences and their interpretation of experiences into a world
view (which O’Toole & Beckett, 2010 regard as a philosophical
paradigm)
Transformative paradigm was developed the growing
dissatisfaction with the positivist and constructionist/interpretative with
Mertens (2007) suggesting that a transformative paradigm includes a cyclical
model of research which involves participants and researchers in the process
through the establishment of partnerships with power relations and ‘culturally
competent practices’ highlighting the key differences from previously mentioned
paradigms
Finally a pragmatic paradigm introduces the idea that there is
no single system of philosophy or reality (Mackenzie & Knipe, 2006). That
being said, the pragmatic paradigm allows the researcher to utilise data
collection strategies and data collection methods which are more likely to
provide insights into their central body of research questions without
“philosophical loyalty to any alternative paradigms” (Mackenzie & Knipe,
2006, pp. 197)
Progressing on, method and methodology are usually used
interchangeably and has led to a range of different meanings(Mackenzie &
Knipe, 2006) Methodology can be understood as a system for the methodical
collection of data (O’ Toole & Beckett, 2010) which informs the choice of
research methods required to satisfy the question in our area of research. For
example if the researcher desires to utilise an ethnographic methodology (i.e.
studying a bounded population and their experiences) then this informs the types
of methods needed to compile the data (artefacts, semi-structured interviews,
observations etc.). However Grix (2002) takes a different approach, suggesting
that research methodology concerns itself ‘with the logic, potentialities and
limitations of the research methods.’ (pp. 179), with research methods linked to
the research question posed and the data sources involved. Mackenzie & Knipe
(2006, pp.196) differ from these definitions, suggesting that the most common
definitions suggest that methodology is the overall approach to research which
is connected to the choice of paradigm whereas method relates to the systemic
collection and analysis procedures
of data.
Nicaise,
Bois, Fairclough, Amorose & Cogérino (2007) utilise a positivist paradigm
for the purposes of their research, mainly due to the researchers’ sole use of
quantitative questionnaires. Paired with this is the majority of the research in
the instruction and subsequent literature review is based on conclusion from
quantitative research, a large sample size of 333 participants (122 boys - 203
girls – four male teachers –four female teachers) and the decision to conduct
all student questionnaires on one occasion. Nicaise, Bois, Fairclough, Amorose
& Cogérino (2007) ultimately sought to use an inductive (theory building)
logic to complete two objectives;
·
To
assess the effects of student and teacher gender on physical education students’
perceptions of their teachers’ feedback and
time
·
To
determine whether PE students’ perceptions of their teachers’feedback and time
were related to students’ performance and psychosocial
growth.
To
achieve this Nicaise,
Bois, Fairclough, Amorose & Cogérino (2007) utilised a positivist
methodology consisting of 4 separate quantitative questionnaires and one
demographic information sheet, which include;
·
Perceived
Teaching Feedback Questionnaire (PTFQ)
·
Students
perception of teachers’ invested time
·
Perceived
competence, effort-importance and
interest-enjoyment
·
Physical
education performance
The questionnaire sequence was structured for the students to
complete the first three questionnaires with the first 15 minutes of one of
their physical education classes during February 2005, with the physical
education performance questionnaire being conducted in November 2004 and June
2005.
Davis & Nicase (2011) however use deductive reasoning (or
theory driven reasoning) to test if physical education teachers still exhibit
inequitable teaching practices, namely through Schon’s (2000) Theory of
reflective practice which focuses on the lack of physical education teachers’
knowledge on inequitable teaching practices. To achieve this Davis & Nicase
(2011) utilise a pragmatic paradigm, which is evident by their choice of
research question;
·
What
are the self-perceptions of the physical education teachers about their verbal
interactions with male and female students
·
Are
there differences between teachers at rural and urban high schools regarding
practices in relation to gender interactions with
students
·
Are
self-perceptions of each teacher compatible with their practices regarding
gender relations?
This
is paired with use of a qualitative methodology which takes precedence over the
use of quantitative methods to make their research claims and discuss their
findings in relation to other research conducted in gender interactions in
secondary education. Triangulation can be viewed as another indicator of their
pragmatic approach, as the researchers aim to substantiate their claims by
utilising three separate methods for collecting data. Davis
& Nicase (2011) utilise audio taped interviews and video-taped observations
of teacher behaviour, detailed field notes were taken. The four teachers
participated in semi-structured interviews separately before and after the
research process; with individualised follow up questions were used so the
participants could fully discuss their experiences. These interviews were
audio-taped, transcribed and ‘member checked’ to ensure reliability and
validity.
Davis
& Nicase (2011) also kept a research journal throughout the research
process, which serves two purposes; firstly the researchers are able to reflect
on their performance throughout the project, potentially furthering their
careers and knowledge. The second reason for this would be adopting a rigorous
reflexivity approach in their research. Subedi (2006) claims that by doing so
the researcher is being open and accountable for the knowledge created,
therefore challenging the notion of the‘all knowing’ or perfect researcher
(which ties into what constitutes ethical conduct in educational
research)
Data Analysis
Davis & Nicase (2011) analysed their data in two phases; a
quantitative analysis of the observation data to compare expected and observed
interactions and a qualitative ‘cross-case’ analysis of the teacher interviews,
the video-taped observations of teacher behaviour and field notes to investigate
the link between the teachers’ espoused theories and their
actions.
The
first aspect of analysis (Quantitative analysis) consisted of four steps
transcribing and coding 24 video-taped classes, focussing on the interactions
the teachers’had with their male and female students, quantifying the frequency
of instances in verbal categories and then assessing inter-rater reliability
concluded the data analysis. For the purposes of coding, the researchers used a
pre-determined procedure based from Martinek & Mancini’s (1989) dyadic
teacher-student observational tool. Davis & Nicase (2011) developed their
four categories (praise, questioning, informational and criticism) by combining
the existing six categories of
Martinek
& Mancini’s approach (praise, acceptance, asks questions, gives information,
gives direction and criticism), but did not elaborate the rationale behind this.
The lessons were then coded by two experienced researcher-observants with
Krippendorff’s alpha (Hayes & Krippendorff, 2007) which established
inter-rater reliability. According to Davis & Nicase (2011), Krippendorff’s
alpha is based around comparing the expected and observed logic of the
chi-square calculation.
Moving onto the qualitative analysis, each participant was
individually interviewed, video-taped, transcribed and analysed to discover
common themes which emerged in regards to gender interactions with students. The
researchers then met to form a consensus on key themes which emerged from there
analysis of the raw data with a cross-case analysis performed to look for
similarities and differences. To check the reliability of the data, the
researchers performed a‘member check’ of the constructed interpretation of the
interviewees responses, that being through discussion with the participant to
see if the data reflected their experiences (Freeman, deMarrais, Preissle,
Roulston & Pierre 2007); All participants confirmed that the qualitative
data was accurate and correct.
Nicaise, Bois, Fairclough, Amorose & Cogérino (2007)
analysed their data through a 2X2 (Student gender X teacher gender) Multivariate
analysis of covariance (MANCOVA) to view students’ perception around teacher
feedback and how teacher gender moderated the differences. In conjunction with a
MANCOVA the authors explored gender differences in Intrinsic Motivation
Inventory variables with a separate MANCOVA and tested for gender differences in
initial PE performance using an analysis of variance (ANOVA). To finish their
analysis, Nicaise, Bois, Fairclough, Amorose & Cogérino (2007) utilised four
regression analyses to test between the hypothesised connection between PTFQ
variables and students perceived competence, effort, enjoyment and
performance.
However
Likert based questionnaires can be problematic (Oliver, 2000) as Nicaise, Bois,
Fairclough, Amorose & Cogérino utilised three Likert methods in their study.
Tergan
(1998) suggests that evaluation based checklists have three key
weaknesses:
Triangulation
is one way of testing validity;
Triangulation
is a process where data or results are corroborated with two other independent
sources to suggest the research data is actually reliable
(Jick,
1979). O’ Toole & Beckett (2010) however raise a critical argument to this,
suggesting that researchers believe triangulation is too positivistic in
nature, as it requires fixed points of reference. This is due to the‘empherial’
nature of education research, as it is ever fluid and like most aspects of
education, in a constant state of shift
(Sikes,
2006). Considering this, reliability then would need to be called into question,
if educational research is in fact in constant shift, how can we (as
researchers) insure that our research data is valid?
O’ Toole & Beckett (2010) suggest that researchers should
focus on four new key words in educational research; that being plausibility,
credibility, resonance
and transferability. From this the researcher can collect and analyse data
to mount an argument that plausible (i.e. Not easily disproven), creditable in
the sense that other researchers accept the results, has resonance with similar
findings in their field of research and transferable to other contexts outside
of the initial study area.
Ethical considerations
Informed consent is the central idea surrounding research
ethics, with Howe & Moses, (1999) suggesting that it is up to the
participants to weigh the risks of participating in research studies and the
only way for this to occur is for participants to be informed about and
essentially understand the research process. Both articles claimed to have
received written consent from their research participants but no there has
mention of the researchers giving the option for the participants (or guardians
in the case of minors) to withdraw from the study at any time and in turn
require the removal of data. This is a huge ethical concern when considering
conducting research with minors as it can suggest that initial informed consent
is final in the research process.
A
special consideration needs to be made due to the fact that the majority of the
research conducted in both papers was on children and adolescents (less than 18
years of age). This suggests that the participants may be vulnerable or coerced
into participation or disclosing unnecessary details; Morrow & Richards
(1996) suggest that children are essentially vulnerable in two ways, physically
with a lack of knowledge and experience and structurally due to a lack of
political and economic power; both potentially making them vulnerable to adult
influence. Although
Morrow
& Richards (1996) may have been writing exclusively about individuals which
may not be considered adolescent or‘pre-adult’, both research papers suggest the
bulk of participants are aged between 12-16 years of age. Both
research groups however approached the ethical implications of their research
by seeking permission from the institutional review board of their university,
both schools’ research offices and the superintendent of the district. The
researchers then received informed consent of all four teachers, the parents of
students and received assent from the students themselves; also the researchers
sought to provide anonymity for the participants and used pseudonyms for the
teachers and schools involved.
However
It can be assumed that both research teams failed to recognise how and where
data would be stored, what protections are placed on the data to prevent
external individuals ( i.e. outside of the research team) and how and if data
which has been collected will be destroyed. O’ Toole & Beckett (2010, pp.101)
suggest ethical research should entail details of the treatment of participant
data, including storage and possible destruction of materials which are
identifiable to the research participants. The only exception to this was by
Nicaise, Bois, Fairclough, Amorose & Cogérino (2007) who simply stated that
the students were assured that their responses would remain confidential and
their teachers would not see their answers, but still fall short of explaining
how this would be done. This highlights Howe & Moses (1999) concept that
privacy during and even after research is the second most important
consideration for educational researchers in regard to the ethical treatment of
research participants, however ethical treatment of participants is subjective
to the researcher as mentioned earlier with Davis & Nicase (2011) opting to
include a research journal to be critiqued by themselves, peers and potentially
the research participants themselves (Subedi,
2006)
Power in research
Davis & Nicase
(2011) detail in their data collect method that in order to reduce reactivity
and social desirability bias
(Fisher, 1993) whichconcerns participants acting in a way feel is right or
acceptable in response to being studied, the first two days of the 8 day
bi-weekly study was discarded. Researcher bias was also a significant concern
for Davis & Nicase (2011), in response to this analysis of the data was
verified by colleagues for reliability, the written reports of case studies were
shared with their respective participant for verification, maintaining a journal
of the research process, following the guidance of previous researchers and
searching for negative instances. In regards to video recorded observations
Davis& Nicase (2011) utilised trained college students in order to limit the
influence of the researcher figure in the classroom and by taking the
aforementioned approach, the power of the researcher in research is
reduced.
Davis & Nicase
(2011) however fail to detail the setting in which the interviews take place; is
it an open space such as a gymnasium, is it a conference room with a large table
separating them or even is it an intimate staff room with the researcher next to
the participant? Mallozzi (2009) highlights the potential environmental
influence on both the researcher and participant, which influences the
credibility of the responses and interpretations which result. Mallozzi does
this by constructing a scenario based around an interview she was preparing to
conduct, with considerations of armchairs and indoor plants being‘trumped’ by a
12 seated table which dominates the room; this triggers her internal dialog
between the three competing research voices, consisting of Postie
(Post-structuralist), Libby (Feminist) and Ed (post-positivist). Questions the
use of empathy arises when considering that a feminist approach encourages a
construction of empathy and environmental influence around socialisation and
rapport where as a post-positivist would argue for a construction of empathy
around maintaining neutrality.
Nicaise, Bois, Fairclough, Amorose & Cogérino (2007) however
eluded to the students’ completing the questionnaires immediately before
physical education classes, suggesting that the students were situated in a
gymnasium (so a wide open space) with their teacher absent; however the authors
fail to detail the layout of the students in relation to the researchers
themselves, how many researchers were present what was the ratio of male to female
researchers in the setting and were the students clustered together or spread
apart? All factors considered it can only be assumed that at least one
researcher was present at each school for the purposes of the questionnaires
(both for students and teachers), which resonates with the power issues
previously eluded to by Mallozzi (2009)
Conclusion
Research can take on various forms, such as the expression of a
population using figures through quantitative means or constructing in-depth
stories of the individuals lived experience through qualitative case studies, as
shown with the two research articles, different approaches to research design
are impossible to distinguish as the best form of educational research. These
articles successfully articulate how two different paradigms can converge to
articulate the same phenomenon in physical education teachers’
practice.
This has been illustrated by the differing ontological and
epistemological position of each article framing the development of the
individual research methods and methodologies, which were clearly articulated
and grounded in their respective
philosophical literature. Unfortunately research ethics was not discussed during
most of the research papers under analysis leaving the reader to make
assumptions upon what ethical research in education looks like.
Just as importantly this research process has highlighted the
responsibility of the researcher to develop and article what ethical research
entails and the potential influence of power relations on the research process,
especially during data collection and analysis. This is essential for the
development and growth of the beginning researcher in the social sciences as it
allows for the development of awareness of the major influences and shifts in
educational paradigms and research convections.
References
Darlaston-Jones, D. (2007). "Making connections: the
relationship between epistemology and
research methods." The Australian
Community Psychologist Vol.19 No. 1,
pp.19-27
Davis, K & Nicase, V. (2011), Teacher student
interactions : Four case studies of gender in
physical education, Journal of Classroom Interaction, Vol. 46, No.2,
pp.11-23
Fisher,
R. J. (1993). Social desirability bias and the validity of indirect
questioning.
Journal of Consumer Research, pp.303-315.
Freeman, M.,
Preissle, J., Roulston, K., & Pierre, E. A. S. (2007). Standards of
evidence
in qualitative
research: An incitement to discourse. Educational researcher, Vol.36, No1, pp.
25-32.
Hayes, A. F., & Krippendorff, K. (2007). Answering
the call for a standard reliability measure for
coding data. Communication Methods and Measures, Vol.1,
No.1,
pp. 77-89.
Howe, K. & Moses, M. (1999). Ethics in educational research.
Review of Research in
Education, vol. 24, pp. 21-60
Jick,
T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in
action. Administrative science
quarterly, Vol.24, No.4, pp. 602-611.
Mertens, D. M. (2007). Transformative paradigm: Mixed methods
and social justice. Journal
of Mixed Methods Research, Vol.1, No.2, pp.212-225
Morrow, V., & Richards, M. (1996). The ethics of social research with
children: An
overview. Children& society, Vol.10, No.2, pp.
90-105.
Nicaise, V., Bois, J. E., Fairclough, S. J.,
Amorose, A. J., & Cogérino, G. (2007). Girls'
and boys' perceptions of physical education teachers' feedback: Effects
on
performance and psychological responses. Journal Of Sports
Sciences, Vol.25, No.8, pp. 915-926
Oliver, M. (2000) An introduction to the Evaluation of Learning Technology.
Educational
Technology & Society, Vol. 3, No.4, pp.
20-30.
O'Toole, J. & Bennett, D. (2010). Educational research:
Creative Thinking & Doing. South Melbourne: Oxford University Press
Sikes, P. (2006). On dodgy ground? Problematics and ethics in educational
research. International
Journal of Research & Method in Education, 29(1),
105-117.
Subedi, B (2006) Theorizing a ‘halfie’ researcher’s identity in
transnational fieldwork,
International Journal of Qualitative Studies in Education, Vol.19,
No.5,
pp. 573-593
Tergan,
S. O. (1998). Checklists for the evaluation of educational software:
critical review
and prospects. Innovations in education and training international,
Vol.35,
No.1, pp.9-20.
Methodology and methods
Introduction
Educational research has been conducted in various forms and
settings throughout the decades, with a constant shift between popular research
designs favouring a positivist, constructionist, transformationist or pragmatic
approach to which paradigm, ontology, epistemology, methodology and method is
the ‘true’ method of educational research. The influence and importance of
gender interactions have been widely discussed and debated within a variety of
disciplines, especially in educational contexts over this time, however little
research has been conducted into investigating the effects
of gender interactions between physical education teachers and their
students (Davis& Nicase, 2011; Nicaise, Bois, Fairclough, Amorose &
Cogérino, 2007)
For
the purposes of analysis, two articles addressing the same research area were
chosen; Davis & Nicase (2011): Teacher student interactions : Four case
studies of gender in physical education, with the aim to better understand
gender interactions between teachers and their students in secondary level
physical education utilising a mix of qualitative and quantitative means; and
Nicaise, Bois, Fairclough, Amorose & Cogérino (2007):
Girls'
and boys' perceptions of physical education teachers' feedback: Effects on
performance and psychological responses,
which
analyses the perceptions of physical education teachers’ feedback patterns with
male and female students through quantitative means.
By analysing these articles on gender interactions between
physical education teachers’ and their students I was able to gain insights into
the utilisation of different research methodologies and methods, the importance
of differing ontological and epistemological views regarding teacher practices
and the systemic approach to conducting effective, ethical, valid and creditable
research in educational contexts. This paper begins with the analysis of various
research approaches, ontology, epistemologies and paradigms used in research,
including approaches used in the research samples; discussion surrounding
various methods and methodologies, discussing the ethical considerations and the
influence of power in educational research and concluding with a discussion on
the major outcomes of the sample research processes.
Research
approach
There
is a substantial plethora of differing (however equally important) approaches to
research in social sciences, especially education; however to understand its
development all prospective researchers must consider the overarching concepts
of which ontological, epistemological and paradigm in which they belong to.
Before doing so, understanding of broad approaches needs to be established, with
O’
Toole & Beckett (2010) suggesting 3 approaches to research;
- Descriptive and interpretative: that being to describe how something works, what makes it
work and why does it behave in such a way; such as ethnography and case
studies - Intervention: What happens if we change or add aspects to
the scenario, which can be attributed towards action research and
experimental research - A
mix of the above, such as viewing a phenomenon and then acting to examine
the effects of changing (or adding) elements, such as reflective
practitioner case studies and narrative inquiry.
Ontology
forms the basis of all research inquiry as it forms the basis for the
construction of reality or as “claims and assumptions that are made about the
nature of social reality, claims about what exists, what it looks like, what
units make it up and how these units interact with each other.” (Blaikie, 2000,
p. 8, as cited in
Grix,
2002, p. 177) Epistemology is concerned with the theory of knowledge, how it is
obtained, how is it validated and new ways of knowing which are more effective
than other knowledge gathering models and theories (Grix, 2002). Paradigm then
results from the combination of the researchers’ ontological and
epistemological approach which informs the approach to research methodology and
research methods (Grix, 2002)
A
positivist paradigm can be viewed as a ‘scientific method’ approach to
educational research which prioritises a quantitative research approach.
O’Toole
& Beckett (2010) describes positivism as human understanding of behaviours
and learning should be positive, tangible and demonstrable.
Whereas
Darlaston-Jones
(2007) ascertain that‘positivism views reality as universal, objective, and
quantifiable’,suggesting that reality is measurably equally and remains the same
regardless of the individual experiencing it. A
constructionist paradigm however surrounds the idea that humans create their own
realities, scaffolding their learning in the process (O’Toole & Beckett,
2010) Progressing this, Darlaston-Jones (2007, pp.19) suggests that “reality is
shaped by the cultural, historical, political, and social norms”, that being the
accumulation of experiences and their interpretation of experiences into a world
view (which O’Toole & Beckett, 2010 regard as a philosophical
paradigm)
Transformative paradigm was developed the growing
dissatisfaction with the positivist and constructionist/interpretative with
Mertens (2007) suggesting that a transformative paradigm includes a cyclical
model of research which involves participants and researchers in the process
through the establishment of partnerships with power relations and ‘culturally
competent practices’ highlighting the key differences from previously mentioned
paradigms
Finally a pragmatic paradigm introduces the idea that there is
no single system of philosophy or reality (Mackenzie & Knipe, 2006). That
being said, the pragmatic paradigm allows the researcher to utilise data
collection strategies and data collection methods which are more likely to
provide insights into their central body of research questions without
“philosophical loyalty to any alternative paradigms” (Mackenzie & Knipe,
2006, pp. 197)
Progressing on, method and methodology are usually used
interchangeably and has led to a range of different meanings(Mackenzie &
Knipe, 2006) Methodology can be understood as a system for the methodical
collection of data (O’ Toole & Beckett, 2010) which informs the choice of
research methods required to satisfy the question in our area of research. For
example if the researcher desires to utilise an ethnographic methodology (i.e.
studying a bounded population and their experiences) then this informs the types
of methods needed to compile the data (artefacts, semi-structured interviews,
observations etc.). However Grix (2002) takes a different approach, suggesting
that research methodology concerns itself ‘with the logic, potentialities and
limitations of the research methods.’ (pp. 179), with research methods linked to
the research question posed and the data sources involved. Mackenzie & Knipe
(2006, pp.196) differ from these definitions, suggesting that the most common
definitions suggest that methodology is the overall approach to research which
is connected to the choice of paradigm whereas method relates to the systemic
collection and analysis procedures
of data.
Nicaise,
Bois, Fairclough, Amorose & Cogérino (2007) utilise a positivist paradigm
for the purposes of their research, mainly due to the researchers’ sole use of
quantitative questionnaires. Paired with this is the majority of the research in
the instruction and subsequent literature review is based on conclusion from
quantitative research, a large sample size of 333 participants (122 boys - 203
girls – four male teachers –four female teachers) and the decision to conduct
all student questionnaires on one occasion. Nicaise, Bois, Fairclough, Amorose
& Cogérino (2007) ultimately sought to use an inductive (theory building)
logic to complete two objectives;
·
To
assess the effects of student and teacher gender on physical education students’
perceptions of their teachers’ feedback and
time
·
To
determine whether PE students’ perceptions of their teachers’feedback and time
were related to students’ performance and psychosocial
growth.
To
achieve this Nicaise,
Bois, Fairclough, Amorose & Cogérino (2007) utilised a positivist
methodology consisting of 4 separate quantitative questionnaires and one
demographic information sheet, which include;
·
Perceived
Teaching Feedback Questionnaire (PTFQ)
·
Students
perception of teachers’ invested time
·
Perceived
competence, effort-importance and
interest-enjoyment
·
Physical
education performance
The questionnaire sequence was structured for the students to
complete the first three questionnaires with the first 15 minutes of one of
their physical education classes during February 2005, with the physical
education performance questionnaire being conducted in November 2004 and June
2005.
Davis & Nicase (2011) however use deductive reasoning (or
theory driven reasoning) to test if physical education teachers still exhibit
inequitable teaching practices, namely through Schon’s (2000) Theory of
reflective practice which focuses on the lack of physical education teachers’
knowledge on inequitable teaching practices. To achieve this Davis & Nicase
(2011) utilise a pragmatic paradigm, which is evident by their choice of
research question;
·
What
are the self-perceptions of the physical education teachers about their verbal
interactions with male and female students
·
Are
there differences between teachers at rural and urban high schools regarding
practices in relation to gender interactions with
students
·
Are
self-perceptions of each teacher compatible with their practices regarding
gender relations?
This
is paired with use of a qualitative methodology which takes precedence over the
use of quantitative methods to make their research claims and discuss their
findings in relation to other research conducted in gender interactions in
secondary education. Triangulation can be viewed as another indicator of their
pragmatic approach, as the researchers aim to substantiate their claims by
utilising three separate methods for collecting data. Davis
& Nicase (2011) utilise audio taped interviews and video-taped observations
of teacher behaviour, detailed field notes were taken. The four teachers
participated in semi-structured interviews separately before and after the
research process; with individualised follow up questions were used so the
participants could fully discuss their experiences. These interviews were
audio-taped, transcribed and ‘member checked’ to ensure reliability and
validity.
Davis
& Nicase (2011) also kept a research journal throughout the research
process, which serves two purposes; firstly the researchers are able to reflect
on their performance throughout the project, potentially furthering their
careers and knowledge. The second reason for this would be adopting a rigorous
reflexivity approach in their research. Subedi (2006) claims that by doing so
the researcher is being open and accountable for the knowledge created,
therefore challenging the notion of the‘all knowing’ or perfect researcher
(which ties into what constitutes ethical conduct in educational
research)
Data Analysis
Davis & Nicase (2011) analysed their data in two phases; a
quantitative analysis of the observation data to compare expected and observed
interactions and a qualitative ‘cross-case’ analysis of the teacher interviews,
the video-taped observations of teacher behaviour and field notes to investigate
the link between the teachers’ espoused theories and their
actions.
The
first aspect of analysis (Quantitative analysis) consisted of four steps
transcribing and coding 24 video-taped classes, focussing on the interactions
the teachers’had with their male and female students, quantifying the frequency
of instances in verbal categories and then assessing inter-rater reliability
concluded the data analysis. For the purposes of coding, the researchers used a
pre-determined procedure based from Martinek & Mancini’s (1989) dyadic
teacher-student observational tool. Davis & Nicase (2011) developed their
four categories (praise, questioning, informational and criticism) by combining
the existing six categories of
Martinek
& Mancini’s approach (praise, acceptance, asks questions, gives information,
gives direction and criticism), but did not elaborate the rationale behind this.
The lessons were then coded by two experienced researcher-observants with
Krippendorff’s alpha (Hayes & Krippendorff, 2007) which established
inter-rater reliability. According to Davis & Nicase (2011), Krippendorff’s
alpha is based around comparing the expected and observed logic of the
chi-square calculation.
Moving onto the qualitative analysis, each participant was
individually interviewed, video-taped, transcribed and analysed to discover
common themes which emerged in regards to gender interactions with students. The
researchers then met to form a consensus on key themes which emerged from there
analysis of the raw data with a cross-case analysis performed to look for
similarities and differences. To check the reliability of the data, the
researchers performed a‘member check’ of the constructed interpretation of the
interviewees responses, that being through discussion with the participant to
see if the data reflected their experiences (Freeman, deMarrais, Preissle,
Roulston & Pierre 2007); All participants confirmed that the qualitative
data was accurate and correct.
Nicaise, Bois, Fairclough, Amorose & Cogérino (2007)
analysed their data through a 2X2 (Student gender X teacher gender) Multivariate
analysis of covariance (MANCOVA) to view students’ perception around teacher
feedback and how teacher gender moderated the differences. In conjunction with a
MANCOVA the authors explored gender differences in Intrinsic Motivation
Inventory variables with a separate MANCOVA and tested for gender differences in
initial PE performance using an analysis of variance (ANOVA). To finish their
analysis, Nicaise, Bois, Fairclough, Amorose & Cogérino (2007) utilised four
regression analyses to test between the hypothesised connection between PTFQ
variables and students perceived competence, effort, enjoyment and
performance.
However
Likert based questionnaires can be problematic (Oliver, 2000) as Nicaise, Bois,
Fairclough, Amorose & Cogérino utilised three Likert methods in their study.
Tergan
(1998) suggests that evaluation based checklists have three key
weaknesses:
- Unknown reliability and validity of criteria; which leans
toward the idea that different assessors can allocate different ratings on
items of the same category, affecting both the validity and reliability of
results. - Shortcomings for assessing instructional efficacy; relating
to the failure of evaluators to take in account of learners cognitive
preconditions. - Lack of tailored criteria; which relates to a generalised
and inflexible structure of the checklist to target specific
aspects
Triangulation
is one way of testing validity;
Triangulation
is a process where data or results are corroborated with two other independent
sources to suggest the research data is actually reliable
(Jick,
1979). O’ Toole & Beckett (2010) however raise a critical argument to this,
suggesting that researchers believe triangulation is too positivistic in
nature, as it requires fixed points of reference. This is due to the‘empherial’
nature of education research, as it is ever fluid and like most aspects of
education, in a constant state of shift
(Sikes,
2006). Considering this, reliability then would need to be called into question,
if educational research is in fact in constant shift, how can we (as
researchers) insure that our research data is valid?
O’ Toole & Beckett (2010) suggest that researchers should
focus on four new key words in educational research; that being plausibility,
credibility, resonance
and transferability. From this the researcher can collect and analyse data
to mount an argument that plausible (i.e. Not easily disproven), creditable in
the sense that other researchers accept the results, has resonance with similar
findings in their field of research and transferable to other contexts outside
of the initial study area.
Ethical considerations
Informed consent is the central idea surrounding research
ethics, with Howe & Moses, (1999) suggesting that it is up to the
participants to weigh the risks of participating in research studies and the
only way for this to occur is for participants to be informed about and
essentially understand the research process. Both articles claimed to have
received written consent from their research participants but no there has
mention of the researchers giving the option for the participants (or guardians
in the case of minors) to withdraw from the study at any time and in turn
require the removal of data. This is a huge ethical concern when considering
conducting research with minors as it can suggest that initial informed consent
is final in the research process.
A
special consideration needs to be made due to the fact that the majority of the
research conducted in both papers was on children and adolescents (less than 18
years of age). This suggests that the participants may be vulnerable or coerced
into participation or disclosing unnecessary details; Morrow & Richards
(1996) suggest that children are essentially vulnerable in two ways, physically
with a lack of knowledge and experience and structurally due to a lack of
political and economic power; both potentially making them vulnerable to adult
influence. Although
Morrow
& Richards (1996) may have been writing exclusively about individuals which
may not be considered adolescent or‘pre-adult’, both research papers suggest the
bulk of participants are aged between 12-16 years of age. Both
research groups however approached the ethical implications of their research
by seeking permission from the institutional review board of their university,
both schools’ research offices and the superintendent of the district. The
researchers then received informed consent of all four teachers, the parents of
students and received assent from the students themselves; also the researchers
sought to provide anonymity for the participants and used pseudonyms for the
teachers and schools involved.
However
It can be assumed that both research teams failed to recognise how and where
data would be stored, what protections are placed on the data to prevent
external individuals ( i.e. outside of the research team) and how and if data
which has been collected will be destroyed. O’ Toole & Beckett (2010, pp.101)
suggest ethical research should entail details of the treatment of participant
data, including storage and possible destruction of materials which are
identifiable to the research participants. The only exception to this was by
Nicaise, Bois, Fairclough, Amorose & Cogérino (2007) who simply stated that
the students were assured that their responses would remain confidential and
their teachers would not see their answers, but still fall short of explaining
how this would be done. This highlights Howe & Moses (1999) concept that
privacy during and even after research is the second most important
consideration for educational researchers in regard to the ethical treatment of
research participants, however ethical treatment of participants is subjective
to the researcher as mentioned earlier with Davis & Nicase (2011) opting to
include a research journal to be critiqued by themselves, peers and potentially
the research participants themselves (Subedi,
2006)
Power in research
Davis & Nicase
(2011) detail in their data collect method that in order to reduce reactivity
and social desirability bias
(Fisher, 1993) whichconcerns participants acting in a way feel is right or
acceptable in response to being studied, the first two days of the 8 day
bi-weekly study was discarded. Researcher bias was also a significant concern
for Davis & Nicase (2011), in response to this analysis of the data was
verified by colleagues for reliability, the written reports of case studies were
shared with their respective participant for verification, maintaining a journal
of the research process, following the guidance of previous researchers and
searching for negative instances. In regards to video recorded observations
Davis& Nicase (2011) utilised trained college students in order to limit the
influence of the researcher figure in the classroom and by taking the
aforementioned approach, the power of the researcher in research is
reduced.
Davis & Nicase
(2011) however fail to detail the setting in which the interviews take place; is
it an open space such as a gymnasium, is it a conference room with a large table
separating them or even is it an intimate staff room with the researcher next to
the participant? Mallozzi (2009) highlights the potential environmental
influence on both the researcher and participant, which influences the
credibility of the responses and interpretations which result. Mallozzi does
this by constructing a scenario based around an interview she was preparing to
conduct, with considerations of armchairs and indoor plants being‘trumped’ by a
12 seated table which dominates the room; this triggers her internal dialog
between the three competing research voices, consisting of Postie
(Post-structuralist), Libby (Feminist) and Ed (post-positivist). Questions the
use of empathy arises when considering that a feminist approach encourages a
construction of empathy and environmental influence around socialisation and
rapport where as a post-positivist would argue for a construction of empathy
around maintaining neutrality.
Nicaise, Bois, Fairclough, Amorose & Cogérino (2007) however
eluded to the students’ completing the questionnaires immediately before
physical education classes, suggesting that the students were situated in a
gymnasium (so a wide open space) with their teacher absent; however the authors
fail to detail the layout of the students in relation to the researchers
themselves, how many researchers were present what was the ratio of male to female
researchers in the setting and were the students clustered together or spread
apart? All factors considered it can only be assumed that at least one
researcher was present at each school for the purposes of the questionnaires
(both for students and teachers), which resonates with the power issues
previously eluded to by Mallozzi (2009)
Conclusion
Research can take on various forms, such as the expression of a
population using figures through quantitative means or constructing in-depth
stories of the individuals lived experience through qualitative case studies, as
shown with the two research articles, different approaches to research design
are impossible to distinguish as the best form of educational research. These
articles successfully articulate how two different paradigms can converge to
articulate the same phenomenon in physical education teachers’
practice.
This has been illustrated by the differing ontological and
epistemological position of each article framing the development of the
individual research methods and methodologies, which were clearly articulated
and grounded in their respective
philosophical literature. Unfortunately research ethics was not discussed during
most of the research papers under analysis leaving the reader to make
assumptions upon what ethical research in education looks like.
Just as importantly this research process has highlighted the
responsibility of the researcher to develop and article what ethical research
entails and the potential influence of power relations on the research process,
especially during data collection and analysis. This is essential for the
development and growth of the beginning researcher in the social sciences as it
allows for the development of awareness of the major influences and shifts in
educational paradigms and research convections.
References
Darlaston-Jones, D. (2007). "Making connections: the
relationship between epistemology and
research methods." The Australian
Community Psychologist Vol.19 No. 1,
pp.19-27
Davis, K & Nicase, V. (2011), Teacher student
interactions : Four case studies of gender in
physical education, Journal of Classroom Interaction, Vol. 46, No.2,
pp.11-23
Fisher,
R. J. (1993). Social desirability bias and the validity of indirect
questioning.
Journal of Consumer Research, pp.303-315.
Freeman, M.,
Preissle, J., Roulston, K., & Pierre, E. A. S. (2007). Standards of
evidence
in qualitative
research: An incitement to discourse. Educational researcher, Vol.36, No1, pp.
25-32.
Hayes, A. F., & Krippendorff, K. (2007). Answering
the call for a standard reliability measure for
coding data. Communication Methods and Measures, Vol.1,
No.1,
pp. 77-89.
Howe, K. & Moses, M. (1999). Ethics in educational research.
Review of Research in
Education, vol. 24, pp. 21-60
Jick,
T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in
action. Administrative science
quarterly, Vol.24, No.4, pp. 602-611.
Mertens, D. M. (2007). Transformative paradigm: Mixed methods
and social justice. Journal
of Mixed Methods Research, Vol.1, No.2, pp.212-225
Morrow, V., & Richards, M. (1996). The ethics of social research with
children: An
overview. Children& society, Vol.10, No.2, pp.
90-105.
Nicaise, V., Bois, J. E., Fairclough, S. J.,
Amorose, A. J., & Cogérino, G. (2007). Girls'
and boys' perceptions of physical education teachers' feedback: Effects
on
performance and psychological responses. Journal Of Sports
Sciences, Vol.25, No.8, pp. 915-926
Oliver, M. (2000) An introduction to the Evaluation of Learning Technology.
Educational
Technology & Society, Vol. 3, No.4, pp.
20-30.
O'Toole, J. & Bennett, D. (2010). Educational research:
Creative Thinking & Doing. South Melbourne: Oxford University Press
Sikes, P. (2006). On dodgy ground? Problematics and ethics in educational
research. International
Journal of Research & Method in Education, 29(1),
105-117.
Subedi, B (2006) Theorizing a ‘halfie’ researcher’s identity in
transnational fieldwork,
International Journal of Qualitative Studies in Education, Vol.19,
No.5,
pp. 573-593
Tergan,
S. O. (1998). Checklists for the evaluation of educational software:
critical review
and prospects. Innovations in education and training international,
Vol.35,
No.1, pp.9-20.