Fechar
Formação
Serviços
voltar atrás
Measuring Inclusive Education in Portuguese Schools: Adaptation and Validation of a Questionnaire
The debate about inclusive education has gained considerable attention in policy, research, and practice in the last decades. Therefore, there is also a growing interest in assessing how inclusive education occurs in schools. Self-assessment and self-reflection tools are strategic to examine schools’ inclusion and guide needed changes. This study attempts to provide Portuguese schools with self-assessment tools for improvement. Themis Inclusion Tool was translated and adapted into a Portuguese version named Resources and Practices for Inclusive Education and evaluated the instrument’s psychometric structure. A web survey was completed online by 924 participants. An EFA suggested two different factors: Inclusive Resources, human, technical and technological resources used to promote learning and; Inclusive Practices, beliefs and behaviors that can be implemented to promote learning. These two factors were supported by confirmatory analysis. Overall, the Resources and Practices for Inclusive Education presented a robust factorial structure and good psychometric properties, appearing to be a valid and reliable measure for assessing inclusive education in Portuguese schools.
Inclusion is one of the leading schools’ concerns as inclusive education remains one of the critical goals of political agendas and educational reforms internationally (OECD, 2015; UNESCO, 2017, 2020a). However, the meaning of inclusive education remains confusing and sometimes controversial (McMaster, 2012; Moya, 2019; Ainscow, 2020; Azorín and Ainscow, 2020). Even though there is a generalized idea that schools and societies should become more inclusive, the practice is sometimes contradictory. The interpretation of inclusive education or inclusive schools varies significantly across Europe (UNESCO, 2020a). Therefore, it is necessary to clarify and agree upon what inclusion means and what actions need to be taken to move practice in a more inclusive direction (McMaster, 2012; Azorín and Ainscow, 2020).
The debate about inclusive education has gained considerable attention in the last decades worldwide. In 2019, the UNESCO International Forum on Inclusion and Equity in Education, organized to celebrate the 25th anniversary of the Salamanca Declaration, had “Every learner matters” as the core theme. This idea underlined the notion of inclusion as a general guiding principle to strengthen equal access to quality learning opportunities for all learners (Ainscow, 2020). Also, the Global Education Monitoring Report, with its 16 editions hosted and published by UNESCO (from 2002 to 2020), brought awareness about progress and achievements related to the fourth Sustainable Development Goal (SDG 4) on education (i.e., ensure inclusive and equitable quality education and promote lifelong learning opportunities for all). The debate took place on how educational systems provide all children opportunities to learn together, recognize and respond to their student’s diverse needs, and identify and overcome barriers for vulnerable and marginalized groups (UNESCO, 2020b). Although in some countries, inclusive education is still thought of as an approach to serving children with disabilities within general education settings, the concept is being broadened as a principle that supports and welcomes diversity amongst all learners (Ainscow, 2020).
In Portugal, such as in other countries, inclusive education has gained relevance in policies and attempts to move schools toward more inclusive perspectives, and practices were being done. In 2018, the Portuguese Ministry of Education enacted a law devoted to inclusive education, bringing school improvement challenges. Inclusion is defined as “the right of all children and students to access and participate, fully and effectively, in the same educational contexts” (Decree-Law 54/2018, Art. 3c). This definition highlights core aspects of inclusion, such as the quality education as a right of all children, the provision of access and participation opportunities for all children, a wider target population as the scope of inclusive education and the mainstream educational contexts as contexts for all children to learn and participate. As such, schools need to change, and practices need to be improved to respond to the diversity of the school population (Alves, 2019; Ainscow, 2020).
Analyzing challenges of inclusive education, Carvalho et al. (2019) have identified specific strengths and weaknesses of Portuguese schools as perceived by their teachers. Portuguese teachers reported the acceptance and respect to the differences, the provision of support for students, and the teachers and students cooperation as the most valuable strengths of Portuguese schools. Nevertheless, teachers reported limited resources and insufficient professional development opportunities related to inclusion. Even though Portuguese teachers value inclusion to attend diversity, this same diversity in the classroom is a challenge for teachers when planning and teaching lessons for all students. Alves et al. (2020) also observed that, although almost all Portuguese students with disabilities currently attend mainstream schools, some students with more significant support needs still spend most of their time segregated from the rest of the class. As reported from some other countries, despite national policies emphasizing the equal rights of children with disabilities to attend mainstream settings, there has been evidence that these same students are still categorized and segregated in their school (cf. Ainscow, 2020). Therefore, schools need to design an “agenda for change” (Ainscow, 2020, p. 12), considering context, processes and resources for inclusion.
By the same token, international literature underscores multiple dimensions involved in inclusive education (Booth and Ainscow, 2011; Azorín et al., 2019; Ainscow, 2020; Azorín and Ainscow, 2020). The available resources, especially teacher resources, are repeatedly identified as a barrier to successfully implementing an inclusive school system (cf. Goldan and Schwab, 2018). The literature on resources primarily includes studies on the economics of inclusive education, taking into account the costs of an inclusive education system, funding, models of resource allocation, and financing for special education in general (Goldan and Schwab, 2018). Despite its relevant contribution to the field, these studies fail to explain how effectively to allocate resources better and how they must be used to contribute to inclusive education (Loreman, 2014). School’s resources, and the perceived available resources, are important variables to consider about high-quality inclusive education. Nonetheless, it must be analyzed concerning other dimensions, such as processes involved in a school’s improvement toward inclusive education (Messiou et al., 2016; Ainscow, 2020). For example, Booth and Ainscow (2011) presented the well-known Index for Inclusion as a framework for examining school factors that may create barriers to learning and participation, and as so to inclusive education, including three core dimensions: (1) cultures, (2) policies and, (3) practices. Another relevant work that has inspired this specific study considers three dimensions: (1) contexts, which refers to circumstances surrounding the schools (from within schools, between schools, and beyond schools), (2) resources (personal, institutional and local) and, (3) processes that have to do with presence, participation and achievement (Azorín et al., 2019; Azorín and Ainscow, 2020).
Some authors argue that inclusion can be seen as a journey, an ongoing and never-ending process of the continuous effort of reflection, change and improvement through all learners’ participation and learning (McMaster, 2014; Nguyen, 2015; Ekins, 2017; Azorín, 2018). The construction of inclusive education is a continually adapting process of education and school to individual and social needs through policies, practices and ethos. This involves restructuring the entire educational system and continuously measuring the practice models and outcomes (Vrasmas, 2018). A school contextual analysis would be an essential baseline to detect improvement areas to render schools more inclusive (López-Azuaga and Riveiro, 2018; Azorín and Ainscow, 2020). Exploring, understanding and improving perspectives and practices about the challenges of inclusive education is one of the starting points of becoming a more inclusive school. Schools need to establish a shared understanding of inclusion so that they can help draw people together around this common purpose—a culture of inclusion—(McMaster, 2012; Moya, 2019; Azorín and Ainscow, 2020) and a common language (Ainscow, 2020).
Self-assessment is a powerful strategy to consider when planning schools’ improvement concerning inclusive education (McMaster, 2012; Bourke and Mentis, 2013). The growing attention to inclusive education also brought an increasing interest in assessing how the response to diversity occurs. Therefore, it is crucial to have tools that enable examining schools’ inclusion culture and a framework to guide change (McMaster, 2012; Azorín et al., 2017; Moya, 2019; Azorín et al., 2021). Collecting data is required to “monitor the progress of children, evaluate the impact of interventions, review the effectiveness of policies and processes, plan new initiatives” (Ainscow, 2020) and also to identify barriers for improvement (Azorín et al., 2021). Data and evidence can be used to support improvement toward inclusion as they can be the basis for starting the necessary dialogue and interconnections about actual concepts and practices and about the changes to address (Ainscow, 2020; Azorín and Ainscow, 2020).
Internationally, some tools exist to assess different dimensions of inclusive education and enhance schools’ plans for more inclusive practices. Instruments such as Index for inclusion (Booth and Ainscow, 2011), Themis Inclusion Tool (Azorín et al., 2019), Guide for Evaluation of Inclusive Practices in Classroom (Guía de Evaluación de Prácticas Inclusivas en el Aula) (García et al., 2011), Perceived School Support for Inclusive Education (Ahmmed, 2013), Sentiments, Attitudes and Concerns about Inclusive Education (Forlin et al., 2011) and Teacher Efficacy on Inclusive Practices (Sharma et al., 2012) are examples of tools that encompass different dimensions, levels of analysis and indicators associated with inclusive education. All of the available instruments to evaluate inclusive education also vary in the areas of interest, such as school practice, classroom practice, teacher competency, and inclusive pedagogy (Forlin and Loreman, 2014; Azorín, 2017; Ewing et al., 2018). The instruments tend to assess perceptions of inclusive education in the educational context as a way to address the reality that teachers, students, parents and school leaders consider a priority to change, focusing more on needs and challenges (Azorín, 2017; Azorín et al., 2017; Navarro-Mateu et al., 2020).
Despite the recent changes in Portuguese law, there are scarce tools available to monitor and evaluate inclusive education. It is worldwide recognized that Portugal has gone further in enacting an explicit legal framework for inclusion in the education of students with and without disabilities (cf. Alves, 2019; Ainscow, 2020). However, some challenges exist, and monitoring schools progress is necessary to put the law in action (Carvalho et al., 2019). Attempts to provide Portuguese educational systems with specific tools to guide schools on improvement are valuable efforts. Data is required to monitor children’s progress, evaluate the impact of support provided and review the effectiveness of processes and policies (Ainscow, 2020). A holistic and comprehensive framework covering dimensions of context, processes and resources is needed to monitor and evaluate Portuguese schools in inclusive education.
Some of the tools mentioned above are available in Portuguese, as is the case of Index for inclusion (Booth and Ainscow, 2011) which is a well-known instrument in Portugal, used in research (e.g., Messiou et al., 2016) but also in schools to support practices (Pereira et al., 2018). Index for inclusion is particularly relevant as it helps schools’ reflect and change toward inclusive education. In addition, other specific instruments like Sentiments, Attitudes and Concerns about Inclusive Education (Forlin et al., 2011) and Teacher Efficacy on Inclusive Practices (Sharma et al., 2012) are translated to Portuguese and are also used in research (e.g., Santos and Cesar, 2010, 2014; Silva, 2019). However, to our knowledge, no published validation studies exist about these instruments, and the knowledge available about the validity and reliability of these Portuguese versions is not enough to support its usage. For Themis, there is no Portuguese version nor validation studies. As Index for Inclusion, this tool can provide a picture of schools contexts considering multiple dimensions. However, Themis has the advantage of being a simple tool that intends to serve as a self-assessment or diagnosis of strengths and weaknesses in terms of response to diversity (Azorín et al., 2019). Inspired on previous studies related to inclusive education tools and assessment (e.g., Index for inclusion) and incorporating new emergent and relevant trends, “Themis gives the opportunity to rethink the contexts, resources and processes of schools, involving teachers in the journey to inclusion and encouraging them to undertake improvement in this regard” (Azorín et al., 2019, p. 28). Being easy and straightforward to answer offers an opportunity for self-reflection and overall approximation on the response to diversity in a specific context, fostering individual and collective discussion from teachers’ perspectives and raising awareness on processes toward inclusion (Azorín et al., 2019).
Themis Inclusion Tool is an example of a self-assessment tool designed to improve inclusive education schools. This tool was developed to help discuss diversity in schools. It may serve as a guide for enhancing processes geared toward inclusive practices in schools considering context, processes and resources (Azorín et al., 2019; Azorín and Ainscow, 2020). Themis Inclusion Tool was developed to assess schools’ perspectives and conceptualizations about inclusive education and resources available to promote participation, effective learning and collaborative work. This instrument has the advantage of allowing a macro-analysis of the school context and an individual evaluation of the pedagogic practice. Therefore, this tool seemed useful for undertaking processes to improve the development of more inclusive practices as expected in the Portuguese educational system.
This instrument was translated and adapted for Portuguese context to provide schools with self-assessment tools for improvement. Even though the relevance and validity of other instruments are recognized, this tool is easy to complete and provides a baseline about the schools’ current state regarding inclusion. Besides, it is a relevant tool for reflection about improvement needs, potentially triggering changes in school professionals’ perspectives and practices (Azorín et al., 2019; Azorín and Ainscow, 2020). This paper presents the Themis Inclusive Tool’s translation and adaptation for Portuguese context and the validation study of the new version of the instrument. It also analyses its psychometric qualities (factorial structure, reliability, validity).
This research aims to translate and adapt the instrument “Themis Inclusion Tool” for Portuguese context and validate the new version. The objectives pursued were: (a) to study the adequacy of the content through experts’ judgment; (b) corroborate the validity of the understanding of the instrument through its application to a pilot sample; (c) determine the multidimensionality of the construct through exploratory factor analysis; (d) confirm the multidimensionality of the construct through confirmatory factor analysis, and (f) analyze the reliability of the questionnaire.
The sample was composed of 924 adults (teachers and specialized technicians) who worked in private and public schools in Portugal (see Table 1). Participation in the study was voluntary, and responses to the questionnaire were online, confidential, and anonymous.
TABLE 1. Sample demographics.
We followed Anderson and Gerbing (1988) suggestion for the analysis: the sample was randomly split. One part was used to develop a model, and the other part of the sample validated the solution obtained from the first analyses. This sample procedure was also conducted by Morgado et al. (2021). The choice of sample size was based on the argument that the sample size for factor analysis should be more than 200 (Brace et al., 2003), and there should be a minimum ratio of 2 observations per variable (Kline, 1994).
The data of 282 participants were randomly collected to perform exploratory factor analyses. Two hundred twenty-eight were female, and 54 were male. The mean age was fifty (SD = 6.95), and participants ranged from 26 to 68 years old. Two hundred forty-seven participants were teachers from kindergarten to high school, and 35 specialized technicians worked in schools (psychologists, physiotherapists, speech therapeutics).
Confirmatory factor analyses were performed in a second group, randomly selected. Six hundred forty-two participants, with no missing data, 523 were females, and 119 were males. The participants’ mean age was fifty (SD = 7.68), ranging between 25 and 66. Five hundred thirty-three participants were teachers from kindergarten to high school and 109 specialized technicians who worked in schools.
Themis Inclusion Tool (Azorín et al., 2019; Azorín and Ainscow, 2020) was developed to facilitate school reflection about the contexts, resources and processes that underpin teachers work. The Themis Inclusion Tool covers 65 items, organized in three dimensions: contexts, resources, and processes in inclusive education. “Contexts” refers to the circumstances surrounding the school, such as socioeconomic status, cultural diversity, home-school collaboration and community participation. “Resources” regard personal, institutional and local school resources available for inclusion. “Processes” refers to the way schools can enhance presence, participation, and achievement (for example, teaching planning, time, and space management). A 5-point Likert scale response format is used (1—completely disagree to 5—completely agree).
This study aimed to translate and adapt the Themis Inclusion Tool into a Portuguese version named “Resources and Practices for Inclusive Education.” We also aimed to evaluate the psychometric structure of the instrument so that it can be used not only to promote a reflection about inclusive settings but also to create knowledge about inclusive practices in Portugal.
Because the instrument was created for a different population, a translation, adaptation and standardization process was required to attain the validity of the content. The items were first translated into Portuguese, then re-translated into the original language from the Portuguese translation (back-translation).
After translation, two experts have reviewed the questionnaire’s first version. They were asked about the revised instrument’s clarity, conciseness, and terminological precision. After expert revision, a spoken reflection (individual and group) with teachers (n = 6) and psychologists (n = 5) was made to check for understanding and clarity of items and relevance/appropriateness. Minor changes were made to meet these criteria. These stages helped ensure that the items were understood and accurately represented the primary contexts, resources and practices for inclusive education.
The only precondition for participation in the study was being a teacher or a specialized technician working in a Portuguese public or private school. All relevant information for the participants’ informed consent was presented before the beginning of the questionnaire. No information that would allow identifying each participant was requested, granting all data anonymity. Participants were required to complete an anonymous self-report, online questionnaire which included an instruction sheet and a consent form. Participants were assured about confidentiality and informed that their participation was voluntary. The online questionnaire was announced through mailing lists. The aims of data collection were briefly described. Upon opening the questionnaire’s link, it provided participants with a complete description of the objectives, institutional framework, length and confidentiality issues. If participants choose to fill out the questionnaire, they supply an online consent form and answer it. It was stated that participation was voluntary, with no incentives for participation. Scale administration occurred between March and June of 2019.
Item analysis was performed to select appropriate items for subsequent factor analysis. The items on each scale were analyzed by examining the distribution of different responses, inter-item correlations, and each item’s correlations with its corresponding subscale. Exploratory factor analysis (EFA) was conducted with principal component extraction and a varimax rotation.
Confirmatory factor analysis (CFA) was used to test the fit of the two-factor model. To assess the global fit of the tested model, the following criteria were used: the chi-square (χ2) values, the ratio between the chi-square and the degrees of freedom (χ2/df), the comparative fit indexes (CFI: Comparative fit index; GFI: Goodness-of-fit index) and the root mean square error of approximation (RMSEA). Model fit was considered acceptable when χ2/df was lower than 3.00, CFI values were higher than 0.90, RMSEA lower than 0.08 and GFI greater than 0.90 (Schermelleh-Engel et al., 2003). The value of chi-square represents the “distance” between the matrices of data variance-covariance and the matrix as measured by the model. The lower its value in relation to the degrees of freedom, the better its adjustment (Jöreskog and Sörbom, 1993; Harlow et al., 2002). The test of chi-square is influenced by the number of cases in the sample, and due to this, it is suggested that the ratio between the chi-square and the degrees of freedom (χ2/df) be calculated, as well as those other indices be consulted (Kline, 1994).
Continue reading the article here Measuring Inclusive Education in Portuguese Schools: Adaptation and Validation of a Questionnaire
