Organizational climate scale for public service: Development and validation

de Matheis, Taiane Keila; Campos, Simone Alves Pacheco; Vieira, Kelmara Mendes; Lehnhart, Eliete dos Reis; Estivalete, Vania de Fátima Barros Article Organizational climate scale for public service: Development and validation Administrative Sciences Provided in Cooperation with: MDPI – Multidisciplinary Digital Publishing Institute, Basel Suggested Citation: de Matheis, Taiane Keila; Campos, Simone Alves Pacheco; Vieira, Kelmara Mendes; Lehnhart, Eliete dos Reis; Estivalete, Vania de Fátima Barros (2024) : Organizational climate scale for public service: Development and validation, Administrative Sciences, ISSN 2076-3387, MDPI, Basel, Vol. 14, Iss. 5, pp. 1-20, https://doi.org/10.3390/admsci14050090 This Version is available at: https://hdl.handle.net/10419/320910 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ Citation: Matheis, Taiane Keila, Simone Alves Pacheco de Campos, Kelmara Mendes Vieira, Eliete dos Reis Lehnhart, and Vania de Fátima Barros Estivalete. 2024. Organizational Climate Scale for Public Service: Development and Validation. Administrative Sciences 14: 90. https://doi.org/10.3390/admsci 14050090 Received: 3 April 2024 Revised: 22 April 2024 Accepted: 23 April 2024 Published: 28 April 2024 Copyright: © 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). administrative sciences Article Organizational Climate Scale for Public Service: Development and Validation Taiane Keila Matheis , Simone Alves Pacheco de Campos , Kelmara Mendes Vieira * , Eliete dos Reis Lehnhart and Vania de Fátima Barros Estivalete Department of Administrative Sciences, Federal University of Santa Maria, Santa Maria 97105-900, RS, Brazil; [email protected] (T.K.M.); [email protected] (S.A.P.d.C.); [email protected] (E.d.R.L.); [email protected] (V.d.F.B.E.) *Correspondence: [email protected] Abstract: This research presents four studies that developed and validated the Organizational Climate Perception Scale for Public Service (OCPS-PS). The first qualitative study consulted the literature and conducted a focus group to develop the initial version of the scale. The second study involved expert evaluation and pre-testing, aiming at the semantic and face validation of the items. This study resulted in 80 items forming the thirteen dimensions of organizational climate. The third study obtained the first quantitative sample for the exploratory validation phase of the scale. The final study, using a new sample, conducted confirmatory tests for the validation of the scale. A methodology for applying the scale was developed, allowing all interested parties to use the OCPS-PS for the assessment of the organizational climate in public service. The results of the four conducted studies indicate the adequacy of the OCPS-PS according to the proposed criteria of validity and reliability. Finally, the OCPS-OS was built to be applied in different public organizations and at different government levels. Keywords: organizational climate; public service; scale; validation; psychometric procedures 1. Introduction In environmental psychology, psychological environments encompass the various meanings individuals associate with their physical surroundings (Hur and Abner 2024; James et al. 2008). Specifically within the organizational setting, the psychological climate refers to the meanings individuals attribute to their jobs, coworkers, leaders, compensation, performance expectations, promotion opportunities, and fairness in treatment (James and Jones 1974). Building upon these concepts, the initial primary purpose of organizational climate was to identify and assess the work environment through the direct and indirect perceptions of individuals working in that environment (Litwin and Stringer 1968). Shortly thereafter, McClelland (1972) observed that organizational climate serves as a tool for managing employee motivation. In the view of Patterson et al. (2005), organizational climate is an essential dimension in the workplace, providing a means of study for organizational behavior research by identifying individual and group behaviors. In this context, Mutonyi et al. (2020) define climate as the cognitive representations of individuals and the psychological interpretations of their organizational environment. These designations already indicate the multidimensional nature of the concept (Pérez-Vallejo and Fernández-Muñoz 2020), regardless of whether it is a public or private organization. Public organizations intervene with private organizations in terms of ownership, financing, and control (Bozeman 1987). Therefore, several factors must be taken into consideration, such as hierarchical levels, objectives and responsibilities, economic forecast/unfeasibility, legal and formal restrictions, flexibility/inflexibility in decision-making, social characteristics and functions, among other differences (Boye et al. 2022;Jain and Dhir Adm. Sci. 2024,14, 90. https://doi.org/10.3390/admsci14050090 https://www.mdpi.com/journal/admsci Adm. Sci. 2024,14, 90 2 of 20 2022;Johansen and Zhu 2014;Meier and O’Toole 2011;Rainey 2014;Torfing et al. 2020; Van Wart 2013). In addition to these, Knies et al. (2022) emphasize the traditional distinction between human resource management in public and private contexts, highlighting the search for equal opportunities in the public sector and the orientation towards performance in private sectors. In this sense, it is observed that management is also different, reflecting autonomy, coexistence with internal and external stakeholders, accountability, and values, affecting the organizational climate of each organization (Boye et al. 2022;Steinfeld 2023). Especially concerning public organizations, there is a pressure for changes, setting a new pace for transformations, requiring constant adaptations in legal structures and political preferences, modifying the organizational climate (OC) unexpectedly and without the involvement of the majority of public servants (Babapour Chafi et al. 2022;Wynen et al. 2020). In several countries, the traditional, hierarchical, legalistic public administrations have been partially replaced by results-oriented public organizations. Public organizations have adopted the two central new public management goals, efficiency and effectiveness (Lapuente and Walle 2020). Performance in the public sector is now universally recognized as outcomes and impacts (Andersen et al. 2016). Additionally, when compared to the private sector, public organizations exhibit significant differences in people management processes, including selection and entry methods, progression and performance rules, job stability, and the occupation of management positions, among others. However, the literature lacks scales specifically constructed and validated for assessing the organizational climate in the public context. One of the first models to study organizational climate was proposed by Litwin and Stringer (1968), who concluded that organizational climate fluctuated and could be modified (Ferreira and Ramal 2013). This reasoning was refined in the 1970s, 1980s, and 1990s, leading to the emergence of new models to identify organizational climate, including Campbell et al. (1970); Sims and LaFollette (1975); Schneider (1975); Sbragia (1983); Kolb et al. (1986), and Rizzatti (1995). The oldest scale found was that of Siegel and Kaemmerer (1978). In addition to this, numerous scales have been constructed for developed countries and adapted for developing countries, but without extensive cross-cultural validation (Banwo et al. 2022;Do Amaral et al. 2005;Santos and Paranhos 2017) or psychometric procedures (Curvo and Heinzmann 2017). There are several motivations that led us to build a new scale, we list four: (i) the mismatch of the main scales already developed in the private context for investigating the organizational climate in the public sector; (ii) the need to adapt some concepts, dimensions, and items from private-oriented scales to the public service; (iii) the creation of new items and dimensions such as public value and professional performance that aim to highlight the perception of public servants; (iv) the scarcity of organizational climate scales that had psychometric validation. In light of the above, the proposition of a specific measure of the organizational climate for public service holds central significance. Thus, the aim is to develop and validate the Organizational Climate Perception Scale for Public Service (OCPS-PS). The construction of an organizational climate scale for public service can offer both theoretical and practical contributions. On the theoretical side, it seeks to deepen academic knowledge in the field of people management in public organizations. From a practical perspective, it provides managers with the means to assess the work environment by identifying employees’ perceptions. Moreover, it serves as an important management tool. In this regard, the scale will enable the investigation of inter- and intra-organizational issues in various public agencies (Tripathi and Tripathi 2022), fostering new approaches. 2. Method To construct and validate the OCPS-PS, four distinct studies were conducted. The implementation period of this study was the year 2023. The first study involved a literature review and a focus group. The literature review was carried out by searching the Web of Science and Scopus databases, focusing on the time frame from 2012 to 2022 and including Adm. Sci. 2024,14, 90 3 of 20 documents with the defined terms “organizational climate scale” in the title. The final sample consisted of 60 articles, allowing for the identification of key definitions, dimensions, and items of existing scales, leading to the preliminary construction of the OCPS-PS dimensions. Subsequently, a focus group was conducted with the participation of ten public servants from diverse socioeconomic, demographic, and professional backgrounds. The second study included expert validation and a pre-test phase. Following DeVellis and Thorpe (2021) recommendation, six experts were selected—three with experience in scale construction and three with subject-matter expertise. A specific instrument was created for this step, providing specific instructions for the experts. For each item, questions were posed to assess the degree of pertinence (1—Should be removed, 2—Should be retained after reformulation, 3—Should be retained as is), the degree of relevance (1— Slightly Relevant, 2—Relevant, 3—Very Relevant), and the dimension represented by the item. To evaluate the level of agreement among the experts, the Content Validity Coefficient (CVC) and Fleiss’ kappa (Fleiss 1971) were used. Content validity refers to the extent to which a scale adequately samples items that represent the construct of interest (Hinkin 1995). The CVC is suitable for Likert-type scales, while Fleiss’ kappa is appropriate for nominal scales. For the pre-test, 10 individuals were selected based on convenience sampling, in line with the suggestion of Boateng et al. (2018) that pre-test samples should include between 5 and 15 participants. The individuals were chosen from a variety of backgrounds to ensure that the language used in the instrument was appropriate for the entire target population. For the third study, a new instrument with the OCPS-PS was developed and applied to a sample of 394 public servants for the purpose of exploratory validation of the scale. The polychoric correlation matrix was used, employing the Robust Diagonally Weighted Least Squares (RDWLS) factor extraction method and Robust Promin rotation (Lorenzo-Seva and Ferrando 2019). Following Timmerman and Lorenzo-Seva’s (2011) recommendation, the parallel analysis was optimally implemented to estimate the number of factors. Additionally, the H index was evaluated (Ferrando and Lorenzo-Seva 2018), indicating factorial replicability. Items with values above 0.80 corresponded to the observed factors, suggesting that the factor structure could be replicated in other studies. Internal consistency was assessed through calculations of Cronbach’s Alpha (Cronbach 1951) and McDonald’s Omega ( ω ) (McDonald 1999). Values equal to or greater than 0.7 were considered adequate for both measures (Hair et al. 2019). In the fourth study, the OCPS-PS was administered to 374 public employees from a higher education institution for the purpose of confirmatory validation of the scale. Convergent validity, unidimensionality, and discriminant validity of the constructs were assessed. The models were estimated using maximum likelihood via a direct procedure. Convergent validity was examined by observing the magnitude and statistical significance of the standardized coefficients, utilizing absolute fit indices such as chi-square statistics ( χ2 ), residual root mean square (RMR), root mean square error of approximation (RMSEA), and the Comparative Fit Index (CFI). Criteria considered for adequacy were values greater than 0.950 for CFI, RMSEA, and RMR values lower than 0.060 and 0.080, respectively, and a chi-square/degrees of freedom relationship with values below 3.0 (Byrne 2010;Hair et al. 2019;Hooper et al. 2008;Kline 2023). The research received approval from the Research Ethics Committee, and the participants provided their informed consent by signing the Free and Informed Consent Term (ICF). The instrument used was completely anonymously. 3. Analysis of Results 3.1. Study 1: Literature Review and Focus Group Potential dimensions for the OCPS-PS were identified based on a literature review. The literature review was carried out for the period of 2012–2022 using the terms “climate organizational scale” in the Web of Science and Scopus databases. After removing dupli- Adm. Sci. 2024,14, 90 4 of 20 cates, a database with 60 articles was obtained. From reading their contents, 10 articles were identified whose scales were used in the most diverse sectors (Benzer and Horner 2015; Hutchinson et al. 2018;Nikolova et al. 2014); industrial (Hannevik et al. 2014;Jafari et al. 2017;Patterson et al. 2005); services (Finsel et al. 2021); educational (Buttner et al. 2012); cooperative (Zappala et al. 2018); and electronic devices (Mor Barak et al. 1998) sectors and which could serve as inspiration for the construction of the OCPS-PS items and dimensions. The focus group included ten public servants with diverse socioeconomic, demographic, and professional profiles. The research protocol outlined the involvement of four researchers as facilitators for the method’s application: a moderator, an observer, and two recorders. The dimensions of the OCPS-PS were presented to the participants, who were encouraged to raise and discuss issues related to the topic. The interview was recorded and transcribed. Appendix Aprovides a summary of the terms that emerged during the focus group. Thus, by the end of this initial study, it was possible to construct the first version of the OCPS-PS. For the construction of the scale, we started from the concept of organizational climate, in which the OC is an essential dimension in the work environment, providing a means for investigating organizational behavior through the identification of individual and group behaviors (Patterson et al. 2005). Organizational climate is thus defined as the cognitive appraisal of the practices, policies, and procedures that are recognized and rewarded in the workplace (D’Amato and Zijlstra 2008;James et al. 1990;Jones and James 1979). Therefore, we define the organizational climate in public service as the cognitive evaluation of the practices, policies, and procedures that are recognized and rewarded in public organizations. The Organizational Climate Perception Scale for Public Service (OCPS-PS) thus assesses the perception of organizational climate in public service. Following the extensive literature arguing that organizational climate cannot be measured from a single dimension (D’Amato 2023), the OCPS-PS was constructed from a set of 13 dimensions. Table 1below presents the definitions of the 13 dimensions of the scale, indicates which items comprise each one, and provides the sources of inspiration for the items. Thus, The OCPS-PS assesses the perception of organizational climate in the public service from a second-order construct formed by these dimensions. The list of the 80 items forming the dimensions is found in Appendix B. Table 1. Dimensions, definitions, and items of OCPS-PS. Dimension Definition Items 1Inspiration Autonomy Evaluates the freedom of the employee to make decisions regarding their work. 1Patterson et al. (2005) 2Patterson et al. (2005) 3 Focus group 4 Focus group Interaction and Cooperation Assesses the contribution of interactions and cooperation among employees to the organizational climate. 5Patterson et al. (2005) 6Benzer and Horner (2015) 7Benzer and Horner (2015) 8Jafari et al. (2017) 9Jafari et al. (2017) 10 Focus group 11 Hutchinson et al. (2018) Participation Assesses the employee’s perception of participation in decision making within the organizational environment. 12 Patterson et al. (2005) 13 Patterson et al. (2005) 14 Benzer and Horner (2015) 15 Focus group 16 Focus group 17 Focus group Adm. Sci. 2024,14, 90 5 of 20 Table 1. Cont. Dimension Definition Items 1Inspiration Leadership Support Assesses the employee’s perception of leadership support. Support is understood as providing assistance, encouraging, promoting, recognizing, and stimulating actions and activities that contribute to the organizational climate. 18 Patterson et al. (2005) 19 Focus group 20 Hutchinson et al. (2018) 21 Zappala et al. (2018) 22 Zappala et al. (2018) 23 Focus group 24 Benzer and Horner (2015) 25 Benzer and Horner (2015) 26 Focus group Training Assesses the extent to which the organization promotes training, development, and qualifications that provide employees with opportunities to develop their skills and expertise. 27 Focus group 28 Focus group 29 Patterson et al. (2005) 30 Nikolova et al. (2014) 31 Patterson et al. (2005) 32 Focus group Formalization Assesses the contribution of rules, regulations, rites, and formal procedures to the organizational climate. 33 Patterson et al. (2005); Benzer and Horner (2015) 34 Benzer and Horner (2015) 35 Focus group 36 Patterson et al. (2005) 37 Patterson et al. (2005) Resistance to Change Indicates a lack of openness to change and how much established ways of doing work are valued by the organization. Thus, the greater the tradition, the worse the climate. 38 Patterson et al. (2005) 39 Patterson et al. (2005); Focus group 40 Patterson et al. (2005) 41 Patterson et al. (2005); Focus group Public Value Assesses how much employees perceive their work as having positive impacts on society. Measures the public value of the employee’s work. 42 Focus group 43 Focus group 44 Patterson et al. (2005) 45 Focus group 46 Patterson et al. (2005) 47 Patterson et al. (2005) 48 Focus group Organizational Clarity Evaluates the degree to which objectives, functions, plans, and goals are clearly defined and communicated to the employee. 49 Patterson et al. (2005) 50 Patterson et al. (2005) 51 Hannevik et al. (2014) 52 Focus group 53 Focus group Inefficiency Assesses how lack of productivity, poor task scheduling, deficiencies in the communication process result in inefficiency in the employee’s work. 54 Patterson et al. (2005) 55 Patterson et al. (2005) 56 Patterson et al. (2005) 57 Patterson et al. (2005) 58 Focus group 59 Focus group 60 Focus group 61 Focus group Professional Performance Evaluates the degree to which the employee understands the work, performs it, and is recognized for what they do. 62 Patterson et al. (2005) 63 Zappala et al. (2018) 64 Patterson et al. (2005) 65 Focus group Performance Identifies how employees perceive job performance, evaluations, and feedback. 66 Patterson et al. (2005) 67 Benzer and Horner (2015) 68 Buttner et al. (2012) 69 Focus group 70 Patterson et al. (2005) Adm. Sci. 2024,14, 90 6 of 20 Table 1. Cont. Dimension Definition Items 1Inspiration Diversity Assesses how organizational behaviors and attitudes respect diversity. 71 Focus group 72 Focus group 73 Finsel et al. (2021) 74 Mor Mor Barak et al. (1998) 75 Focus group 76 Mor Mor Barak et al. (1998) 77 Focus group 78 Focus group 79 Focus group 80 Focus group Note: 1The items are described in Appendix B. 3.2. Study 2: Expert Analysis and Pre-Test In order to assess content validity, intelligibility, and relevance of the dimensions and items of the scale, a convenience sample of six experts was selected based on the desired type of experience (scale construction and scale theme) (DeVellis and Thorpe 2021). All experts hold doctoral degrees, are researchers in human resource management with experience in survey research, and three have extensive experience in scale construction. Data collection for observations took place through a Google Forms instrument. For each item in the instrument, questions were presented regarding the item’s relevance (1—Should be removed, 2—Should be kept after reformulation; 3—Should be kept as it is), the degree of relevance (1—Slightly relevant, 2—Relevant; 3—Very relevant), the wording of the item in terms of language (1—Adequate, 2—Inadequate), and the represented dimension. The Content Validity Coefficient (CVC) was calculated for each dimension based on the experts’ responses. Table 2summarizes the results. Table 2. Content Validity Coefficient of the dimensions in the OCPS-PS. Dimension CVC Pertinence Relevance Language Min. Mean Max. Min. Mean Max. Min. Mean Max. Autonomy 0.889 1.000 1.000 0.944 0.944 1.000 0.833 1.000 1.000 Interaction and Cooperation 0.667 0.889 1.000 0.611 0.889 1.000 0.833 0.917 1.000 Participation 0.778 0.889 1.000 0.778 0.833 1.000 0.833 0.917 1.000 Leadership Support 0.833 1.000 1.000 0.833 1.000 1.000 0.917 1.000 1.000 Training 0.889 0.972 1.000 0.889 0.944 1.000 0.917 0.958 1.000 Formalization 0.944 0.944 1.000 0.667 0.833 1.000 0.833 0.917 1.000 Resistance to Change 0.833 0.944 1.000 0.778 0.944 1.000 0.917 1.000 1.000 Public Value 0.778 0.944 1.000 0.833 0.944 1.000 0.833 1.000 1.000 Organizational Clarity 0.778 0.944 1.000 0.833 0.944 1.000 0.917 1.000 1.000 Inefficiency 0.667 0.889 1.000 0.667 0.833 1.000 0.917 1.000 1.000 Professional Performance 0.889 0.889 1.000 0.833 0.889 1.000 0.917 1.000 1.000 Performance 0.889 1.000 1.000 0.889 0.944 1.000 0.917 1.000 1.000 Diversity 0.889 0.889 1.000 0.944 0.944 1.000 1.000 1.000 1.000 OCPS-PS 0.667 0.994 1.000 0.611 0.944 1.00 0.833 1.000 1.000 It is observed that all the average content validity coefficients were above 0.800. However, for the Inefficiency Dimension, it was necessary to remove Item 55 from the scale as it presented a CVC below the recommended threshold of 0.700, as established by Hair et al. (2019). Thus, all 79 items were revised based on the experts’ comments, and a new printed instrument was created for the pre-test. Ten public servants participated in the pre-tests, selected for convenience, with different socioeconomic and demographic profiles. After analyzing the observations from the Adm. Sci. 2024,14, 90 7 of 20 pre-test and making adjustments, the final instrument remained with 13 dimensions and 79 items. 3.3. Study 3: Exploratory Validation For this study, a sample of 394 instruments was collected through an online application among public servants. Exploratory factor analysis was used with a polychoric correlation matrix and estimation by Robust Diagonally Weighted Least Squares (RDWLS). Following the approach of Hair et al. (2019), items with factor loadings below 0.5 were removed from each dimension. The removed items for each dimension were as follows: autonomy, item 4 ( − 0.446); participation, item 16 ( − 0.081), item 15 ( − 0.266), and item 17 ( − 0.469); formalization, item 35 (0.301) and item 33 (0.480); professional performance, item 65 (−0.318). After removal, the results for each dimension of OCPS-PS are presented in Table 3. Table 3. Dimensions, factor loadings, explained variance, parallel analysis, H index, Cronbach’s alpha, and McDonald’s omega for the OCPS-PS. Dimension Item Factor Loading Final Explained Variance (%) Parallel Analysis (Mean of Random) H Index Alpha and Omega Autonomy Item 1 0.958 88.281 97.759 (68.422) 0.953 α= 0.935 ω= 0.935 Item 2 0.937 Item 3 0.831 Interaction and Cooperation Item 5 0.865 59.905 64.716 (29.054) 0.923 α= 0.886 ω= 0.887 Item 6 0.880 Item 7 0.836 Item 8 0.702 Item 9 0.635 Item 10 0.615 Item 11 0.696 Participation Item 12 0.916 89.274 96.417 (67.729) 0.953 α= 0.939 ω= 0.940 Item 13 0.960 Item 14 0.873 Leadership Support Item 18 0.867 79.769 84.995 (84.995) 0.971 α= 0.968 ω= 0.968 Item 19 0.859 Item 20 0.882 Item 21 0.924 Item 22 0.920 Item 23 0.881 Item 24 0.879 Item 25 0.866 Item 26 0.855 Training Item 27 0.531 63.082 67.965 (34.330) 0.906 α= 0.880 ω= 0.883 Item 28 0.771 Item 29 0.819 Item 30 0.816 Item 31 0.827 Formalization Item 32 0.782 72.171 86.722 (68.999) 0.833 α= 0.806 ω= 0.810 Item 34 0.665 Item 36 0.854 Item 37 0.775 Resistance to Change Item 38 0.774 61.094 67.950 (51.230) 0.948 α= 0.780 ω= 0.800 Item 39 0.500 Item 40 0.970 Item 41 0.534 Adm. Sci. 2024,14, 90 8 of 20 Table 3. Cont. Dimension Item Factor Loading Final Explained Variance (%) Parallel Analysis (Mean of Random) H Index Alpha and Omega Public Value Item 42 0.772 67.360 74.198 (29.169) 0.929 α= 0.918 ω= 0.919 Item 43 0.838 Item 44 0.836 Item 45 0.795 Item 46 0.860 Item 47 0.796 Item 48 0.614 Organizational Clarity Item 49 0.786 79.129 81.848 0.951 α= 0.918 ω= 0.919 Item 50 0.942 Item 51 0.892 Item 52 0.851 Item 53 0.881 Inefficiency Item 54 0.596 59.240 65.697 0.905 α= 0.848 ω= 0.854 Item 56 0.716 Item 57 0.537 Item 58 0.886 Item 59 0.826 Item 60 0.763 Item 61 0.749 Professional Performance Item 62 0.935 76.868 87.039 (67.166) 0.905 α= 0.848 ω= 0.854 Item 63 0.765 Item 64 0.729 Performance Item 66 0.712 65.860 76.351 (40.694) 0.877 α= 0.870 ω= 0.870 Item 67 0.737 Item 68 0.756 Item 69 0.751 Item 70 0.833 Diversity Item 71 0.832 66.403 73.971 (20.238) 0.953 α= 0.943 ω= 0.943 Item 72 0.814 Item 73 0.856 Item 74 0.641 Item 75 0.699 Item 76 0.731 Item 77 0.865 Item 78 0.853 Item 79 0.875 Item 80 0.769 Table 3indicates that the factor loadings were high, with all values above 0.5, indicating adequate representability of each item in the dimension. All values from the parallel analysis were higher than the random mean, and all explained variances were high, confirming unidimensionality. Additionally, the H indices were above 0.800, characterizing the possibility of replicating the factors in new studies. The values of Cronbach’s Alpha and McDonald’s Omega were also satisfactory (greater than 0.700), denoting internal consistency for the dimensions. Thus, the third study suggested that the OCPS-PS can be maintained with the 13 dimensions and that 72 items are suitable for its measurement. 3.4. Study 4: Confirmatory Validation In this study, a second quantitative sample was used, making a total of 374 respondents. Table 4lists the results of the initial and final fit indices for each of the dimensions. For constructs in which the initial models were not adequate, the model improvement strategy was adopted, mainly with the removal of variables with low factor loadings. Adm. Sci. 2024,14, 90 15 of 20 Public Value Organization’s performance and its importance; Promotion of activities carried out; Sense of pride/Belonging; Service complaints (Negative Impact); Communicate with society; Defense of public service; Recognition of one’s work; Understanding the entire process; Sense of work; Addressing public issues; Impacts of the activity; Delivery to society; Ownership; Recognition; Delivery to society; Belonging; Organization promotion; Organization’s goals and objectives; Organizational policy; Dissemination of goals and objectives. Organizational Clarity Well-established formal communication; Daily activity versus institutional mission; Participation, involvement in defining; Justice, transparency, appreciation; Equity; Daily activities; Knowledge and priority: everything that is a priority has no ordering; Well-defined formal organizational communication; Communication; Quantity of questions about a particular subject/norms/communications; Frequency with which the organization’s reason for existence is referenced internally; Priorities; Participation; Something tangible; Clear information; How important is my daily activity to my organization; Fulfilling its mission; Setting goals, except for atypical ones, to improve activity execution; Understanding the function one performs, their role as part of the process; Is the development of work transparent, clear? Inefficiency Leadership/managerial support; Planning disconnected from reality; Actions/Measures to prevent inefficiency; Resource scarcity/same workload; Satisfaction; Low self-esteem; Training; Tradition; Lack of clear evaluation mechanisms; Lack of communication within the team; Errors, conflicting/distorted information; Work organization/Ineffective tools; Goal monitoring with direct support from leadership; Lack of work; Well-defined workflows; Rework; Accountability; Employee overload; Resource availability; Delayed deliveries; Organizational clarity. Professional Acting Socialization; Training; Understanding; Clarity; Well-defined goals/Supervisory monitoring; Predefined competencies; Professional recognition/Appreciation; Job description (Restrictive?) versus Performance (Job deviation); Clear deliverables and goals; Possibility of promotions; Professional appreciation; Institutional recognition; Recognition by management of innovative initiatives; Weight of management for recognition; Employee feels capable of performing their job; Adequate development conditions; Mastery of activities; Effective recognition; Sense of belonging; Involvement with work. Perfomance Incentive mechanism; Employed criteria; Continuous assessment; Opportunity for improvement; Familiarity with expectations; Dialogue; Fair evaluation; Clear employment contract; Degree of fairness; Knowledge; Agreement; Change tool; Opportunity to assess and propose changes; Feedback: Reinforcement of the positive; Utilize the performance evaluation process; Performance evaluation; Mastery of activities and effective recognition; Improve or enhance the work structure. Diversity Inclusion; Sensitivity; Understanding; Inclusive management; Preparation and training to correctly address differences; Emotional intelligence to deal with personal differences; Employee perception regarding opportunities for diversity; Adopt effective actions regarding diversity; Inclusive diversity; Technical capability; Professional development; Veiled Decisions/Actions. Appendix B. OCPS-PS Items Dimension Item Autonomy 1. My boss allows me to make relevant decisions about the work I do. 2. My boss allows me to organize how the work is done. 3. I have autonomy to solve problems that arise during the execution of my work. 4. My autonomy in task execution is hindered by the rigidity of the rules. b Interaction and Cooperation 5. Collaboration among employees from different departments is effective. 6. There are cooperative interactions among employees from different departments. 7. The development of activities allows for friendly interactions with other employees. 8. Employees can openly discuss any work-related issues with their supervisors. c 9. There is a harmonious atmosphere among employees in my department. c 10. Informal relationships among employees contribute to the improvement of the organizational climate. c 11. The organization responds quickly to cases of interpersonal conflict. c Adm. Sci. 2024,14, 90 16 of 20 Participation 12. In this organization, employees participate in decisions that affect them. 13. Managers involve employees when changes that affect them are implemented. 14. In this organization, decisions are widely shared. 15. In this organization, the concentration of assigning the same people for activities overburdens the employees who make themselves available to participate. b 16. In this organization, the level of interest of employees in participating in decisions affects the organizational climate. b 17. The political appointment of employees to participate in committees and/or projects reduces my effective possibilities to contribute to the organization’s decisions. b Management Support 18. My boss establishes friendly relationships with subordinates. 19. My boss encourages training and development actions. c 20. My boss promotes measures that encourage the psychological safety of subordinates. 21. My boss recognizes a job well done. 22. My boss is committed to improving working conditions. 23. My boss communicates decisions made widely. 24. My boss encourages the involvement of subordinates in meeting goals. 25. My boss demonstrates trust in their subordinates. 26. My boss makes time available to listen to their subordinates. Training 27. Employees prioritize participation in training that promotes career advancement. c 28. Management encourages participation in training that assists in achieving institutional goals.c 29. Employees are strongly encouraged to develop their skills. c 30. This organization provides sufficient resources for the development of employees’ competencies. 31. In this organization, training and development actions are consistent with the duties of employees’ positions. 32. In this organization, there are clear criteria for supporting training and development actions. Formalization 33. In this organization, work requires adherence to high standards of quality and precision. b 34. In this organization, the existence of formal procedures contributes to improving the work environment. 35. In this organization, the employee’s work is restricted to activities inherent to their position. b 36. In this organization, the formalization of work rules reduces conflicts in the workplace. 37. The prevalence of formality in interactions contributes to maintaining a healthy climate between sections. Resistance to Change 38. In this organization, the way of doing work changes very slowly. 39. In this organization, management is not interested in trying new ideas. 40. In this organization, organizational changes happen very slowly. 41. In this organization, traditional ways of performing activities are valued. c Public Value 42. In this organization, the work performed positively contributes to the development of society. c 43. This organization seeks to address the demands of society in its planning. 44. Employees are concerned with improving ways to serve citizens. c 45. The fact that this organization has positive impacts on society is a source of fulfillment for employees. c 46. In this organization, the needs of society are considered top priorities. 47. This organization responds quickly to the needs of society. 48. I am proud to work for this public organization. c Organizational Clarity 49. Employees have a good understanding of what the organization is seeking to achieve. 50. The organization’s planning is clearly communicated to everyone. 51. In this organization, roles and responsibilities are clearly defined. 52. Employees have clarity about their responsibilities for achieving organizational objectives. 53. In this organization, implemented changes are widely communicated. Inefficiency 54. The way financial resources are spent in this organization is inefficient. c 55. Activities could be done much more efficiently if people took the time to think. a 56. Poor scheduling of activities often results in unmet goals. c 57. Productivity could be improved if job responsibilities were fulfilled. 58. In this organization, the lack of transparency in decisions hinders administrative efficiency. 59. Lack of impartiality contributes to the inefficiency of this organization. 60. In this organization, the internal communication process is inefficient. 61. Poor distribution of the workforce hampers the organization’s performance. c Adm. Sci. 2024,14, 90 17 of 20 Professional Acting 62. In this organization, employees always perform to the best of their abilities. 63. In this organization, employees are familiar with the tools, technologies, and resources available to deliver quality work. 64. In this organization, employees are willing to make an extra effort to do a good job. 65. In this organization, some employees dedicate more effort than others to do a good job. b Performance 66. In this organization, employees always perform to the best of their abilities. 67. In this organization, employees are familiar with the tools, technologies, and resources available to deliver quality work. 68. In this organization, employees are willing to make an extra effort to do a good job. c 69. In this organization, some employees dedicate more effort than others to do a good job. c 70. Employees are encouraged to meet established goals. Diversity 71. In this organization, regardless of position, all employees are respected. 72. In this organization, employees’ conduct adheres to the principle of impersonality. c 73. In this organization, employees feel included. 74. In this organization, sufficient resources and time are invested in diversity-related actions. c 75. In this organization, prejudiced actions are punished. c 76. In this organization, employees feel that there is no prejudice. c 77. In this organization, the principle of equality prevails. 78. In this organization, ethical principles are respected. 79. In this organization, employees are treated fairly. 80. In this organization, employees are prepared to handle diversity. c Legend: removed items (a, study 2; b, study, 3; c, study 4). Appendix C. Correlations between OCPS-PS Errors Correlations Values Significance e43↔e47 0.290 *** e12↔e13 0.362 *** e66↔e67 0.275 *** e70↔e24 0.252 *** e51↔e52 0.453 *** e49↔e52 0.323 *** e51↔e49 0.128 *** e25↔e26 0.194 *** e20↔e24 0.142 *** e21↔e18 0.162 *** e58↔e39 0.234 *** e77↔e12 0.199 *** e53↔e14 0.274 *** e63↔e40 0.333 *** e62↔e46 0.274 *** e2↔e18 0.213 *** e22↔e40 0.309 *** esu↔eaut 0.489 *** ede↔eatu 0.369 *** etra↔einef 0.314 *** Adm. Sci. 2024,14, 90 18 of 20 epar↔einef 0.333 *** eaut↔ediv 0.233 *** efor↔eaut 0.226 *** etra↔eatu 0.257 *** einef↔ediv 0.227 *** etra↔eva 0.183 *** eva↔esu 0.179 *** epa↔eva 0.233 *** esu↔eint 0.182 *** Note: *** Sig < 0.01, significant at 1%. References Andersen, Lotte Bøgh, Andreas Boesen, and Lene Holm Pedersen. 2016. Performance in public organizations: Clarifying the conceptual space. Public Administration Review 76: 852–62. [CrossRef] Babapour Chafi, Maral, Annemarie Hultberg, and Nina Bozic Yams. 2022. Post-pandemic office work: Perceived challenges and opportunities for a sustainable work environment. Sustainability 14: 294. [CrossRef] Banwo, Adeleke Oladapo, Uchechi Onokala, and Bola Momoh. 2022. Organizational climate–institutional environment nexus: Why context matters. Journal of Global Entrepreneurship Research 12: 357–69. [CrossRef] Benzer, Justin, and Margaret Horner. 2015. A meta-analytic integration and test of psychological climate dimensionality. Human Resource Management 54: 457–82. [CrossRef] Boateng, Godfred O., Torsten B. Neilands, Edward A. Frongillo, Hugo R. Melgar-Quiñonez, and Sera L. Young. 2018. Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health 6: 149. [CrossRef] Boye, Stefan, Rebecca Risbjerg Nørgaard, Emily Rose Tangsgaard, Mathilde Andreassen Winsløw, and Mathias Rask Østergaard- Nielsen. 2022. Public and private management: Now, is there a difference? A systematic review. International Public Management Journal 2022: 1–34. [CrossRef] Bozeman, Barry. 1987. All Organizations Are Public: Bridging Public and Private Organizational Theories. San Francisco: Jossey-Bass. Buttner, E. Holly, Kevin B. Lowe, and Lenora Billings-Harris. 2012. An empirical test of diversity climate dimensionality and relative effects on employee of color outcomes. Journal of Business Ethics 110: 247–58. [CrossRef] Byrne, Barbara M. 2010. Structural Equation Modeling with Amos: Basic Concepts, Applications, and Programming. New York: Routledge. Campbell, John P., Marvin D. Dunnette, Edward E. Lawler, III, and Karl E. Weick, Jr. 1970. Managerial Behavior, Performance, and Effectiveness. New York: McGraw. Cronbach, Lee J. 1951. Coefficient alpha and the internal structure of tests. Psychometrika 16: 297–334. [CrossRef] Curvo, Leandro Dias, and Lígia Maria Heinzmann. 2017. Estudo do clima organizacional da secretaria de gestão de pessoas de uma universidade federal. Revista Eletrônica Científica do CRA-PR-RECC 4: 1–18. D’Amato, Alessia. 2023. From research to action and back again: The long journey of organizational climate—A review of the literature and a summative framework. Journal of General Management 2023: 03063070231152010. D’Amato, Alessia, and Fred R. Zijlstra. 2008. Psychological climate and individual factors as antecedents of work outcomes. European Journal of Work and Organizational Psychology 17: 33–54. [CrossRef] DeVellis, Robert F., and Carolyn T. Thorpe. 2021. Scale Development: Theory and Applications. New York: Sage Publications. Do Amaral, Derly Jardim, Andreza Heringer Venicio, Enido Fabiano de Ramos, and Vanessa Scalise. 2005. A influência do clima organizacional na remuneração por competência. Revista Ibero Americana de Estratégia 4: 71–77. Ferrando, Pere J., and Urbano Lorenzo-Seva. 2018. Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educational and Psychological Measurement 78: 762–80. [CrossRef] [PubMed] Ferreira, Patricia I., and Andrea Ramal. 2013. Clima Organizacional e Qualidade de Vida no Trabalho. Rio de Janeiro: LTC. Finsel, Julia, Anne Wöhrmann, Mo Wang, Max Wilckens, and Jürgen Deller. 2021. Organizational Practices for the Aging Workforce: Validation of an English Version of the Later Life Workplace Index. Innovation in Aging 5 S1: 826–27. [CrossRef] Fleiss, Joseph. L. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76: 378. [CrossRef] Fornell, Claes, and David F. Larcker. 1981. Structural equation models with unobservable variables and measurement error: Algebra and statistics. Journal of Marketing Research 18: 382–88. [CrossRef] Hair, Joseph F., William C. Black, Barry J. Babin, and Rolph E. Anderson. 2019. Multivariate Data Analysis. Upper Saddle River: Pearson Education Limited. Hannevik, Martine B., Jon A. Lone, Roald Bjørklund, Cato A. Bjørkli, and Thomas Hoff. 2014. Organizational climate in large-scale projects in the oil and gas industry: A competing values perspective. International Journal of Project Management 32: 687–97. [CrossRef] Adm. Sci. 2024,14, 90 19 of 20 Hinkin, Timothy R. 1995. A review of scale development practices in the study of organizations. Journal of Management 21: 967–88. [CrossRef] Hooper, Daire, Joseph Coughlan, and Michael R. Mullen. 2008. Structural Equation Modelling: Guidelines for Determining Model Fit. Electronic Journal of Business Research Methods 6: 53–60. Hur, Hyunkang, and Gordon Abner. 2024. What makes public employees want to leave their job? A meta-analysis of turnover intention predictors among public sector employees. Public Administration Review 84: 115–42. [CrossRef] Hutchinson, Derek M., Stephanie A. Andel, and Paul E. Spector. 2018. Digging deeper into the shared variance among safety-related climates: The need for a general safety climate measure. International Journal of Occupational and Environmental Health 24: 38–46. [CrossRef] [PubMed] Jafari, Mohammad Javad, Davood Eskandari, Firouz Valipour, Yadollah Mehrabi, Hossein Charkhand, and Mostafa Mirghotbi. 2017. Development and validation of a new safety climate scale for petrochemical industries. Work 58: 309–17. [CrossRef] [PubMed] Jain, Mahima, and Sanjay Dhir. 2022. Antecedents of organization ambidexterity: A comparative study of public and private sector organizations. Technology in Society 70: 102046. [CrossRef] James, Lawrence R., and Allan P. Jones. 1974. Organizational climate: A review of theory and research. Psychological Bulletin 81: 1096. [CrossRef] James, Lawrence R., Carol C. Choi, Chia-Huei Emily Ko, Patrick K. McNeil, Matthew K. Minton, Mary Ann Wright, and Kwang-il Kim. 2008. Organizational and psychological climate: A review of theory and research. European Journal of Work and Organizational Psychology 17: 5–32. [CrossRef] James, Lawrence R., Lois A. James, and Donna K. Ashe. 1990. The meaning of organizations: The role of cognition and values. Organizational Climate and Culture 40: 84. Johansen, Morgen, and Ling Zhu. 2014. Market competition, political constraint, and managerial practice in public, nonprofit, and private American hospitals. Journal of Public Administration Research and Theory 24: 159–84. [CrossRef] Jones, Allan P., and Lawrence R. James. 1979. Psychological climate: Dimensions and relationships of individual and aggregated work environment perceptions. Organizational Behavior and Human Performance 23: 201–50. [CrossRef] Kline, Rex B. 2023. Principles and Practice of Structural Equation Modeling. New York: Guilford Publications. Knies, Eva, Rick T. Borst, Peter Leisink, and Elaine Farndale. 2022. The distinctiveness of public sector HRM: A four-wave trend analysis. Human Resource Management Journal 32: 799–825. [CrossRef] Kolb, David A., Irwin M. Rubin, and James. M Mcintyre. 1986. Psicologia Organizacional: Uma Abordagem Vivencial. São Paulo: Atlas. Lapuente, Victor, and Steven Van de Walle. 2020. The effects of new public management on the quality of public services. Governance 33: 461–75. [CrossRef] Litwin, George H., and Robert A. Stringer, Jr. 1968. Motivation and Organizational Climate. Cambridge: Harvard University Press, Graduate School of Business. Lorenzo-Seva, Urbano, and Pere J. Ferrando. 2019. Robust Promin: A method for diagonally weighted factor rotation. Liberabit 25: 99–106. [CrossRef] McClelland, David C. 1972. What is the effect of achievement motivation training in the schools? Teachers College Record 74: 129–45. [CrossRef] McDonald, Roderick P. 1999. Test Theory: A Unified Treatment. Mahwah: Lawrence Erlbaum. Meier, Kenneth J., and Laurence J. O’Toole, Jr. 2011. Comparing public and private management: Theoretical expectations. Journal of Public Administration Research and Theory 21 S3: i283–i299. [CrossRef] Mor Barak, Michal E., David A. Cherin, and Sherry Berkman. 1998. Organizational and personal dimensions in diversity climate: Ethnic and gender differences in employee perceptions. The Journal of Applied Behavioral Science 34: 82–104. [CrossRef] Mutonyi, Barbara R., Terje Slåtten, and Gudbrand Lien. 2020. Organizational climate and creative performance in the public sector. European Business Review 32: 615–31. [CrossRef] Nikolova, Irina, Joris Van Ruysseveldt, Hans de Witte, and Karen Van Dam. 2014. Learning climate scale: Construction, reliability and initial validity evidence. Journal of Vocational Behavior 85: 258–65. [CrossRef] Patterson, Malcolm G., Michael A. West, Viv J. Shackleton, Jeremy F. Dawson, Rebecca Lawthom, Sally Maitlis, David L. Robinson, and Alison M. Wallace. 2005. Validating the organizational climate measure: Links to managerial practices, productivity and innovation. Journal of Organizational Behavior 26: 379–408. [CrossRef] Pérez-Vallejo, Carlos, and Juan JoséFernández-Muñoz. 2020. Quality of leadership and organizational climate in a sample of spanish workers. The moderation and mediation effect of recognition and teamwork. International Journal of Environmental Research and Public Health 17: 32. [CrossRef] Rainey, Hal G. 2014. Understanding and Managing Public Organizations, 5th ed. San Francisco: John Wiley & Sons Inc. Rizzatti, Gerson. 1995. Análise de Fatores Significativos do Clima Organizacional da Universidade Federal de Santa Catarina: Contribuição para Implantação do Programa de Qualidade. Master’s thesis, Programa de Pós-Graduação em Administração, Universidade Federal de Santa Catarina, Florianópolis, Brazil. Santos, Leda Jung, and Mauricio D. Paranhos. 2017. Os trabalhadores das Equipes de Saúde da Família no Rio de Janeiro: Aspectos da liderança em pesquisa de clima organizacional. Ciência & Saúde Coletiva 22: 759–70. [CrossRef] Sbragia, Roberto. 1983. Um estudo empírico sobre o clima organizacional em instituições de pesquisa. Revista de Administração18: 30–39. Adm. Sci. 2024,14, 90 20 of 20 Schneider, Benjamin. 1975. Organizational climates: An essay. Personnel Psychology 28: 447–79. [CrossRef] Siegel, Saul M., and William F. Kaemmerer. 1978. Measuring the perceived support for innovation in organizations. Journal of Applied Psychology 63: 553. [CrossRef] Sims, Henry P., Jr., and Wiliiam LaFollette. 1975. An assessment of the Litwin and Stringer organizational climate questionnaire. Personnel Psychology 28: 19–38. [CrossRef] Steinfeld, Joshua M. 2023. Stewardship Theory over Agency Theory. Public-Private Stewardship: Achieving Value-for-Money in Public-Private Partnerships. Cham: Springer International Publishing, pp. 123–34. Timmerman, Marieke E., and Urbano Lorenzo-Seva. 2011. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods 16: 209–20. [CrossRef] [PubMed] Torfing, Jacob, Lotte Bøgh Andersen, Carsten Greve, and Kurt K. Klausen. 2020. Public Governance Paradigms: Competing and Co-Existing. Cheltenham: Edward Elgar Publishing. Tripathi, Swati, and Divya Tripathi. 2022. Organizational climate and organizational politics: Understanding the role of employees using parallel mediation. In Evidence-Based HRM: A Global Forum for Empirical Scholarship. Bingley: Emerald Publishing Limited, vol. 10, pp. 241–56. [CrossRef] Van Wart, Montgomery. 2013. Administrative leadership theory: A reassessment after 10 years. Public Administration 91: 521–43. [CrossRef] Wynen, Jan, Jan Boon, Bjorn Kleizen, and Koen Verhoest. 2020. How multiple organizational changes shape managerial support for innovative work behavior: Evidence from the Australian Public Service. Review of Public Personnel Administration 40: 491–515. [CrossRef] Zappala, Salvatore, Vicente Martínez-Tur, and Marco Giovanni Mariani. 2018. Service climate in organizations: Validating the Italian version of the Service Climate Scale (ISCS). TPM-Testing, Psychometrics, Methodology in Applied Psychology 25: 5–20. [CrossRef] Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.