Are bad health and pain making us grumpy? An empirical evaluation of reporting heterogeneity in rating health system responsiveness

Fiorentini, Gianluca; Ragazzi, Giovanni; Robone, Silvana Working Paper Are bad health and pain making us grumpy? An empirical evaluation of reporting heterogeneity in rating health system responsiveness Quaderni - Working Paper DSE, No. 933 Provided in Cooperation with: University of Bologna, Department of Economics Suggested Citation: Fiorentini, Gianluca; Ragazzi, Giovanni; Robone, Silvana (2014) : Are bad health and pain making us grumpy? An empirical evaluation of reporting heterogeneity in rating health system responsiveness, Quaderni - Working Paper DSE, No. 933, Alma Mater Studiorum - Università di Bologna, Dipartimento di Scienze Economiche (DSE), Bologna, https://doi.org/10.6092/unibo/amsacta/3993 This Version is available at: https://hdl.handle.net/10419/159772 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc/3.0/ ISSN 2282-6483 Are bad health and pain making us grumpy? An empirical evaluation of reporting heterogeneity in rating health system responsiveness Gianluca Fiorentini Giovanni Ragazzi Silvana Robone Quaderni - Working Paper DSE N°933 1 Are bad health and pain making us grumpy? An empirical evaluation of reporting heterogeneity in rating health system responsiveness Gianluca Fiorentini1, Giovanni Ragazzi2 and Silvana Robone1 1 Department of Economics, University of Bologna, Italy 2 Agency for Health Care and Social Services of Emilia-Romagna, Italy Abstract This paper considers the influence of patients’ characteristics on their evaluation of a health system’s responsiveness, that is, a system’s ability to respond to the legitimate expectations of potential users regarding non-health enhancing aspects of care (Valentine et al. 2003a). Since responsiveness is evaluated by patients on a categorical scale, their selfevaluation can be affected by the phenomenon of reporting heterogeneity (Rice et al. 2012). A few studies have investigated how standard socio-demographic characteristics influence the reporting style of health care users with regard to the question of the health system’s responsiveness (Sirven et al. 2012, Rice et al. 2012). However, we are not aware of any studies that focus explicitly on the influence that both the patients’ state of health and their experiencing of pain have on the way in which they report on system responsiveness. This paper tries to bridge this gap by using data regarding a sample of patients hospitalized in four Local Health Authorities (LHA) in Italy’s Emilia-Romagna region between 2010 and 2012. These patients have evaluated 27 different aspects of the quality of care, concerning five domains of responsiveness (communication, social support, privacy, dignity and quality of facilities). Data have been stratified into five sub-samples, according to these domains. We estimate a generalized ordered probit model (Terza, 1985), an extension of the standard ordered probit model which permits the reporting behaviour of respondents to be modelled as a function of certain respondents’ characteristics, which in our analysis are represented by the variables “state of health” and “pain”. Our results suggest that unhealthier patients are more likely to report a lower level of responsiveness, all other things being equal, while patients experiencing pain are more likely to make use of the extreme categories of responsiveness, that is, to choose the category “completely dissatisfied” or the category “completely satisfied”. These results hold across all five domains of responsiveness. Keywords: Health care responsiveness, Health system performance, Reporting heterogeneity, Generalised ordered probit, Self-reported health, Pain. Contact Author: Dr. Silvana Robone, Department of Economics, University of Bologna, P.zza Scaravilli 2, 40126 Bologna, Italy. Email: [email protected] Acknowledgements: This research was funded by the Agency for Health Care and Social Services of Emilia-Romagna. We would like to thank Matteo Lippi Bruni, Cristina Ugolini, Irene Mammi, Teresa Bago d’Uva and the participants at the Health, Econometrics and Data Group Seminar Series (University of York), for their useful comments. JEL Classification: I1, I110, I190, C50, C25 2 1. Introduction In recent years, the concept of responsiveness has been put forward as one desirable measure of the performance of health systems. Responsiveness concerns a system’s ability to respond to patients’ legitimate expectations regarding the non-health enhancing and nonfinancial aspects of health care. “Responsiveness is defined as the way in which individuals are treated and the environment in which they are treated, encompassing the notion of an individual’s experience of contact with the health system” (Valentine et al. 2003a). The concept covers eight dimensions of quality of care, perceived in terms of respect for human dignity and of the interpersonal side of healthcare (Valentine et al. 2009). Human rights make reference to concepts such as respecting patient autonomy and dignity, while the interpersonal nature of care (or “client orientation”) focuses on patient accommodation and the quality of basic amenities (Rice et al 2012). The eight domains chosen to represent responsiveness are as follows: autonomy, choice, clarity of communication, confidentiality of personal information, dignity, prompt attention, quality of basic amenities and access to family and community support. Table 1 provides definitions of these domains. The evaluation of health systems’ responsiveness has become an important, evidencebased means of identifying the strengths and weaknesses of health systems, of appraising their evolution over time, and of informing policy reform (Jones et al. 2011). The importance of this instrument has been confirmed at the international level by the European Ministerial Conference on Health Systems, culminating in the Tallin Charter (2008) which points to the importance that policy makers should place on the evaluation of health systems’ performance (WHO 2008). At the national level, the same has been recently confirmed by recent guidelines published by the National Institute for Health and Care Excellence (NICE), a nondepartmental public body within the UK Department of Health, designed to guide British policy makers in several areas of healthcare. These NICE guidelines specifically indicate the 3 users’ perspective as an instrument with which to evaluate the UK health system (NICE 2012). Health system responsiveness has been investigated both by adopting an international comparison perspective (see, for example, Valentine et al. 2008, Blendon et al. 2003, Robone et al 2011, Rice et al. 2012), and by more fully evaluating this performance indicator at a national level (Puentes Rosas et al. 2006, Pelzer 2009, Njeru et al. 2009, Kowal et al. 2011, Rashidian et al. 2011, Adesanya 2012). Our paper falls within the latter category, since it considers the influence of patients’ characteristics on the evaluation of health system responsiveness using Italian data only. Health system responsiveness is usually measured through the self-evaluations of respondents, which rate their experiences of health systems according to a categorical scale (usually a 5-point scale ranging from “very good” to “very bad”). One common problem is that when individuals are faced with an instrument comprising ordinal response categories, their interpretation of the response categories may systematically differ across populations or populations sub-groups, also depending on their preferences and norms (Rice et al 2010). In such a case, a given level of performance is unlikely to be rated equally by all respondents. This phenomenon has been termed “reporting heterogeneity”. A few studies have investigated how standard socio-demographic characteristics (such as gender or education) may influence the heterogeneity in the reporting of health care users about responsiveness (Puentes Rosas et al. 2006, Sirven 2012, Rice et al. 2012). The findings of such studies show that reporting heterogeneity is an issue in the case of self-reporting on the question of responsiveness. However, we are not aware of any study that specifically focus on the influence that patients’ state of health and experiencing pain have on their reporting behaviour with regard to the matter of responsiveness. Valentine et al. (2003b) represents the only paper we are aware of that considers the influence of patients’ self- 4 reported health on their reporting of responsiveness. However, they only use this relationship as a control in their regression model, and do not specifically investigate the way in which self-reported health affects the reporting behaviour of patients. Sirven et al. (2012) investigate the influence on responsiveness of much more specific health measures than selfreported health, by using a dummy based on the Euro-d scale, which is considered as a standard measure of depression (Dewey and Prince 2005), and a dummy indicating whether the respondent has difficulties with basic activities of daily living (ADL) or instrumental activities of daily living (IADL). Moreover, Sirven et al. (2012) only investigate a few of the responsiveness domains we consider in our analysis (for example, they do not consider dignity, confidentiality or social support). There is evidence in the literature regarding the fact that the experience of pain has a negative influence on patients’ satisfaction with clinical outcomes (Baker et al. 2007), but not on non-clinical outcomes such as responsiveness. Our paper helps to bridge these gaps in the literature by exploring a relationship which no other study has explicitly considered before. Our study uses a representative sample of patients (about 2,500 individuals) hospitalized in the Italian Emilia-Romagna Region. The data were collected by the Agency for Health Care and Social Services of Emilia-Romagna (ASSR) between January 2010 and December 2012. Respondents were asked to rate 5 domains of health system responsiveness (communication, social support, privacy, dignity and quality of facilities). We estimate a generalized ordered probit model (Terza, 1985), an extension of the standard ordered probit model which permits the reporting behaviour of respondents to be modelled as a function of certain characteristics of such respondent, which in our analysis are represented by the variables “state of health” and “pain”. We also control for the standard socio-demographic characteristics of patients, for hospital dummy variables and for certain characteristics of treatment (medicine vs. surgery). 5 Our results suggest that patients in worse health, all other things being equal, are more likely to report a lower level of responsiveness than are those patients in better health, while patients experiencing pain are more likely to use extreme categories when evaluating responsiveness, that is, the categories “completely dissatisfied” or “completely satisfied”. 2. Institutional background The Italian National Health Service (SSN) is based on the principle of the universal coverage and comprehensive insurance of most health risks. It is mainly financed through general taxation, and it provides standard levels of care for the entire population. Central government funds the different Regional Health Services by means of a formula based substantially on a per capita rule, albeit adjusted to take account of certain epidemiological factors. The Regional Health Services allocate funds to Local Health Authorities (LHAs) again on a per capita basis, adjusted once again for the aforesaid epidemiological variables. The LHAs use these resources to fund all health care provided to the population under their responsibility, both through providers under their direct control, and through independent public and private healthcare service providers. In the Emilia-Romagna Region, most providers of hospital care are publicly owned, and less than 10% of all cases are treated by private hospitals. Approximately two-thirds of Italy’s public hospitals are directly managed by the LHAs, while the remaining third are run by independent public bodies (IPH). Patients, often advised by their GPs and/or by specialists, can choose the hospital they wish to be treated at, and public hospital treatment is completely free of charge. In Emilia-Romagna, since 1995 LHAs have funded IPHs within the Region under a contractual mechanism based on a pay-per-case rule, using the Diagnosis Related Groups (DRGs) weighting system. The effects of the pay-per-case rule can be significantly modified 6 due to the implementation of pre-determined constraints, both on the overall financial transfers and on the volume of activities, with reductions in the event of inappropriate hospitalisation. Those hospitals that are directly managed by the LHAs are given targets in terms of the volume of activity, and are mainly financed on a cost-reimbursement basis. Accordingly, both the IPHs and the hospitals run by the LHAs face relatively weak incentives to attract patients when they get near to the upper limits foreseen in their contracts with the Region’s LHAs, while they are more strongly motivated to take care of patients from other regions or other countries, for whom no such upper financial limits are foreseen. Table 1. Domains of responsiveness Autonomy: respect for patients’ views of what is appropriate, and allowing patients to make informed choices. Choice: an individual’s right or opportunity to choose a healthcare institution and health provider, and to request a second opinion and access specialist services when required. Clarity of communication: the offering of a clear explanation to patients and family regarding the nature of the illness, together with details of treatment and of any available options. Confidentiality of personal information: privacy in the environment in which consultations are conducted, and the concept of the privileged communication and confidentiality of medical records. Dignity: the opportunity for patients to receive care in a respectful, caring, non-discriminatory setting. Prompt attention: the opportunity to receive care rapidly in emergencies, or readily with short waiting times in the case of non-emergencies. Quality of basic amenities: the physical environment and services often referred to as ‘hotel facilities’, including clean surroundings, regular maintenance, adequate furniture, sufficient ventilation and adequate space in waiting rooms. Access to family and community support: the extent to which patients have access to their family and friends when receiving care, and the maintenance of regular activities (e.g. the opportunity to carry out religious and cultural practices). Note: Source: Rice et al. (2012). The eight domains of responsiveness are defined by the World Health Organization (see Valentine et al. (2003a) for a full exposition of these domains). The response categories available to respondents are: ‘very good’, ‘good’, ‘moderate’, ‘bad’ and ‘very bad’. 3. Data In order to investigate our research hypothesis, we use a dataset collected by the Agency for Health Care and Social Services of Emilia-Romagna (ASSR) regarding patient satisfaction with the hospital services offered by the Italian National Health Service. The data pertain to a sample of about 2500 individuals who were admitted to nine hospitals located within the Parma and Modena LHAs, the Parma’s IPHs, and to the Orthopaedic Hospital 7 Istituto Ortopedico Rizzoli (Bologna) in 2009 and 2010. The data were collected using a probabilistic approach, so as to guarantee that they could be regarded as a statistically representative sample of the proportion of inpatients treated in those wards where data were collected, and of the population in general in terms of gender. Individuals were interviewed about their satisfaction with several aspects of their interaction with the health system (29 items), which can be considered part of the “traditional” domains of responsiveness. Table 2 shows the correspondence between the “traditional” domains of responsiveness, as defined by the World Health Organization (WHO) and described in Valentine et al. (2003a), and the items comprised in the questionnaire administered by the ASSR. The domains of responsiveness included in the analysis are communication, social support, privacy, dignity, waiting times and the quality of facilities. The response categories were: “completely unsatisfied”, “very unsatisfied”, “unsatisfied”, “satisfied”, “very satisfied” and “completely satisfied”. Since the percentage of respondents claiming they were “completely unsatisfied” or “very unsatisfied” is extremely low (only about 2% of all respondents) we aggregate these two categories. In order to evaluate the appropriateness of the correspondence between the domains of responsiveness as defined by the WHO, and the items of patient satisfaction expressed in the ASSR’s questionnaire, we investigate the psychometric property of “validity” in the case of the latter questionnaire. We focus on “validity” since this property allows to explore the level of homogeneity among the items which form a responsiveness domain, and the unidimensionality of the concept itself as represented by the responsiveness domain (Cortina 1993). Table 2 shows that in the ASSR questionnaire, multiple items are included for each responsiveness domain, with the exception of the domain “social support”, for which only one item is available. Cronbach alphas are the standard tools for measuring validity. A minimum value within the 0.6 to 0.7 range has been suggested for the Cronbach alpha in order to support the validity of the instrument (e.g. Labarere 2001; Steine et al. 2001). Table 3 shows Cronbach alphas computed for the responsiveness domains present in Table 2 (with the exception of waiting times). The values of the Cronbach alphas suggest that all of the responsiveness domains (for which is possible to apply this technique) are greater than 0.85 and lie within the desired range. The “waiting times” domain consists of two items only. Therefore, a Pearson’s correlation appears better suited for the purpose of evaluating validity for this domain. The correlation between the two items appears to be positive and significant, but not very strong (about 40%), suggesting that the internal validity of this domain requires 14 5. Results Table 5 reports the estimated coefficients and related standard errors for standard OPROBIT models, where satisfaction with responsiveness is regressed on the full sets of dummy variables described in the Data Section. The reference individual is male, possesses qualifications lower than a secondary school diploma, or no academic qualifications at all, is married or living with a partner, is not registered for work, lives in the Province where the hospital is based, is treated in the IPH of Parma, and receives some medical treatment. He has never been previously admitted to the ward where he receives treatments, and his current admission was planned in advance. He declares he is in good or excellent health, and is not experiencing pain. The first column of the table presents results for the Communication domain, the second column for Social Support, the third for Confidentiality, the fourth for Dignity, and the fifth for Quality of Facilities. Each column also shows the number of observations used to estimate each regression model, and the values assumed by the Akaike and Bayesian information criteria (AIC and BIC), which are standard measures of fit used for model selection (Green 2008). Table 6 shows the estimated coefficients and standard errors for the GOP model. Since we have established that the categorical variables for responsiveness may take on 5 possible values, then 4 cut-points “divide” the response categories. Therefore, there are 4 cut-point equations in the GOP model. A positive (negative) sign of a coefficient means that the cutpoints tend to shift rightwards (leftwards), and therefore that people have higher (lower) expectations with regard to that level of responsiveness. We choose to model the cut-points as functions of the variable poor/moderate health and pain. This choice, together with the formal statistical test described in the previous Section, also depends on the observation that in Table 5 (OPROBIT model) these variables always display a negative coefficient, which is also statistically highly significant for most of the responsiveness domains. It is quite unlikely that the Emilia-Romagna Regional Health Service is systematically less responsive to patients who are in poor or moderate health, or in pain. Therefore, the negative influence of poor/moderate health and pain on the reported levels of responsiveness is more likely to be due to a bias in the reporting of those patients who find themselves in such states, rather than to a “true” negative responsiveness effect. The first three columns in Table 7 present the results of tests for homogeneity in reporting behaviour for the five responsiveness domains considered. For both poor/moderate health and pain, χ2 statistics and p-values from a Wald test of the joint significance of the 15 estimated coefficients across the four cut-points of each model are reported. Rejection of the null hypothesis indicates that cut-points are functions of the variables in question. In addition to separate tests for each variable, the first column also reports a joint test across the two variables in question. For all domains, the null hypothesis of homogenous reporting can be rejected. The heterogeneous reporting behaviour of individuals appear to depend on both poor/moderate health and pain, with the exception of the domain of Social Support, where poor/moderate health does not appear to have any influence on the reporting style of the respondents (the p-value of the Wald test is 0.07). Columns four and five of Table 7 present the results of tests for parallel cut-point shift for the responsiveness domains in question. We can reject the null hypothesis of a parallel cut-point shift for all combinations of domains and variables, except for the combination poor/moderate health and Social Support (which appears to be characterised by a homogeneous reporting style) and Confidentiality. The Akaike and Bayesian information criteria (shown in Tables 5 and 6) also suggest that the GOP model performs better than the standard OPROBIT model, since their values are lower for each responsiveness domain in the former model. Therefore, we choose to focus on the results obtained using the GOP model. When we consider the main responsiveness equation, the effect on reported responsiveness of being a woman and of having a high level of education is always negative (with the exception of woman for Communication), but seldom statistically significant. Note that in Table 5, the variable high education has a positive effect on responsiveness in 4 out of 5 responsiveness domains, and this effect is almost always statistically significant. Therefore, relying on the standard OPROBIT model rather than the GOP model, could produce misleading results. With regard to the gender variable, in the literature on responsiveness the findings concerning the effects of being a woman on reporting behaviour are not univocal (Sirven et al. 2012). However, if we look at the broader literature examining the question of health status, our results for gender appear to be in line with those of Bago d’Uva et al. (2008). The latter study finds that women usually have higher expectations than men with regard to their health. The effects of occupational status are almost never significant, while those of being single are negative in 4 out of 5 domains and statistically significant in 3 of them. The variables outside the province or outside the region or country always have a positive influence on the reported levels of responsiveness, and for the latter variable this influence is almost always statistically significant. As we indicated in Section 2, this effect could be accounted for by the greater financial incentives for hospitals to treat patients from other regions or countries, as the corresponding revenues are not subject to the usual financial constraints imposed by block 16 contracts in the case of patients from the region itself. Accordingly, hospitals may instruct personnel to be more responsive towards such external patients, in order to attract further patients from outside the region in question by means of reputational mechanisms. Note that the signs of several coefficients for the variables single and outside the province differ from Table 5 to Table 6. Therefore, the standard OPROBIT model could lead to biased results even in the case of these variables. We find no consistent pattern, across domains, in the sign of the coefficients for the hospital dummies, and many of such coefficients are statistically highly significant. Analogously, in the case of the variables surgery and emergency, no consistent pattern in the sign of the coefficients could be found, and in this case most of the coefficients are not statistically significant. The variable previous hospitalization appears to have a negative influence on responsiveness. With regard to the cut-point equations, being in a state of health worse than good, tends to have a negative, statistically significant effect on the reporting of responsiveness compared to being in good or excellent health. This is suggested by the fact that most of the coefficients which are statistically significant possess a positive sign across all cut-points. This holds for all domains, which the exception of Confidentiality, which displays a negative and statistically significant coefficient for the second cut-point. However, as Table 8 shows, we cannot reject the hypothesis of parallel cut-point shift in the case of this domain. When we estimate the GOP model for Confidentiality by making this assumption (results are available on request), the coefficient for poor and moderate health in the cut-point equation is 0.108, and it is statistically significant at the 99% level. Therefore, also in this domain, patients in poorer health appear to have higher expectations with regard to responsiveness. This result is in line with the findings of Sirven et al. (2012), who highlight that the reporting style of respondents is in fact affected by their state of health, since in their study respondents with depressive symptoms systematically report worse health system responsiveness. Across all domains, the variable pain has a positive effect on the first cut-point (the one dividing “completely or very unsatisfied” from “unsatisfied ” with responsiveness), but a negative effect on the last cut-point (dividing “very satisfied ” from “completely satisfied” with responsiveness). This implies that patients in pain tend to opt more often for the extreme categories, that is, they rate responsiveness as either extremely bad or very good more often than do patients who are not in pain. Table 8 reports the marginal effects concerning the GOP model (the increase in the probability of choosing the “completely satisfied” category) and the related standard deviations. The marginal effects have been computed assuming that all the respondents have 17 the reporting style of an individual characterized by average values of health and pain (that is, we use the average values of the cut-points). All the marginal effects are statistical significant, since they have a p-value of less than 0.001. The magnitude of the effects of socio-economic variables on the evaluation of responsiveness is not particularly high, since the marginal effects for these variables vary from about 0.5% to about 5%. Similar results hold for the variables surgery, emergency and previous hospitalization, since their marginal effects vary from about 1% to about 3%. Marginal effects appear to be larger for outside the province and outside the region or country (which vary from about 1% to about 13%, with several higher than 5%), and for the hospital dummies (which vary from about 1% to 31%, with several higher than 5-10%). 18 Table 5: Ordered probit model for self-reported responsiveness satisfaction with responsiveness Communication Social Support Confidentiality Dignity Quality of Facilities b/se b/se b/se b/se b/se woman -0.025 0.029 0.01 -0.013 -0.047** 0.019 0.056 0.04 0.032 0.016 high education 0.153*** 0.014 0.109* 0.124*** -0.084*** 0.022 0.064 0.045 0.037 0.019 work -0.046* 0.074 -0.023 0.062 0.014 0.023 0.069 0.049 0.04 0.02 single -0.043* 0.028 -0.001 -0.062 -0.027 0.02 0.057 0.04 0.033 0.017 outside the province -0.015 -0.006 0.054 0.007 -0.006 0.035 0.100 0.072 0.059 0.03 outside the region or country 0.361*** 0.487*** 0.391*** 0.393*** 0.252*** 0.033 0.099 0.068 0.057 0.028 hospital dummy 2 0.214*** 0.17 0.247** 0.403*** 0.181*** 0.047 0.114 0.081 0.068 0.034 hospital dummy 3 -0.220*** 0.188 -0.264** -0.215** -0.468*** 0.043 0.128 0.090 0.072 0.037 hospital dummy 4 0.172 0.007 0.262 0.501** 0.180* 0.103 0.283 0.201 0.167 0.086 hospital dummy 5 0.479*** 0.284** 0.386*** 0.334*** 0.174*** 0.034 0.098 0.069 0.057 0.029 hospital dummy 6 -0.563*** -0.472*** -0.569*** -0.588*** -0.542*** 0.041 0.119 0.084 0.066 0.035 hospital dummy 7 -0.179** -0.398 -0.169 0.063 0.019 0.068 0.207 0.146 0.116 0.060 hospital dummy 8 0.530*** 0.857*** 0.700*** 0.358*** 0.503*** 0.054 0.160 0.114 0.092 0.047 hospital dummy 9 -0.106** -0.142 -0.180** -0.129* 0.056* 0.032 0.095 0.067 0.054 0.028 surgery -0.061* 0.058 -0.112* -0.096* 0.023 0.027 0.080 0.057 0.046 0.023 poor/moderate health -0.467*** -0.266*** -0.421*** -0.446*** -0.300*** 0.020 0.058 0.041 0.033 0.017 emergency -0.142*** -0.107 -0.05 -0.100** -0.021 0.022 0.065 0.046 0.037 0.019 pain -0.113*** -0.06 -0.037 -0.102** -0.113*** 0.02 0.059 0.041 0.034 0.017 previous hospitalization -0.084*** -0.150* -0.129** -0.032 -0.155*** 0.021 0.062 0.044 0.035 0.018 cut-point 1 constant -2.614*** -2.504*** -2.538*** -2.679*** -2.334*** 0.046 0.139 0.096 0.081 0.037 cut-point 2 constant -2.072*** -1.916*** -1.976*** -2.207*** -1.707*** 0.041 0.117 0.083 0.07 0.033 cut-point 3 constant -0.391*** -0.147 -0.288*** -0.768*** -0.235*** 0.036 0.106 0.075 0.062 0.031 cut-point 4 constant 0.389*** 0.563*** 0.467*** 0.166** 0.562*** 0.037 0.106 0.076 0.061 0.031 aic 32507.499 3834.510 7709.923 11558.322 45942.591 bic 32680.936 3958.781 7850.087 11708.282 46122.137 N13915 1641 3275 5014 18149 Note: *** p<0.001, ** p<0.01, * p<0.05 . 19 Table 6: Generalized ordered probit model for self-reported responsiveness satisfaction with responsiveness Communication Social Support Confidentiality Dignity Quality of Facilities b/se b/se b/se b/se b/se woman -0.068** -0.035 -0.09 -0.03 -0.114*** -0.024 -0.072 -0.05 -0.043 -0.02 high education 0.031 -0.111 -0.009 -0.044 -0.148*** -0.027 -0.081 -0.057 -0.049 -0.022 work -0.082** 0.039 -0.023 0.026 0.011 -0.03 -0.09 -0.062 -0.056 -0.024 single -0.065** 0.025 -0.035 -0.093* -0.061** -0.024 -0.072 -0.051 -0.044 -0.02 outside the province 0.013 0.15 0.248** 0.018 0.036 -0.044 -0.127 -0.092 -0.081 -0.036 outside the region or the country 0.408*** 0.122 0.280** 0.276** 0.142*** -0.047 -0.14 -0.095 -0.089 -0.036 hospital dummy 2 0.079 0.08 0.268* 0.195 0.149*** -0.061 -0.147 -0.106 -0.102 -0.041 hospital dummy 3 -0.277*** 0.258 -0.196 -0.193* -0.462*** -0.052 -0.162 -0.108 -0.093 -0.042 hospital dummy 4 0.441*** 0.113 0.187 0.839** 0.192 -0.127 -0.341 -0.262 -0.273 -0.104 hospital dummy 5 -0.174*** -0.02 0.008 -0.225* -0.142*** -0.05 -0.135 -0.096 -0.089 -0.036 hospital dummy 6 -0.404*** -0.395** -0.378*** -0.377*** -0.410*** -0.047 -0.14 -0.097 -0.081 -0.039 hospital dummy 7 -0.141 -0.235 -0.104 0.243 0.290*** -0.077 -0.232 -0.17 -0.151 -0.068 hospital dummy 8 0.099 0.734** 0.459** 0.043 0.168** -0.074 -0.237 -0.165 -0.135 -0.061 hospital dummy 9 -0.073 -0.2 -0.091 -0.168* 0.127*** -0.04 -0.118 -0.082 -0.072 -0.033 surgery -0.028 0.068 -0.109 -0.072 0.048 -0.035 -0.103 -0.074 -0.065 -0.028 emergency -0.106*** -0.036 0.05 -0.007 0.036 -0.028 -0.081 -0.058 -0.05 -0.023 previous hospitalization -0.02 -0.07 -0.068 0.099* -0.101*** -0.026 -0.077 -0.054 -0.047 -0.021 cut-point 1 poor/moderate health 0.171* 0.443 0.672** 0.287 -0.083 -0.083 -0.278 -0.242 -0.191 -0.055 pain 0.359*** 0.404 0.653** 0.406 0.216*** -0.092 -0.238 -0.201 -0.23 -0.058 constant -2.858*** -3.106*** -3.565*** -3.047*** -2.491*** -0.109 -0.311 -0.285 -0.283 -0.062 cut-point 2 poor/moderate health 0.048 -1.444 -0.812* -0.063 -0.176 -0.261 -0.807 -0.389 -0.546 -0.134 pain 0.166 -0.081 -0.109 0.092 0.429** -0.33 -0.355 -0.245 -0.708 -0.163 constant -1.539*** -0.672 -0.253 -1.198 -1.498*** -0.382 -0.472 -0.347 -0.862 -0.157 cut-point 3 poor/moderate health 0.313** -0.083 -0.072 0.343 0.290*** -0.096 -0.273 -0.184 -0.206 -0.055 pain -0.11 0.038 -0.418* -0.019 -0.134* -0.096 -0.305 -0.2 -0.203 -0.057 constant -0.653*** -0.438 -0.149 -0.843*** -0.437*** -0.113 -0.286 -0.196 -0.249 -0.055 cut-point 4 poor/moderate health 0.008 0.109 0.106* -0.008 0.134*** -0.023 -0.065 -0.046 -0.048 -0.019 pain -0.171*** -0.227*** -0.198*** -0.235*** -0.155*** -0.023 -0.065 -0.047 -0.049 -0.019 constant 0.749*** 0.793*** 0.723*** 0.658*** 0.552*** -0.021 -0.057 -0.042 -0.047 -0.017 aic 17856.523 2058.335 4202.625 5580.336 28608.692 bic 18075.204 2215.024 4379.353 5769.416 28835.077 N13915 1641 3275 5014 18149 Note: *** p<0.001, ** p<0.01, * p<0.05 20 Table 7: Tests of homogenous reporting and parallel cut-point shift chi2(8) p chi2(4) p chi2(4) p chi2(3) p chi2(3) p Communication 297.6 0.00 207.6 0.00 68.6 0.00 22.0 0.00 68.5 0.00 Social Support 20.8 0.01 8.7 0.07 12.4 0.01 4.2 0.25 12.0 0.01 Confidentiality 45.9 0.00 21.5 0.00 25.4 0.00 6.9 0.07 22.2 0.00 Dignity 113.8 0.00 81.4 0.00 26.1 0.00 8.7 0.03 26.0 0.00 Quality of Facilities 300.6 0.00 191.1 0.00 90.5 0.00 32.8 0.00 88.8 0.00 test for homogeneous reporting test for parallel cut point shift All poor/moderate health pain poor/moderate health pain Table 8: Marginal effects related to the generalized oprobit model (increase in the probability of choosing the category “completely satisfied” compared to the base category) and the related standard deviations coeff st dev coeff st dev coeff st dev coeff st dev coeff st dev woman -0.019 0.003 -0.010 0.002 -0.027 0.004 -0.010 0.001 -0.036 0.005 higher education 0.009 0.001 -0.033 0.005 -0.003 0.000 -0.014 0.002 -0.047 0.007 work -0.023 0.004 0.012 0.002 -0.007 0.001 0.008 0.001 0.004 0.001 single -0.018 0.003 0.008 0.001 -0.010 0.001 -0.030 0.003 -0.019 0.003 outside the province 0.004 0.001 0.046 0.007 0.080 0.009 0.006 0.001 0.011 0.002 outside the region or country 0.129 0.014 0.037 0.005 0.090 0.010 0.094 0.008 0.046 0.007 hospital dummy 2 0.023 0.004 0.024 0.004 0.088 0.010 0.066 0.006 0.049 0.007 hospital dummy 3 -0.071 0.013 0.082 0.011 -0.056 0.009 -0.059 0.008 -0.124 0.019 hospital dummy 4 0.145 0.018 0.034 0.005 0.060 0.007 0.313 0.012 0.063 0.009 hospital dummy 5 -0.047 0.009 -0.006 0.001 0.003 0.000 -0.069 0.009 -0.043 0.007 hospital dummy 6 -0.100 0.017 -0.102 0.017 -0.101 0.015 -0.109 0.014 -0.113 0.017 hospital dummy 7 -0.038 0.007 -0.064 0.011 -0.030 0.005 0.084 0.007 0.098 0.012 hospital dummy 8 0.029 0.005 0.259 0.022 0.157 0.014 0.014 0.002 0.055 0.008 hospital dummy 9 -0.020 0.004 -0.056 0.009 -0.027 0.004 -0.052 0.006 0.041 0.006 surgery -0.008 0.001 0.020 0.003 -0.033 0.005 -0.023 0.003 0.015 0.002 emergency -0.030 0.005 -0.011 0.002 0.015 0.002 -0.002 0.000 0.011 0.002 previous hospitalization -0.006 0.001 -0.020 0.003 -0.020 0.003 0.032 0.003 -0.031 0.005 Quality of Facilities Communication Social Support Confidentiality Dignity Note: all the marginal effects have a p-value <0.001 21 6. Discussion This paper considers the influence of patients’ characteristics on the evaluation of health system responsiveness, by using a sample of about 2500 in-patients collected by the ASSR between January 2010 and December 2012. The dataset we use in our analysis is not large, but it has never been explored before, and the richness of the information it contains allows us to investigate original research questions. In particular, we focus on the influence that both the state of health of the in-patients and their experiencing pain have on their reporting behaviour with regard to the question of the health system’s responsiveness. Our results suggest that patients in poorer health have a tendency to report a lower level of responsiveness than do patients in better health, all other things being equal, while patients experiencing pain are more likely to make use of the extreme categories of responsiveness, that is, the “completely dissatisfied” or “completely satisfied” categories. When we look at the magnitude of the influence on responsiveness of the regressors included in our econometric model, the aforesaid influence appears to be particularly strong and statistically significant for the hospital dummies (the probability of choosing the response category “completely satisfied” is between 1% and 31% higher, compared to the base hospital category). This empirical result suggests that an investigation of which hospital characteristics might have a “true” influence on responsiveness could be a fruitful direction for future research. Studies making international comparisons, for instance, have pointed to a positive correlation, at national level, between high health expenditure per capita, high educational qualifications and high levels of responsiveness (World Health Report 2000, Anderson G., Hussey P. 2001, Blendon et al. 2001, Valentine et al.2003b, Robone et al. 2011). Therefore, at a more micro level, it might be interesting to investigate the relative strength of various supply-side factors affecting the responsiveness of the health system, such 22 as the level of hospital staff training (in particular, their training in managing interpersonal relationships), and the level of spending on non-clinical facilities or the latest technology. We are able to adjust the dependent variables for the presence of reporting heterogeneity by estimating a generalised ordered probit model (GOP). Several recent studies addressing the issue of reporting heterogeneity when investigating responsiveness have employed the hierarchical ordered probit model (HOPIT) and have taken advantage of anchoring vignettes (see, for example, Valentine et al. 2003b, Sirven et al. 2012, Rice et al. 2012). “Vignettes represent hypothetical descriptions of fixed levels of a latent construct such as health system performance. Since the vignettes are fixed and pre-determined, any systematic variation across individuals in the rating of the vignettes can be attributed to differences in reporting behaviour” (Rice et al. 2010) and can be utilized to adjust selfreported experiences of health system performance to enhance comparability across individuals (Bago d’Uva et al. 2008). We cannot make use of the HOPIT model in our paper, since vignettes are not available in the dataset collected by the ASSR. From a more general viewpoint, the inclusion of vignettes within the survey necessarily entails a significant increase in the cost of data collection, which is not always affordable for the institutions implementing it. Therefore, the use of the GOP model – which does not require the employment of vignettes - can be considered as a second-best strategy. Despite the identification restriction of excluding each covariate from either the cut-point equation or the main responsiveness equation, the GOP model permits adjustment for reporting heterogeneity in those cases where vignettes are not available. In our analysis, correcting for reporting heterogeneity has proved extremely important in order to obtain unbiased results in the main responsiveness equation (for certain regressors, the results obtained using the OPROBIT model vary considerably from the results obtained using the GOP model). Therefore, the use of techniques capable of addressing the 23 issue of reporting heterogeneity has to be suggested to policy makers for sound policy recommendations to be formulated.