A- A+
Alt. Display

# Cost-Effectiveness of In-House Versus Contracted-Out Vision Rehabilitation Services in England

Research Fellow

## Abstract

Context: Vision rehabilitation (VR) services in England promote users’ health and wellbeing, and support all aspects of daily living through two dominant models: in-house and contracted-out VR services. The two models differ in terms of service delivery, but they share a common aim to enhance service users’ quality of life and reduce utilisation of social and health care services.

Objective: This study investigated the cost-effectiveness of in-house versus contracted-out VR services.

Methods: The analysis was performed from a social care perspective and a social and health care perspective. The analyses used data from a six-month follow-up observational study of VR users. Regression analysis was used to estimate differential outcomes and costs, taking user and local authority characteristics into account.

Findings: At a cost-effectiveness threshold of £13,000 and £30,000 per QALY, in-house VR services have a high probability (greater than 90% vs. contracted-out VR services) of being cost-effective from a social care perspective. In-house VR services have a lower probability (lower than 25% vs. contracted-out VR services) of being cost-effective from a social and health care perspective.

Limitations: Observational studies are prone to selection bias compared to randomised controlled trials due to confounding. We employed econometric techniques that control for several user and LA characteristics to reduce potential bias.

Implications: Contracted-out VR services may be better value for money compared to in-house VR services in the context of integrated social and health care due to substantial healthcare resource savings.

Keywords:
How to Cite: Longo, F., Saramago, P., Weatherly, H., Rabiee, P., Birks, Y., Keding, A. and Sbizzera, I., 2020. Cost-Effectiveness of In-House Versus Contracted-Out Vision Rehabilitation Services in England. Journal of Long-Term Care, (2020), pp.118–130. DOI: http://doi.org/10.31389/jltc.26
Published on 28 Sep 2020
Accepted on 31 Jul 2020            Submitted on 11 Sep 2019

## 1. Introduction

Vision impairment, or sight loss, is particularly prevalent in older people and can impact wellbeing, including activities of daily living (e.g., getting in and out of the bed, ambulating) and mental health (Rabiee et al., 2016). The World Health Organisation (WHO) estimates that no less than 2.2 billion people (i.e., 28.6% of the world population in 2019) live with some form of vision impairment and that at least 45% of these people may have their vision impairment prevented or addressed (World Health Organization, 2018). In 2013, the World Health Assembly produced a global action plan 2014–19 to support the use of more and better prevention and rehabilitation services for individuals with vision impairment across multiple sectors of the economy (World Health Organization, 2013). This plan has stimulated rapid growth of vision rehabilitation (VR) services, which aim to promote independence and to prevent crisis. VR services aim to improve quality of life and reduce the demand for social care and health care services (Rabiee et al., 2015).

Across countries, different models of VR services exist, including clinic-based, community-based, and multidisciplinary VR services (Alma et al., 2012; Rees et al., 2013; Fontenot et al., 2018), and it is not clear which approach offers the best value for money. Clinic-based VR services focus on use of health care services (e.g., eye surgery) to reduce visual impairment, whilst community-based VR services aim to reduce the broader impact of vision impairment on activities of daily living, safety, and social life. Multidisciplinary VR services provide a mix of clinic- and community-based services.

The evidence about the effects of VR services is increasing. A systematic review by Binns et al. (2012) suggested that VR improved clinical and functional ability outcomes; however, clear conclusions were not drawn about its effects on vision-related quality of life, which is a condition-specific outcome (Angeles-Han et al., 2011), nor generic health-related quality of life (HRQoL). Broader outcomes such as generic care-related quality of life (CRQoL), which captures the impact of social care services on quality of life, were not covered (Netten et al., 2011; Netten et al., 2012). Two cost-consequence studies were included in the review. Eklund et al. (2005) explored the effect on perceived level of security and costs of a group-based health education programme for people with age-related macular degeneration compared to an individual-based programme in Sweden. The group-based health education programme increased users’ level of security and reduced costs compared to the individual-based programme. Stroupe et al. (2008) analysed the impact on functional ability and costs of an outpatient rehabilitation programme versus residential rehabilitation for blind veterans in the US. The outpatient rehabilitation programme had a weaker effect on users’ functional ability but also lower costs. Neither study formally examined the cost-effectiveness of interventions. More recently, Rabiee et al. (2015) suggested that community-based VR services may have a beneficial effect on activities of daily living and psychological outcomes but found no evidence on cost-effectiveness.

The present study focuses on community-based VR services in England. However, many other countries, such as Australia (Matti et al., 2011; Rees et al., 2013), Netherlands (Langelaan et al., 2009; Alma et al., 2012), and US (Ekin, 2005; Jones et al., 2009) provide similar services and, therefore, this study might have broader relevance. In the UK, almost two million people (3% of the UK population, 2019) live with vision impairment, and this figure is forecasted to more than double by 2050 (RNIB, 2013). The Care Act 2014 (UK Parliament, 2014) has emphasised the vital role that community-based VR services offer in supporting visually impaired people and in reducing demand pressures on public services such as the NHS. In England, 145 of the 152 local authorities (LAs) with responsibilities for adult social care are known to deliver community-based VR services (Rabiee et al., 2015). One hundred thirty-seven (94%) of these 145 LAs provide VR services through two dominant models which are the focus of this study: in-house (71% of LAs) and contracted-out (23% of LAs) VR services although, in practice, the picture is more mixed (Rabiee et al., 2015: 81).

In-house VR services are funded and delivered directly by LAs, while contracted-out VR services are funded by LAs but they are delivered by external organisations. All individuals having difficulty with activities of daily living related to sight loss are potentially eligible for these VR services and most are free-of-charge (Rabiee et al., 2015).1 Both in-house and contracted-out VR services are goal driven short-term services and their effects are intended to be relatively immediate on VR users’ activities of daily living and independence. Generally, both services consist of specialist vision impairment services such as training in independent living skills (e.g., dressing, cooking), orientation and mobility (e.g., shopping, going out with friends), and use of aids, adaptations, and equipment (e.g., liquid level indicators, talking labeller devices). LAs and external organisations, however, differ in terms of remit. LAs employ teams which often carry out wider sensory impairment work, usually vision and hearing rehabilitation and often additional generic work (e.g., arranging social care, advising on benefits, mental health services) either through the rehabilitation officer or through referral to other LA services, whereas external organisations generally employ teams which solely focus on VR. Therefore, in-house VR services tend to include a broader range of services, while contracted-out organisations tend to allocate more resources to sight loss specific group activities (e.g., social events, leisure activities).

Although there are substantial differences in the delivery of these two models of community-based VR, no evidence exists to inform the choice about which model to invest in. This study investigated the cost-effectiveness of in-house versus contracted-out VR services in England using a six-month follow-up, observational study of VR users. Cost-effectiveness was examined from two perspectives comprising a social care perspective and a social and health care perspective. The latter covered the perspective of social care and health care decision makers combined. This is of interest because, in some areas of England, in-house and contracted-out VR models are provided within integrated social and health care systems.

## 2. Methods

### 2.1. Analysis perspectives

An economic evaluation of in-house and contracted-out VR services was undertaken from the (i) social care perspective and the (ii) social care and health care perspective combined. The social care perspective is relevant because in-house and contracted-out VR services are commissioned by LAs. LAs are key social care decision makers with an interest in the impact of VR on users’ CRQoL, and on the use of other social care services they provide (e.g., residential care, community care). However, the social care and health care perspective is also relevant where LAs and the NHS pay for the service and are, therefore, the shared decision makers. From this perspective, CRQoL and HRQoL are key outcomes and the impact on social and health care resources is central.

### 2.2. Data sources

Data on VR user outcomes, resource use, and characteristics were collected through a survey administered in 2017 at four points in time: baseline and after one, two, and six months. A six-month time horizon was chosen because VR services typically aim to help users achieve their goals in no longer than eleven weeks (less than three months). This six-month study is therefore likely to capture the key effects of VR services, although further longer-term impacts cannot be excluded.

Nine in-house and nine contracted-out VR services were recruited on a self-selecting basis from an invitation sent to the 133 LAs known to deliver either in-house or contracted-out VR services only (see Section 1). VR services typically recruited eligible VR users upon their consent, via service managers. Eligible VR users included those who were newly referred to the service, or existing users if they experienced a major change in their circumstances and needs. Eligible participants were aged 18 and over, living in independent accommodation in the community, were not born blind and were able to speak and understand English. Data on outcomes and resource use were collected through telephone interviews conducted by the research team at baseline and after one, two, and six months. Data on user characteristics were collected at baseline by the VR services. In addition, we collected data on LA characteristics from the UK 2011 Census data (Office for National Statistics, 2012) which were used as additional control variables to account for potential sources of selection bias (see Section 2.5 and 2.5.1 for greater detail).

### 2.3. Social care- and health-related quality of life

To measure CRQoL, we used the Adult Social Care Outcomes Toolkit (ASCOT-SCT4) as recommended by the National Institute for Health and Care Excellence (NICE), for the economic evaluation of social care services (National Institute for Health Care Excellence, 2018). To measure HRQoL, we used the EQ-5D-5L, as also recommended by NICE (National Institute for Health Care Excellence, 2018). Both of these instruments are generic preference-based measures which describe and value care and health states respectively, for use in cost-effectiveness analyses and for making comparisons across interventions where these outcomes are relevant. More details about these instruments are included in Section A1 of the Appendix. To calculate social care quality-adjusted life years (SC-QALYs) using ASCOT, and health QALYS (H-QALYs) using EQ-5D-3L, the area under the curve method was used (Drummond et al., 2015). These scores were multiplied by time in state for three time periods: from baseline to month one (defined as month 1), from month one to month two (defined as month 2), and from month two to month six (defined as month 6).

### 2.4. Resource use and costs

The main analysis focused on the costs of those social and health care services most likely to be used by VR users, and of key interest to social and health care decision makers. The study expert advisory group informed this choice. Costs for the financial year 2016/7 were calculated at baseline, and month 1, 2, and 6 by multiplying the amount of resources used by relevant national unit costs (included in Table A1 of the Appendix). Since the resource use questionnaire included questions about the resource use in the past month (to reduce recall bias), costs at six months were multiplied by four to cover the period from month three to month six.

Social care costs for VR users included the running costs of VR, the cost of LA-provided home care, council day care, social service meals, social services appointments, and befriending and transport services. The running costs of VR were calculated by multiplying the unit cost of VR by the total number of hours of VR (number of sessions by the average length of each session). Health care costs for primary health care service use (GP, nurse, ophthalmologist, and occupational therapist appointments) and secondary health care service use (outpatient appointments, elective, day case and emergency admissions and their length, and emergency telephone line and ambulance services) were calculated.

### 2.5. Analytical approach

In the absence of randomisation of the VR users in this study, comparing unadjusted means across in-house and contracted-out VR services is likely to yield misleading conclusions due to selection bias. This form of endogeneity might arise at the LA level rather than at the user level. This is because LAs might self-select into one of the models of service depending on their demand- and supply-side characteristics. For example, those LAs with greater demand pressures due to a population with greater needs and unable to service such high demand might be more likely to contract-out VR services. In this scenario, service users would have no choice as to the model of VR service used since this decision had been made at the level of the LA. Therefore, if there are systematic differences between VR users this may reflect the LA choice to provide either in-house or contracted-out VR services. The analysis controls for observed factors that may bias the results through several covariates measured at both user and LA level.

#### 2.5.1. Covariates

The key covariate of interest is a dummy indicating whether a user received in-house VR services (reference: contracted-out VR services). To control for potential confounding the analysis controls for individual user and LA characteristics. User characteristics were used to account for demand-side factors across LAs, while LA characteristics captured both LA demand- and supply-side factors. Tables 1 and 2 include the full list of user-level and LA-level covariates, respectively. These covariates were controlled for in all our regressions.

Table 1

Descriptive statistics of user characteristics at baseline.

User characteristic All users In-house Contracted-out In-house vs contracted-out

User Mean User Mean User Mean Diff p-value

VR type (%)
Contracted-out (ref) 230 50.9% 113 0.0% 117 100.0%
In-house 230 49.1% 113 100.0% 117 0.0%
Baseline outcomes and costs (£)
Baseline ASCOT score 226 0.682 111 0.678 115 0.685 –0.006 0.780
Baseline EQ-5D score 230 0.552 113 0.556 117 0.548 0.008 0.804
Baseline social care costs 221 795 108 706 113 880 –174 0.448
Baseline social and health care costs 224 234 110 239 114 229 10 0.836
Age (%)
Older than 65 years old 219 53.4% 105 51.4% 114 55.3% –3.8% 0.570
Gender (%)
Female 230 63.5% 113 64.6% 117 62.4% 2.2% 0.728
Ethnicity (%)
Non-British white 221 10.9% 106 13.2% 115 8.7% 4.5% 0.282
Living situation (%)
Living with someone else 224 62.5% 107 59.8% 117 65.0% –5.1% 0.427
Nature of impairment (%)
Acquired 230 58.3% 113 54.9% 117 61.5% –6.7% 0.305
Congenital 230 2.2% 113 1.8% 117 2.6% –0.8% 0.680
Other 230 13.0% 113 15.0% 117 11.1% 3.9% 0.376
Unknown 230 37.8% 113 40.7% 117 35.0% 5.7% 0.376
Other health conditions (%)
Physical 230 67.8% 113 69.0% 117 66.7% 2.4% 0.702
Sensory 230 17.4% 113 16.8% 117 17.9% –1.1% 0.821
Mental health 230 14.3% 113 21.2% 117 7.7% 13.5% 0.003***
Other 230 9.6% 113 10.6% 117 8.5% 2.1% 0.593
Unknown 230 20.9% 113 19.5% 117 22.2% –2.8% 0.608
Users registered as visually impaired (%)
Registered as blind (ref) 208 38.9% 101 40.6% 107 37.4% 3.2% 0.682
Registered as partially sighted 208 35.6% 101 36.6% 107 34.6% 2.1%
Not registered although visually impaired 208 25.5% 101 22.8% 107 28.0% –5.3%
Previous use of VR (%)
No 205 65.4% 100 63.0% 105 67.6% –4.6% 0.487

VR = vision rehabilitation, User = number of VR users, Diff = mean difference between in-house and contracted-out VR users, p-value = p-value of the mean difference statistical test, ref = reference category.

Descriptive statistics were calculated using the available observations for each variable after excluding three outliers as described in Section 3.1.

To test mean differences, we used a bootstrapped t-test with 1,000 replications for continuous variables and chi-square test for dummy and categorical variables.

Table 2

Descriptive statistics of LA characteristics.

LA characteristic All Las In-house Contracted-out In-house vs contracted-out

Prov Mean SD Min Max Prov Mean Prov Mean Diff p-value

Demand factors
Total population (per 100 K individuals) 18 5.21 3.80 1.98 14.64 9 4.58 9 5.84 –1.27 0.464
Population density (individuals per Hectare) 18 19 24 1 85 9 20 9 18 2 0.856
Proportion of people of age 65 and over 18 15.6 3.9 6.7 21.6 9 15.5 9 15.7 –0.1 0.935
Deprivation
Proportion of no deprived people (ref) 18 41.5 6.1 25.0 49.6 9 38.8 9 44.1 –5.3 0.050**
Proportion of people deprived in one domain 18 32.6 1.6 30.8 38.1 9 33.0 9 32.3 0.7 0.368
Proportion of people deprived in two domains 18 19.8 3.2 15.0 26.0 9 21.3 9 18.3 3.0 0.025**
Proportion of people deprived in three domains 18 5.6 2.0 3.0 10.0 9 6.3 9 4.9 1.4 0.127
Proportion of people deprived in four domains 18 0.5 0.3 0.3 1.3 9 0.6 9 0.5 0.1 0.320
Supply factors
Type of LA (%)
London (ref) 18 11.1% 32.3% 0 1 9 11.1% 9 11.1% 0.0% 0.446
Metropolitan 18 27.8% 46.1% 0 1 9 44.4% 9 11.1% 33.3%
Non-Metropolitan 18 33.3% 48.5% 0 1 9 22.2% 9 44.4% –22.2%
Unitary 18 27.8% 46.1% 0 1 9 22.2% 9 33.3% –11.1%

LA = local authority, VR = vision rehabilitation, Prov = number of VR providers, SD = standard deviation, Diff = mean difference between in-house and contracted-out VR users, p-value = p-value of the mean difference statistical test.

To test mean differences, we used a boostrapped t-test with 1,000 replications for continous variables and chi-square test for dummy and categorical variables.

Data on all LA characteristics were collected from the UK 2011 Census (http://infuse.ukdataservice.ac.uk/).

#### 2.5.2. Econometric specifications

The effect of in-house versus contracted-out VR on user outcomes and costs was estimated using panel data linear regressions:

(1)
${y}_{\mathrm{ijt}}=\mu +\beta {d}_{j}+{\delta }^{\prime }{X}_{i}+{\varphi }^{\prime }{Z}_{j}+{\alpha }_{i}+{\kappa }_{j}+{\tau }_{t}+{\epsilon }_{\mathrm{ijt}}$
(2)
${c}_{\mathrm{ijt}}=\eta +\gamma {d}_{j}+{\theta }^{\prime }{X}_{i}+{\rho }^{\prime }{Z}_{j}+{\omega }_{i}+{\psi }_{j}+{\pi }_{t}+{\xi }_{\mathrm{ijt}}$

where yijt was the outcome and cijt was the cost of patient i (=1,…,I) in LA j (=1,…,18) at time t (=month 1, 2, and 6), μ and η were the intercepts, dj was a dummy taking a value of one if the user located in LA j providing in-house VR services or zero otherwise, Xi was a vector of time-invariant covariates measuring user characteristics (see Section 0), Zj was a vector of user- and time-invariant covariates capturing LA characteristics (see Section 0), αi and ωi captured time-invariant unobserved heterogeneity at the user level (e.g., user’s genetic makeup), κj and ψj captured time-invariant unobserved heterogeneity at the LA level (e.g., sense of community of the local population), τt and πt included year dummies to capture time trends (e.g., seasonal effects), and εit and ξit were the error terms (assumed to be independent from each other).2 We controlled for time-invariant covariates solely, because our analysis focused on a short time window of six months, during which relevant user and LA characteristics were not expected to vary.

We estimated (1) and (2) by the generalised least square (GLS) random-effects (RE) estimator, and we clustered standard errors at the LA level to account for heteroscedasticity and autocorrelation within LAs.3 We also implemented multiple imputation chained equations (MICE), as detailed in Table A2 of the Appendix, and run the analysis under the missing at random (MAR) assumption (Van Buuren et al., 1999; Carpenter and Kenward, 2012). The key coefficient of interest was $\stackrel{^}{\beta }$ in (1) and $\stackrel{^}{\gamma }$ in (2). If then users of in-house VR services had, on average, greater outcomes (costs) compared to users of contracted-out VR services.

#### 2.5.3. Cost-effectiveness analysis

Cost-effectiveness was evaluated using the incremental net health benefit (INHB) as follows (Drummond et al., 2015: 299):

(3)
$\mathrm{INHB}=\stackrel{^}{\beta }-\frac{\stackrel{^}{\gamma }}{\lambda }$

where $\stackrel{^}{\beta }$ and $\stackrel{^}{\gamma }$ are the estimated incremental outcomes (measured in SC- or H-QALYs) and costs from regression (1) and (2), respectively, and λ is the opportunity cost or threshold of the sector under analysis. The INHB captures the overall social care or health gain of an in-house VR service user compared to a contracted-out VR service user. This measures the difference between the social care or health benefits to a user of in-house VR services compared to a user of contracted-out VR services ($\stackrel{^}{\beta }$) and the effect on SC- or H-QALYs to an individual, on average, due to changes in resource use if in-house VR services were used $\left(\stackrel{^}{\gamma }/\lambda \right)$. This effect on SC- or H-QALYs is considered to be worthwhile if in-house VR services reduces costs compared to contracted-out VR services (i.e., if $\stackrel{^}{\gamma }<0$), and assuming that the savings generated could be used at productivity levels λ to improve SC- or H-QALYs of the average individual. In contrast, this effect will be considered detrimental if in-house VR services require more resources compared to contracted-out VR services (i.e., if $\stackrel{^}{\gamma }>0$) at additional cost with equivalent reductions in resources available to utilise for other services which could increase SC- or H-QALYs of the average individual at productivity levels λ. Therefore, for a given λ, a positive INHB implies that in-house VR services are cost-effective compared to contracted-out VR services, while a negative INHB implies suggested that contracted-out VR services are cost-effective. We calculated the probability of cost-effectiveness for multiple values of λ including the health care sector λ of £13,000 (Claxton et al., 2015), and £20,000 and £30,000 per QALY (National Institute for Health Care Excellence, 2018).4 A cost-effectiveness plane was used to illustrate graphically the uncertainty associated with the pair of estimates $\stackrel{^}{\beta }$ from regression (1) and $\stackrel{^}{\gamma }$ from regression (2). Uncertainty was estimated parametrically assuming that $\stackrel{^}{\beta }$ and $\stackrel{^}{\gamma }$ had a multivariate normal distribution. Following Briggs et al. (2006: 95), all estimated parameters in the outcome and cost regression were correlated and 10,000 random draws for both $\stackrel{^}{\beta }$ and $\stackrel{^}{\gamma }$ were generated. Finally, the probability of cost-effectiveness was represented graphically with a cost-effectiveness acceptability curve for a range of λ between £0 and £100,000 per QALY.

#### 2.5.4. Sensitivity analysis

To test the robustness of the results, the analyses were re-run, testing alternative assumptions of the missing data mechanism (Carpenter and Kenward, 2012). First, complete case analysis (CCA) was used which assumes that data were missing completely at random (MCAR). Second, the data were assumed missing not at random (MNAR), that is, the probability that some information was missing depended on the underlying unobserved value even after taking into account the observed data. To identify under which conditions the results might change, multiple MNAR scenarios were examined: first, the data was imputed as above; second, the value of the imputed outcomes and costs were increased, then decreased, up to the point where the results changed. This generated nine MNAR scenarios: the imputed outcome was decreased for (a) in-house and contracted-out VR users, (b) in-house VR users only, and (c) contracted-out VR users only; the imputed costs were increased for (d) in-house and contracted-out VR users, (e) in-house VR users only, and (f) contracted-out VR users only; and the imputed outcome was reduced and the imputed costs increased for (g) in-house and contracted-out VR users, (h) in-house VR users only, and (i) contracted-out VR users only.

Using the multiply imputed data (i.e., under the MAR assumption), a number of additional sensitivity analyses were run. First, whether results changed if in-house VR unit costs were twice or half the contracted-out VR unit costs. Second, whether, under the social and health care perspective, results changed when assuming that the social care sector’s threshold was double or half that of the health care sector. Third, the robustness of our estimated incremental effects to different parametric assumptions were tested by estimating outcomes and costs regressions using RE generalised linear model (GLM) with log link and gamma distribution for outcomes, and a two-part model for costs (where the first part estimated a RE logit regression and the second part estimated a linear model by RE GLM with log link and gamma distribution). Finally, subgroup analyses by age, living situation, nature of impairment and health condition were implemented.

## 3. Results

### 3.1. Descriptive statistics

In total, we recruited 233 VR users.5 From the initial 233 VR users, three VR users were removed as they were substantially different from other VR users in terms of outcomes or costs, that is, those with social and health care costs greater than £30,000 or out-of-pocket costs greater than £4,000. On average, the sample included 13 users per VR site across 18 sites, with the number of users per site varying between three and 27. There were 113 in-house VR users and 117 contracted-out VR users. In-house VR users had on average about eight sessions in eight weeks (i.e., a session per week), and they were expected to achieve their VR goals within eleven weeks. Contracted-out VR users had on average about five sessions in five weeks (i.e., a session per week) and were expected to achieve their VR goals within less than seven weeks. In addition, compared to contracted-out VR users, in-house VR users’ waiting time was on average four weeks longer. Table 1 shows descriptive statistics on VR user characteristics at baseline in the final sample. In-house VR users had a statistically significant higher proportion of people with a mental health condition (21.2% vs. 7.7%). There was no statistical difference between the VR services on other user characteristics. Table 2 includes descriptive statistics on LA characteristics across the 18 VR sites. Sites providing in-house VR services had a statistically significantly lower proportion (5.3%) of people who were not in deprived areas compared to sites providing contracted-out VR services, and a statistically significant higher proportion (3%) of people who were deprived in two out of five domains (where the five domains are employment, education, health and disability, and housing).

Table 3 shows the descriptive statistics on outcomes and costs for the available observations in the final sample. For all VR users, SC-QALY was on average 0.116, with in-house VR users having slightly lower SC-QALY compared to contracted-out VR users (0.116 vs 0.117). For all VR users, H-QALY was on average 0.091, with in-house VR users having greater SC-QALY compared to contracted-out VR users (0.094 vs 0.089).

Table 3

Descriptive statistics of outcomes and costs over six months.

Variable All users In-house Contracted-out In-house vs contracted-out

Obs User Mean Standard deviation Min Max User Mean Std err User Mean Std err 95% CI of mean difference

Ov User-level LA-level

Btw With Btw With

Outcome
SC-QALYs 387 190 0.116 0.091 0.070 0.073 0.019 0.089 0.015 0.333 95 0.116 0.006 95 0.117 0.007 –0.019 0.018
H-QALYs 397 192 0.091 0.080 0.066 0.060 0.023 0.078 –0.053 0.306 94 0.094 0.006 98 0.089 0.006 –0.018 0.027
Costs (in 2017 Pound Sterling)
Social and health care costs 468 208 1,118 2,645 1,654 2,037 658 2,580 0 24,780 101 1,259 179 107 980 167 –301 860
Social care costs 472 208 226 546 412 378 160 533 0 5,814 101 232 32 107 220 39 –103 127
VR costs 473 208 144 403 308 269 153 388 0 5,664 101 150 21 107 139 30 –77 99
Home care costs 485 210 44 306 239 216 65 299 0 3,801 102 41 18 108 47 21 –75 63
Day centre costs 485 210 5 94 48 77 16 93 0 2,072 102 1 1 108 9 9 –25 10
Social worker costs 485 210 8 47 64 15 6 46 0 902 102 7 2 108 9 4 –9 4
Other social care costs 484 210 23 130 113 91 29 127 0 1,474 102 31 11 108 16 5 –16 46
Health care costs 480 210 875 2,471 1,543 1,910 639 2,408 0 22,708 102 1,001 171 108 752 148 –288 785
Primary care costs 484 210 77 144 97 110 31 141 0 1,618 102 89 11 108 65 7 1 47
Secondary care costs 481 210 803 2,399 1,495 1,856 614 2,340 0 22,708 102 914 164 108 694 145 –297 737

Obs = total number of observations, User = number of VR users, Ov = overall, Btw = between, With = within, Std err = standard errors, CI = confidence intervals, VR = vision rehabilitation.

Descriptive statistics were calculated using the available observations for each variable excluding three outliers as indicated in Section 3.1.

Table A3 of the Appendix shows ASCOT and EQ-5D scores at each time point. Overall, social care costs and social and health care costs were on average £226 and £1,118, respectively. Table A4 of the Appendix includes detailed information on resource use. In-house VR users had greater social care costs (£232 vs. £220) compared to contracted-out VR users, and this was mostly driven by differences in VR costs and costs of other social care services including meals, befriending activity and transport. In addition, in-house VR services had greater health and social care costs (£1,259 vs. £980) compared to contracted-out VR users, which reflected higher primary (£89 vs. £65, with this difference being statistically significant at the 5% level) and secondary (£914 vs. £694) health care costs for in-house VR users. The overall standard deviation of the key outcomes and costs under analysis (i.e., SC-QALYs, H-QALYs, social care costs, and social and health care costs) were no less than 77% of the average (for SC-QALYs) indicating substantial variability across VR users. Moreover, both user- and LA-level between-group standard deviations were no less than 21% of the overall standard deviation, and both user- and LA-level within-group standard deviations were no less than 69% of the overall standard deviation. This suggested that there was a substantial between- and within-group variability at both user and LA level which, in turn, warranted the implementation of the panel data methods described in Section 2.5.2. The distributions of total outcomes and total costs were left-skewed and right-skewed, respectively.

### 3.2. Missing data

As reported in Table 4, the proportion of missing data for outcomes and costs was generally low at baseline varying from 0% (EQ-5D-3L score) to 4.3% (social and health care costs). The proportion of missing data increased at month 1 varying between 30.9% (EQ-5D-3L score) and 31.8% (ASCOT score), and at month 2 varying between 33.5% (EQ-5D-3L score) and 36.1% (social and health care costs). Compared to month 1 and month 2, the proportion of missing data at month 6 was slightly lower varying between 25.3% (ASCOT score) and 27.9% (social and health care costs). Table A5 in the Appendix shows that the proportion of missing data for user characteristics was low varying from 0% (gender, nature of impairment, and other health conditions) to 11.2% (previous use of VR). Moreover, in the CCA sample used in the sensitivity analysis, the proportion of missing data further increased varying from 64.8% (EQ-5D-3L score) to 69.1% (social and health care costs). Table 5 shows the reasons for the missing data. Finally, as reported in Table A6 in the Appendix, users with complete data were statistically different from users with incomplete data in terms of age, acquired and unknown impairment, physical, sensory and unknown condition, and visual impairment registration status. This implies that the MCAR assumption might not hold true and, in turn, this warrants the multiple imputation (MI) analysis under the MAR assumption.

Table 4

Missing data for outcomes and costs.

Variable Baseline Month 1 Month 2 Month 6 CCA

Miss Resp Miss Resp Miss Resp Miss Resp Miss User Obs

Outcome
ASCOT score 1.7% 229 31.8% 159 35.2% 151 25.3% 174 66.5% 78 234
EQ-5D-3L score 0.0% 233 30.9% 161 33.5% 155 26.2% 172 64.8% 82 246
Costs
Social care 3.0% 226 31.3% 160 35.6% 150 26.6% 171 67.4% 76 228
Social and health care 4.3% 223 31.3% 160 36.1% 149 27.9% 168 69.1% 72 216

CCA = complete case analysis, Miss = proportion of missing responses, Resp = number of respondents, User = number of VR users, Obs = total number of observations (=number of VR users × 3 time points).

Table 5

Reasons for missing data.

Reason for missing data Baseline Month one Month two Month six

Prop User Prop User Prop User Prop User

Users not reached 0.0% 0 30.0% 70 32.6% 76 26.6% 62
Missed calls 0.0% 0 7.3% 17 9.0% 21 16.3% 38
Unwilling to respond 0.0% 0 6.9% 16 8.2% 19 9.0% 21
Not contactable 0.0% 0 15.9% 37 15.5% 36 0.0% 0
Dead 0.0% 0 0.0% 0 0.0% 0 1.3% 3
Users who responded 100.0% 233 70.0% 163 67.4% 157 73.4% 171
Total amount of users 100.0% 233 100.0% 233 100.0% 233 100.0% 233

Prop = proportion, User = number of VR users, Not contactable = users who could not be reached due to interviewers’ sickness absence.

### 3.3. Cost-effectiveness results

Table 6 reports the incremental effects on outcomes and costs estimated under the MAR assumption (i.e., using the multiply imputed data). In-house VR users had, on average, 0.001 more SC-QALYs and 0.029 more H-QALYs by compared to contracted-out VR services. From the social care perspective, in-house VR users cost £1,792 less, on average. From the social and health care perspective, they cost £3,897 more.6 These differences, however, were statistically insignificant.

Table 6

Incremental effect on outcomes and costs for in-house versus contracted-out VR services.

Variable Obs User Δ SE 95% CI

Outcome
SC-QALYs 690 230 0.001 0.018 –0.035 0.037
H-QALYs 690 230 0.029 0.021 –0.014 0.072
Costs (£)
Social care costs 690 230 –1,792 1,077 –3,914 329
Social and health care costs 690 230 3,897 4,531 –5,037 12,830

Obs = total number of observations; User = number of VR users, Δ = incremental effect of in-house vs contracted-out VR services over the six months under analysis, i.e. $\stackrel{^}{\beta }×3$ in regression (1) for outcomes and $\stackrel{^}{\gamma }×3$ in regression (2) for costs; SE = standard error; CI = confidence intervals.

Analysis conducted under the MAR assumption.

User covariates: VR type dummy, baseline outcome/costs, age dummy, gender dummy, ethnicity dummy, living situation dummy, nature of impairment dummies, other health conditions dummies, vision impairment status dummies, previous use of VR dummy.

LA covariates: total population, population density, proportion of people aged 65 and over, proportion of people deprived variables, type of LA dummies.

Standard errors were clustered at the user level.

Table 7 shows the estimates of the INHB for multiple values of the threshold, i.e., £13,000, £20,000, and £30,000 per SC- or H-QALY. Under the social care perspective, the average INHB was positive for all values of the threshold. The 95% confidence intervals associated with positive values of the INHB, however, suggested that the INHB was never statistically different from zero (at the 5% level). In addition, under the social care perspective, the probability of cost-effectiveness varied between 93% and 94%. The top and bottom of Figure 1 shows the cost-effectiveness plane and the cost-effectiveness acceptability curve, respectively. The probability that in-house VR services were cost-effective compared to contracted-out VR services ranged between 93% for a zero-threshold and 80% for a threshold of £100,000 per SC-QALY.

Table 7

Incremental net health benefit and probability of cost-effectiveness for in-house vs contracted out VR services.

Perspective λ = £13,000 per QALY λ = £20,000 per QALY λ = £30,000 per QALY

INHB 95% CI Prob INHB 95% CI Prob INHB 95% CI Prob

Social care 0.139 –0.027 0.305 94% 0.091 –0.021 0.202 94% 0.061 –0.018 0.140 93%
Social and health care
Using SC-QALYs –0.299 –0.983 0.385 18% –0.194 –0.639 0.252 19% –0.129 –0.427 0.169 19%
Using H-QALYs –0.271 –0.955 0.414 21% –0.166 –0.612 0.280 23% –0.101 –0.400 0.198 25%

λ = marginal productivity; INHB = incremental net health benefit; CI = confidence intervals; Prob = Probability of in-house VR services being cost-effective.

Analysis conducted under the MAR assumption.

Figure 1

Cost-effectiveness plane and acceptability curve under the social care perspective.

Under the social and health care perspective, the INHB was negative for all values of the threshold for both SC- and H-QALYs. Also in this case, confidence intervals suggested that the INHB was never statistically different from zero (at the 5% level). As pictured in Figure 2, the probability of cost-effectiveness for the in-house VR service ranged between 18% and 22% when considering SC-QALYs, and between 19% and 42% when considering H-QALYs. The results of the sensitivity analysis are discussed in Section A2 of the Appendix.

Figure 2

Cost-effectiveness plane and acceptability curve under the integrated care perspective.

## 4. Discussion

To our knowledge, this is the first study that has undertaken a cost-effectiveness analysis of in-house versus contracted-out provision of VR services. The analysis aimed to inform decision makers from a social care and a social and health care perspective. Findings showed that, for a threshold between £13,000 and £30,000 per QALY, in-house VR services have a high probability (about 90% vs. contracted-out VR services) of being cost-effective under a social care perspective. This might be because in-house VR services cover a broader range of rehabilitation activities and link VR users with other LA services more easily. In turn, better links with other LA services might produce a better assessment of need allowing more tailored interventions and a more effective and efficient use of LA resources. Under a social and health care perspective, however, in-house VR services have a much lower probability (about 25% vs. contracted-out VR services) of being cost-effective. We conjecture that a better assessment of needs for users of in-house VR services might also make them more aware of their underlying health conditions (e.g., early symptoms of dementia) increasing their demand for health care services. Indeed, our findings indicate that users of in-house VR services tend to use more hospital services. This suggests that better coordination (or integration) between LA and NHS services might offer larger cost savings and better outcomes for the public sector.

This study contributes to the international literature on the economic evaluation of VR services. Rabiee et al. (2016: 250) suggest that UK in-house VR services may have characteristics in common with those in other countries where VR services feature a multidisciplinary approach. For example, recent guidelines in the US (Fontenot et al., 2018) highlighted the importance of referring visually impaired individuals to multidisciplinary VR services. Such services aim to address the broader impact of vision loss on patients’ lives (e.g., by optimising activities of daily living, safety, and social life) in addition to improve their visual ability. Also studies on VR in other countries, such as Australia (e.g. Rees et al., 2013) and Netherlands (e.g. Alma et al., 2012), highlight the necessity to integrate social and health care VR services through a multidisciplinary approach. In general, our findings are relevant to all those countries providing community-based VR services or a mix of clinic- and community-based VR services.

The current study has some limitations. The gold standard design for cost-effectiveness studies is the randomised controlled trial, which offers the best potential to estimate unbiased intervention effects on outcomes and costs. In this context, however, the large majority of LAs across England already provide VR services, therefore it was not possible to obtain a no VR comparator. Although observational studies are prone to selection bias due to confounding, econometric techniques based on selection on observables were used to control for several user and LA characteristics to reduce the risk of bias. However, unobserved factors may still bias the results. For example, differences in the proportion of users with mental health problems across in-house and contracted-out VR services might suggest that these models serve populations that differ on certain unobserved aspects. To reduce the risk of selection bias further, future observational studies on VR services could consider whether econometric methods based on selection on unobservables (e.g., instrumental variables, difference-in-difference) would be useful. In addition, the proportion of missing data was not irrelevant in this study. We ran the MI analysis and the MAR assumption to address these concerns, but our results incorporate greater uncertainty because the true missing data mechanism remains unknown. Finally, the survey sample used in this study was powered using the VFQ score rather than the outcomes or costs used in the economic evaluation and, therefore, differential effects for these variables might be too small to be statistically detected.

We investigated the cost-effectiveness of the two VR service models under a social and health care perspective rather than a societal perspective. The societal perspective is commonly employed in cost-effectiveness studies in order to account for all possible relevant resources in the economy, although it is unclear who the societal decision maker is. If a societal perspective had been taken, multiple assumptions would have been required. For example, it would have been important to handle differing thresholds across sectors to compute a single cost-effectiveness indicator. Therefore, we used a narrower social and health care perspective as this perspective has relevance to LAs integrated with the NHS. A note of caution about the social and health care perspective is that none of the recruited LAs were fully integrated within the NHS. The results provide insight for informing decision makers; however, it remains unclear as to how these results might transfer across to providers of fully integrated social and health care services. In conclusion, these limitations suggest that our findings should be interpreted with caution and that further research on this aspect of VR services might be relevant.

Appendix

The Appendix includes additional information on outcomes (Section A1) and sensitivity analysis (Section A2, Table A7–A12 and Figure A1). Moreover, it includes more details on unit costs (Table A1), the imputation model (Table A2), and descriptive statistics (Table A3–A6). DOI: https://doi.org/10.31389/jltc.26.s1

## Notes

1Some equipment may be paid for by some users.

2More precisely, αi, ωi, κj and ψj are independent and identically distributed random variables with zero-mean and constant variance. These variables are capturing unobserved factors at user (αi and ωi) or LA (κj and ψj) level impacting the dependent variable and that do not vary over time (Cameron and Trivedi, 2009: 700). For example, each user’s genetic makeup is unobserved but may have an impact on social and health care outcomes. Similarly, the sense of community of the LA’s population may have an effect on social and health care outcomes.

3Our panel data approach and the cross-sectional ordinary least square (OLS) approach analysing the total QALY, calculated over the whole six-month period under analysis, would produce identical results under the assumption of homoscedasticity. The cross-sectional approach, however, is likely to estimate biased standard errors in the presence of autocorrelation over time which, instead, can be taken into account in the panel data model.

4At present, there is no official guidance on the opportunity costs threshold to use when evaluating social care interventions. We therefore employ the thresholds that are generally used in the health care sector as suggested by NICE (National Institute for Health Care Excellence, 2018) or other studies in this field (Claxton et al., 2015).

5We carried out the calculation of the minimal detectable effect size before conducting the survey using the National Eye Institute Visual Function Questionnaire (or VFQ for short), which is a condition specific instrument for measuring outcomes in people with vision impairment. At the available sample size of 230 VR users, we could detect an effect of 0.44 standard deviations difference between in-house and contracted-out VR services on the VFQ scale (ranging from zero for dead to 100 for full health). Using the unweighted VFQ score as an outcome, the estimated incremental effect of in-house vs. contracted-out VR was –0.879 (i.e., 13% of the average) with a standard error of 1.833, that is, the effect size was 0.48 (= 0.879 ÷ 1.833), which is larger than the minimal detectable effect size of 0.44.

6From the social and health care perspective, higher costs for in-house VR users compared to contracted-out VR users are driven by hospital costs.

## Acknowledgements

The study reports on Vision Rehabilitation Services: Investigating the impacts of two service models (IRIS) and it represents independent research funded by the National Institute for Health Research (NIHR) School for Social Care Research (SSCR) over three years (2015–2018). Project reference: C088/CM/UKYB-P79. The views expressed are those of the authors and not necessarily those of the NIHR, SSCR, Department of Health and Social Care, or NHS. Ethical approval was granted in January 2016 from the National Health System, Health Research Authority, Social Care Research Ethics Committee (15-IEC08_0066). We are grateful to members of the IRIS expert advisory group for their suggestions. Moreover, we thank two anonymous reviewers, participants at the 2018 ISPOR conference in Barcelona, 2019 HESG meeting in Norwich, 2019 EuHEA PhD student-supervisor, and ECR conference in Porto, who commented on this study.

## Competing Interests

The authors have no competing interests to declare.

## References

1. Alma, MA, Groothoff, JW, Melis-Dankers, BJ, Post, MW, Suurmeijer, TP and Van Der Mei, SF. 2012. Effects of a multidisciplinary group rehabilitation programme on participation of the visually impaired elderly: A pilot study. Disability Rehabilitation, 34: 1677–1685. DOI: https://doi.org/10.3109/09638288.2012.656795

2. Angeles-Han, ST, Griffin, KW, Harrison, MJ, Lehman, TJ, Leong, T, Robb, RR, Shainberg, M, Ponder, L, Lenhart, P and Hutchinson, A. 2011. Development of a vision-related quality of life instrument for children ages 8–18 years for use in juvenile idiopathic arthritis–associated uveitis. Arthritis Care & Research, 63: 1254–1261. DOI: https://doi.org/10.1002/acr.20524

3. Binns, AM, Bunce, C, Dickinson, C, Harper, R, Tudor-Edwards, R, Woodhouse, M, Linck, P, Suttie, A, Jackson, J and Lindsay, J. 2012. How effective is low vision service provision? A systematic review. Survey of Ophthalmology, 57: 34–65. DOI: https://doi.org/10.1016/j.survophthal.2011.06.006

4. Briggs, A, Sculpher, M and Claxton, K. 2006. Decision modelling for health economic evaluation. OUP: Oxford.

5. Cameron, AC and Trivedi, PK. 2009. Microeconometrics: Methods and applications. Cambridge University Press.

6. Carpenter, J and Kenward, M. 2012. Multiple imputation and its application. John Wiley & Sons. DOI: https://doi.org/10.1002/9781119942283

7. Claxton, K, Martin, S, Soares, M, Rice, N, Spackman, E, Hinde, S, Devlin, N, Smith, PC and Sculpher, MJHTA. 2015. Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold, 19: 1. DOI: https://doi.org/10.3310/hta19140

8. Drummond, MF, Sculpher, MJ, Claxton, K, Stoddart, GL and Torrance, GW. 2015. Methods for the economic evaluation of health care programmes. Oxford University Press.

9. Ekin, DC. 2005. Community-based vision-rehabilitation services provide help and hope. Archives of Ophthalmology, 123: 285–285. DOI: https://doi.org/10.1001/archopht.123.2.285-c

10. Eklund, K, Sonn, U, Nystedt, P and Dahlin-Ivanoff, S. 2005. A cost-effectiveness analysis of a health education programme for elderly persons with age-related macular degeneration: A longitudinal study. Disability and Rehabilitation, 27: 1203–1212. DOI: https://doi.org/10.1080/09638280500052716

11. Fontenot, JL, Bona, MD, Kaleem, MA, Mclaughlin, WM, Jr, Morse, AR, Schwartz, TL, Shepherd, JD and Jackson, ML. 2018. Vision Rehabilitation Preferred Practice Pattern. Ophthalmology, 125: P228. DOI: https://doi.org/10.1016/j.ophtha.2017.09.030

12. Jones, GC, Rovner, BW, Crews, JE and Danielson, ML. 2009. Effects of depressive symptoms on health behavior practices among older adults with vision loss. Rehabilitation Psychology, 54: 164. DOI: https://doi.org/10.1037/a0015910

13. Langelaan, M, De Boer, MR, Van Nispen, RM, Wouters, B, Moll, AC and Van Rens, GH. 2009. Change in quality of life after rehabilitation: Prognostic factors for visually impaired adults. International Journal of Rehabilitation Research, 32: 12–19. DOI: https://doi.org/10.1097/MRR.0b013e3283063503

14. Matti, AI, Pesudovs, K, Daly, A, Brown, M and Chen, CS. 2011. Access to low-vision rehabilitation services: Barriers and enablers. Clinical Experimental Optometry, 94: 181–186. DOI: https://doi.org/10.1111/j.1444-0938.2010.00556.x

15. National Institute for Health Care Excellence. 2018. Developing NICE guidelines: The manual. Manchester, England: National Institute for Health and Care Excellence.

16. Netten, A, Beadle-Brown, J, Caiels, J, Forder, J, Malley, J, Smith, N, Towers, A-M, Trukeschitz, B, Welch, E and Windle, K. 2011. ASCOT adult social care outcomes toolkit: Main guidance v2. 1.

17. Netten, A, Burge, P, Malley, J, Potoglou, D, Towers, A-M, Brazier, J, Flynn, T and Forder, J. 2012. Outcomes of social care for adults: Developing a preference-weighted measure. Health Technology Assessment, 16: 1–166. DOI: https://doi.org/10.3310/hta16160

18. Office for National Statistics. 2012. 2011 Census Data [Online]. Available: http://infuse.ukdataservice.ac.uk/ [Accessed 1 March 2019].

19. Rabiee, P, Bernard, S, Baxter, K and Parker, G. 2016. Community-based vision rehabilitation provision in England. British Journal of Visual Impairment, 34: 248–261. DOI: https://doi.org/10.1177/0264619616658313

20. Rabiee, P, Parker, GM, Bernard, SM and Baxter, K. 2015. Vision rehabilitation services: What is the evidence? Final report.

21. Rees, G, Mellor, D, Holloway, EE, Sturrock, BA, Hegel, MT, Casten, R, Xie, J, Finkelstein, E, Lamoureux, E and Keeffe, JE. 2013. Integrated depression management: A proposed trial of a new model of care in a low-vision rehabilitation setting. Ophthalmic Epidemiology, 20: 321–329. DOI: https://doi.org/10.3109/09286586.2013.787102

22. RNIB. 2013. Sight loss UK 2013: The latest evidence.

23. Stroupe, KT, Stelmack, JA, Tang, XC, Reda, DJ, Moran, DA, Rinne, S, Mancil, R, Wei, Y, Cummings, R and Mancil, G. 2008. Economic evaluation of blind rehabilitation for veterans with macular diseases in the Department of Veterans Affairs. Ophthalmic Epidemiology, 15: 84–91. DOI: https://doi.org/10.1080/09286580802027836

24. UK Parliament. 2014. Care Act 2014. Available: http://www.legislation.gov.uk/ukpga/2014/23/pdfs/ukpga_20140023_en.pdf [Accessed 15 Mar 2019].

25. Van Buuren, S, Boshuizen, HC and Knook, DL. 1999. Multiple imputation of missing blood pressure covariates in survival analysis. Statistics in Medicine, 18: 681–694. DOI: https://doi.org/10.1002/(SICI)1097-0258(19990330)18:6<681::AID-SIM71>3.0.CO;2-R

26. World Health Organization. 2013. Universal eye health: A global action plan 2014–2019.

27. World Health Organization. 2018. Blindness and Vision Impairment [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment [Accessed 10 January 2019].