To support efficient resource allocation in health and social care, many countries have systems that promote evidence-based practice through national guidelines, led by health technology assessment (HTA) or Guideline agencies. National guidelines typically include systematically developed statements on intervention effectiveness that seek to assist practitioners, managers and commissioners in their decisions about delivering or funding appropriate care to users of care in specific circumstances (Grimshaw et al., 2004). The body responsible for developing evidence-based guidelines on health, public health and social care in England is the National Institute for Health and Care Excellence (NICE). Whilst originally mandated to develop clinical guidelines, NICE’s remit was broadened in 2012 to encompass social care. It is the first national HTA/Guideline agency to develop social care guidelines, therefore offering a unique learning opportunity for other health and care systems. To date, NICE has developed approximately 20 adult social care guidelines (Table 1), with each guideline encompassing between 50 and 100 recommendations. Recommendations vary widely with regard to their scope, ranging, for example, from those that describe over-arching principles of care to those focused on specific interventions.
|Publication date||Title||Guideline programme||Titles of economic evidence reviews and economic analysis||Type of economic analysis (as outlined in the ‘economic plan’)|
|2015||Dementia, disability and frailty in later life – mid-life approaches to delay or prevent onset (NICE, 2015a)||Clinical and public health||Evidence review 3 economic modelling report||No economic plan for additional analysis economic modelling report is titled ‘Costs and benefits of increasing physical activity to prevent the onset of dementia’|
|2015||Older people: independence and mental wellbeing (NICE, 2015d)||Clinical and public health||Evidence review 5; Health economic analysis||Cost-utility and cost-consequences analyses of various wellbeing interventions (arts; internet and computer training; friendship programme; school-based volunteering)
Perspective: Partial societal
|2015||Challenging behaviour and learning disabilities: prevention and interventions for people with learning disabilities whose behaviour challenges (NICE, 2015a)||Social care||Appendices S,T; Appendix W||Cost-utility analyses
of psychosocial interventions (parent training), pharmacological interventions (risperidone aripiprazole) and intervention for sleep problems (psychological intervention, melatonin, combination therapy)
Cost perspective not stated in economic plan
|2015||Transition between inpatient hospital settings and community or care home settings for adults with social care needs (NICE, 2015e)||Social care||Appendix C1;
|Costs savings analysis of geriatric assessment and care vs. usual care; geriatric assessment and care included specialist care provided in a specialist unit; usual care involved the admission to a general ward under non-specialist care;
Costs perspective: National Health Service (NHS) and Personal Social Services (PSS); impact of including unpaid care on cost savings (wider societal perspective) explored
|2015||Older people with social care needs and multiple long-term conditions (NICE, 2015c)||Social care||Appendix C1; Appendix C3||Cost-utility analysis of geriatric assessment and care planning intervention (Geriatric Resources for Assessment and Care of Elders ‘GRACE’ model)
Cost perspective: National Health Service (NHS) and Personal Social Services (PSS)
|2015||Home care: delivering personal care and practical support to older people living in their own homes (NICE, 2015b)||Social care||Appendix C1; Appendix C3||Cost-effectiveness of social care service packages that include home care
Cost perspective: Publicly-funded healthcare and personal social services (NHS and local authorities)
|2012, updated 2016||Autism spectrum disorder in adults: diagnosis and management (NICE, 2016a)||Clinical and public health||Appendix 18||No economic plan for additional analysis|
|2016||Mental health problems in people with learning disabilities: prevention, assessment and management (NICE, 2016b)||Clinical and public health||Appendix S||No economic plan for additional analysis|
|2016||Transition between inpatient mental health settings and community or care home settings (NICE, 2016b)||Social care||Appendix C1; Appendix C3||Costs utility analysis of various interventions for people in early stages of bipolar disorder, vs. generic outpatient treatment; interventions include: 2-year multi-staged psychological intervention; guideline-based pharmacological treatment; 3-staged psychological intervention; 6 weeks manual-based psychoeducational intervention Costs perspective: National Health Service (NHS) and Personal Social Services (PSS)|
|2017||Intermediate care including reablement (NICE, 2017)||Social care||Appendix C1; Appendix C3||Cost-savings analyses of nurse-led, bed based intermediate care and (home care) reablement vs. standard care;
Cost perspective: National Health Service (NHS) and Personal Social Services (PSS)
|2018||Decision-making and mental capacity (NICE, 2018b)||Social care||Appendix C1; Appendix C3||Cost-utility analysis of advance care planning
Cost perspective: National Health Service (NHS) and Personal Social Services (PSS)
|2018||Care and support of people growing older with learning disabilities (NICE, 2018a)||Social care||Appendix B;
|Cost-utility analysis of annual health checks
Cost perspective: National Health Service (NHS)
|2018||Learning disabilities and behaviour that challenges: service design and delivery (NICE, 2018d)||Social care||Appendix C2, Appendix C3||Cost-utility (threshold) analysis of additional respite care
Cost perspective: National Health Service (NHS) and Personal Social Services (PSS)
|2018||Dementia: assessment, management and support for people living with dementia and their carers (NICE, 2018c)||Clinical and public health||Appendix E; Appendix J||Cost-utility analyses of various non-pharmacological interventions that support cognitive functioning, functional ability or wellbeing (including cognitive stimulation, reminiscence, exercise, music, occupational therapies or training)
Cost perspective: National Health Service (NHS) and Personal Social Services (PSS)
|2019||End-of-life care for adults in the last year of life: service delivery (NICE, 2019a)||Clinical and public health||Included in all evidence review documents; Economic appendix||Costs-savings (threshold) analyses of out-of-hours services and models, and various end-of-life community services that reduce inappropriate admission, and/or support people stay in their preferred place of death
Cost perspective: National Health Service (NHS) and Personal Social Services (PSS)
|2020||Supporting adult carers (NICE, 2020h)||Social care||Included evidence review documents (A to I)||Cost-utility and cost-minimisation analyses of replacement care
Cost perspective: Societal
|Guidelines in development|
|To be completed 2021||Safeguarding adults in care homes (NICE, 2020g)||Clinical and public health||Only evidence review H (8) includes economic evidence Appendix J||Cost-utility analyses of face-to-face training and e-learning
Cost perspective: Social care
|To be completed 2022||Integrated health and care for people who are homeless through being roofless (NICE, 2020d)||Clinical and public health||Not yet available||Not yet available|
|To be completed 2022||Adults with complex needs: social work interventions including assessment, care management and support (NICE, 2020a)||Social care||Not yet available||Not yet available|
|Not yet known||Advocacy services for adults with health and social care needs (NICE, 2020b)||Social care||Not yet available||Not yet available|
Recommendations made by NICE need to be informed by evidence not only on what works but also on what is considered ‘good value for money’. The important role of economic evidence in NICE guidelines goes back to its beginnings in 1999, when NICE was created with the specific purpose to ensure a more equal distribution of healthcare and health outcomes under resource constraints. Since then, economic evaluations have played a key role in informing recommendations (Dakin et al., 2015), providing information about the added benefits from an intervention relative to its costs, in comparison with standard care, using value-for-money thresholds that allow consistent decision making across various areas of health (Ciani & Jommi, 2014; Taylor et al., 2004). (These thresholds are used to inform decisions about recommendations. If, for example, an intervention that is being appraised is more effective than standard practice but also more expensive, it is not immediately clear whether – on economic grounds – to recommend its widespread use. Having a threshold provides a reference point to guide discussions about whether the higher costs are justified by the better outcomes. We discuss cost-effectiveness thresholds later in the paper.)
In some countries, technology appraisal of this kind is used to control upward pressure on prices of care technologies, particularly drugs, and to promote affordable and appropriate pricing consistent with scientific and social judgements (Sorenson, Drummond, & Kanavos, 2008). Cost-effectiveness assessments conducted as part of HTAs, using decision-analytic modelling techniques informed by real-world data or assumptions, have also played an important role in generating evidence about potential costs and outcomes in actual practice (Makady et al., 2018).
Guideline-driven evidence-based practice is newer to social care than to the clinical field (Gould, 2010; Specht, 2013). In part, this is because of challenges of generating evidence on what works for heterogeneous, relatively small populations and contexts (Aaron & Enola, 2003; Matthew Owen & Jeffrey, 2003). Whilst social care is difficult to define, in England, it typically includes personal care, safeguarding services, and support for people with needs arising from illness, disability, old age or poverty (Weatherly et al., 2017). Delivered through a local ‘mixed economy’ of statutory and non-statutory providers and funding sources (Wistow et al., 1994), content, coverage and quality of social care can vary considerably between localities depending also on economic, financial and political contexts. Agreeing and implementing consistent approaches to care is therefore both important and challenging.
Historically, social care evidence developed in unstructured ways that are experience-based rather than experiment-based (Gould & Kendall, 2007). Therefore unsurprisingly, there are many fewer (economic) evaluations in social care than in health care (Tinelli et al., 2020). However, the role of national social care guidelines in facilitating evidence-based practice has more recently received attention by some governments, such as in England (Gould & Kendall, 2007; Leng, 2019) and Canada (Beauchamp, 2015). In contrast to what is known about the process and impact of clinical guidelines (Sorenson, Drummond, & Kanavos, 2008), including the positive role they can play in increasing adherence to evidence-based practice and improving patient outcomes (Farrar, Tuffnell, & Sheldon, 2020; Hassan et al., 2005; Hawley et al., 2016), there is limited research describing and analysing the guideline development in social care. A recent national review in England found that few care home staff were aware of the NICE social care guidelines (Leng, 2019), highlighting the need for more information about how they are developed and why they are valuable.
In this paper, we describe the NICE approach to development of national social care guidelines, specifically how economic evidence is reviewed, analysed, synthesised and presented to inform practice-focused recommendations. In doing so, we seek to support the social care sector’s understanding of, and capacity to engage with and make use of economic evidence. While NICE guidance refers to England, it is widely recognised as of wider international relevance, and referenced as a useful resource for health and care practitioners internationally (Anderson et al., 2018).
We take the following approach to providing this overview. First, in the section ‘NICE social care guidelines: history and development process’, we describe who has been developing the guidelines, involving which processes, and the types of guidelines produced thus far. Next, in the section ‘Economic evidence reviews for NICE social care guidelines’ we outline how topics are selected (‘Scoping’), how economic evidence is identified, analysed (‘Searching for, screening and critical appraisal of economic evidence’) and synthesised to inform recommendations of guidelines (‘Economic evidence synthesis and interpretation’). Since an important type of evidence used to inform recommendation is produced during the guideline development process in the form of de novo economic analysis, we describe this step in detail.
From 2012 to 2017, social care guidelines were produced by the NICE Collaborating Centre for Social Care (NCCSC), a partnership of five organisations led by the Social Care Institute for Excellence, commissioned by NICE (Social Care Institute for Excellence, 2020). Partners comprised the EPPI-Centre at University College London, Research in Practice, Research in Practice for Adults and the Care and Policy Evaluation Centre (CPEC; formerly Personal Social Services Research Unit) at the London School of Economics and Political Science. The CPEC team was responsible for reviewing and synthesising economic evidence and conducting economic analysis. Since 2018, the National Guideline Alliance (NGA), based at the Royal College of Obstetricians and Gynaecologists, has developed NICE social care guidelines.
As mentioned, at least twenty guidelines relevant to adult social care were developed or are currently in development. Exact numbers are difficult to determine as some of the social care-relevant guidelines were developed under clinical and public health programmes, such as those for dementia or autism.
The process of developing guidelines, including the role of reviewing economic evidence, is set out in the NICE methods manual (NICE, 2020c). A variety of evidence is considered, consisting of qualitative studies, quantitative studies including cost-effectiveness evidence and expert testimonies. Studies are systematically reviewed by a technical team and, guided by independent topic expert committees (the ‘guideline committee’), transformed into practice recommendations. Structured guideline development group meetings – scheduled to take place every 6–8 weeks, on average – form the backbone of this process. These meetings – which are chaired by a subject matter expert and held in accordance with NICE processes – involve the committee assessing the evidence emerging from the reviews and agreeing its reliability and usefulness for informing recommendations.
As in the clinical and public health fields, guideline development in social care includes an assessment of economic value (‘Is it worth it?’), in addition to assessment of value (‘Does it work?’) (Leng, 2019). Other considerations are also taken into account, such as equity and feasibility of implementation (NICE, 2020e). For example, for social care guidelines, an important part of the review process includes eliciting evidence on social care users’ and carers’ experiences so that recommendations reflect acceptability, accessibility and choice.
In terms of eliciting economic evidence, the guideline development involves two main steps: data from systematic reviews and additional economic analysis in the form of modelling based on published data; both types inform recommendations. NICE sets out a reference case for how review and economic analysis methods should look, which we will describe in more detail in the next chapter.
The process of developing a social care guideline can be lengthy (up to 24 months, depending on its scope) and involves multiple organisations and individuals from diverse backgrounds working in close collaboration (Figure 1). Whilst there are many activities involved, from consultation to revision, quality assurance and sign-off, the main focus of this paper is on the development stage, conducted primarily by the technical team and the guideline committee.
The technical team, based at the NGA or another developer organisation, reviews evidence and presents it to the guideline committee in accessible format. The team includes information specialist(s), reviewer(s) and economist(s) who collaborate with NICE and the guideline committee, and are responsible for: supporting the committee, and documenting their recommendations, discussions and decisions; conducting evidence reviews and syntheses; and ensuring appropriate methods are used. The guideline committee is an independent advisory group that ultimately co-authors the guideline (NICE, 2020c). Committee members are appointed following a voluntary application process and include practitioners, commissioners, service providers, people with lived experience and carers, and researchers. They meet regularly (every 1.5 months) over the course of the guideline development process (c.18 months). At those meetings, the committee prioritises and interprets evidence presented to them by the technical team, and develops recommendations for practice, taking account of stakeholder views. Registered stakeholders, that is, organisations that want to be involved in the process, can apply to attend a workshop that informs development of the scope, which is then published for consultation (NICE, 2020i). Stakeholders can also register at any time during the development, including during the final consultation period. Draft guidance is published for formal consultation. In this way, a large and diverse group of people can influence final recommendations.
Table 1 presents an overview of NICE adult social care guidelines and the economic evidence reviews and analyses that were conducted as part of those. This includes guidelines already published and those in development under different guideline programmes. A commentary on a subset of those is available on the Economics of Social Care Compendium (ESSENCE) website (Tinelli et al., 2020).
An important part of the development process is the review of evidence, which includes review of economic evidence. There are a number of steps. The scoping of the guideline involves deciding the extent of the area and subject matter to be addressed: this includes formulating review questions, including questions of cost-effectiveness, and identifying priorities for any original economic analyses as part of guideline development. Searching of literature includes development of search strategies, which are then applied to electronic databases to identify relevant (economic) studies. Screening involves decisions about which (economic) studies should be included, based on predefined inclusion and exclusion criteria. Critical appraisal includes assessment of relevance and quality of included (economic) studies. Evidence synthesis refers to summarising findings into evidence statements. Finally, the analysis step includes interpretation of evidence and formulation of recommendations. The guideline development process also involves generating new evidence in the form of additional economic analysis. In addition to a wide range of other evidence on effectiveness and views and experiences, the guideline committee will, therefore, use two main sources of cost-effectiveness evidence: 1) economic evaluation studies that are published, identified and summarised in the reviews; and 2) additional economic analyses. More detail on those two methods is provided below.
As mentioned, the review of evidence starts with development of a scoping document for the topic area, including the rationale for looking at the topic, aims of the guideline, areas to be covered, and draft review questions. Areas covered by the guideline, and which are subject to evidence reviews, are defined by populations, interventions and comparators, practice approaches, settings and outcomes. The scope for a new topic generally includes 10–15 review questions, although there can be far fewer (3–4 questions), depending on the topic in question and the overall timeframe for delivery. Review questions in social care relate to effectiveness, cost-effectiveness, views and experiences of service users, carers, practitioners in relation to interventions, or aspects of service delivery. Every question about effectiveness is accompanied by a question about cost-effectiveness (e.g., ‘What is the effectiveness and cost-effectiveness of crisis-response interventions?’).
NICE offers different approaches for identifying and synthesising published evidence. Working with the guideline committee, the technical team can make decisions about which types of evidence to prioritise for inclusion, set out in the systematic review protocol, found on the NICE website. For example, they might decide to consider or exclude studies from particular countries, or consider studies published only after a certain year, particularly if there have been major recent service reforms. Whilst in clinical guidelines the main source of (cost-)effectiveness evidence is usually taken from randomised controlled trials, in the social care field a wider set of study designs is more common and appropriate. This might include studies in which the control group (i.e., the group receiving ‘standard’ care) is not randomised, or sometimes even studies with no control group (e.g., high-quality observational studies, in which typically large number of people are followed over time and where the receipt of a particular intervention can be linked to outcomes).
In terms of the type of economic evaluation conducted, for clinical topics (and many public health topics) cost-utility analysis is most common. In cost-utility analysis, the difference in total costs between the two study groups is compared with the between-group difference in health-related quality of life (expressed in quality-adjusted life years). For social care topics, other types of economic evaluations are more common and are considered in the review process. This includes cost-savings, cost-effectiveness and cost-consequences studies (explained below). The guidelines ‘Transition between inpatient mental health settings and community or care home settings’ and ‘Intermediate care including reablement’ offer examples of the types of cost-savings analyses not uncommon in the social care field, which investigate costs savings from NHS and/or social care perspective (Table 1).
In addition, studies might be reviewed if they only measure a partial set of costs or outcomes. For example, studies that measure important aspects of service use and costs can be helpful in adding to the evidence for interventions already known to work. If they can demonstrate potential cost savings or no difference in costs, some conclusions might be drawn by the committee about the likely cost-effectiveness. For all economic studies included in the review, data extraction is conducted by the economist. Economic evidence tables are downloadable for each guideline from the NICE website. For included economic studies, applicability and quality are assessed on a range of criteria: whether all relevant costs and outcomes were included; design and time-horizon of the study; relevance to the review question or the guideline more generally. Checklists for assessing relevance and quality of economic studies are available in the NICE manual (NICE, 2020c).
As mentioned earlier, in addition to considering evidence from reviewed studies, the guideline committee considers findings from any de novo economic analyses conducted by the economist in the technical team. NICE uses what it calls an ‘economic plan’, a document that outlines areas in which additional economic analyses will be carried out, prioritising two or three review questions for modelling. The economist, together with the guideline committee, drafts the economic plan during the early stages of guideline development. An abbreviated version of the plan can be found on the NICE website under the respective guideline. NICE requests that additional economic analyses should be carried out for areas of interventions or service delivery likely to have greatest impact on costs and benefits. In social care, the final choice of areas for economic analyses is strongly influenced by availability of data to carry out additional analyses. NICE provides guidance on prioritising economic areas in its manual. A detailed description of the additional economic analysis can be found on the NICE website as part of the evidence reviews. Features of the additional economic analyses in social care and associated challenges are described below.
Types of analyses. The economic analysis typically uses decision-analytic modelling, such as a decision tree or Markov modelling (Squires & Tappenden, 2011), although other methods are sometimes used too (in particular, when data that are available do not permit use of such a model). In modelling, the pathway of ‘events’ likely to occur for people who receive an intervention (e.g., a particular care or support package) is compared to the pathway for those who do not. An example of an ‘event’ is the admission to a care home. Costs (and outcomes) are then attached to model states (Squires & Tappenden, 2011). In comparison to the clinical field, a lot less evidence and data are available for social care topics. Additional economic analysis might therefore involve relatively simple modelling techniques. In addition, more assumptions need to be made to fill data gaps. The guideline committee has an important role in informing assumptions about model parameters, and on filling data gaps. Other experts might be contacted by the technical team to inform assumptions.
Population and intervention coverage. For each analysis, the population and intervention are specified. The population is described in terms of age group, gender, setting, health or social care condition. The intervention needs to be funded by the public sector with a social care focus and can either benefit the person using services or their carer (NICE, 2020c). The intervention is typically an innovative or existing intervention that is not yet considered ‘standard care’. The comparator refers to one or more interventions routinely delivered by the social care sector.
Time horizon. The time horizon refers to the number of months or years covered by a model. Whether a short- or long-term model is more appropriate depends on the expected impact on costs and outcomes. The aim of a model is to capture all costs and outcomes relevant to the decision. Modelling utilises data from various published sources. Typically, a wider range of relevant costs and outcomes can be included in the analysis than would be feasible through primary data collection carried out for a single study. This includes extrapolation of costs and outcomes known or expected to occur as a result of the intervention beyond a typical study period. An example is costs and outcomes linked to an ongoing need for home care, which might be influenced by interventions that seek to restore or maintain a person’s independence (e.g., reablement).
Perspective on costs. An important decision for the economic analysis concerns the types of costs that should and can be included. This decision refers to the perspective of the analysis, and it has an important influence on findings and recommendations. The focus of NICE is on costs to the health and social care sectors, but other costs might be included as part of sensitivity (scenario) analysis if relevant. In social care, the contribution of unpaid care is often important. Carer time and skills represent resources to which costs can be attached. Hours of unpaid care typically make up a high proportion of the overall care provided for someone using social care (Brimblecombe et al., 2018). However, until recently few evaluations have measured hours of unpaid care and therefore there is limited knowledge about the impact of interventions on this key resource. In addition, there is currently no agreed approach as to how unpaid care should be costed in monetary terms (indeed carer time can be represented on the benefit side), which complicates comparisons of these costs across evaluations.
Methods for considering outcomes. Whilst for clinical and public health guidelines, only or primarily health effects can be considered, in social care there is no such restriction and all benefits as they are experienced by people using services or their carers can be included. In social care guidelines, the consideration of user and carer outcomes and experiences is particularly important (Bauer et al., 2020) and can influence the need for adapting processes of guideline development such as how evidence is reviewed, and selection of guideline committee members (NICE, 2018e). The decision about which outcomes are included in the analysis determines the type of economic evaluation. If outcomes are expressed in the form of quality-adjusted life years (QALYs), which combines quantity and quality of life in a single measure, then it is called a cost-utility analysis. For clinical guidelines this is the type of analysis that NICE expects, whilst in social care other types of economic evaluations are also accepted. However, a cost-utility analysis needs to be conducted as the ‘base-case’ (=main) analysis (NICE, 2020c). Additional analysis can include cost-effectiveness analysis, in which costs are compared against outcomes presented in ‘natural units’ (e.g., independence in activities of daily living), cost-consequences analysis, which evaluates several outcomes alongside the costs, cost-benefit analysis, which converts outcomes into monetary values, or cost-minimisation analysis, which only considers costs expended versus costs saved (Drummond et al., 2015).
Different from clinical and public health guidelines, NICE allows the inclusion of capability or social care-related quality of life measures such as the ICEpop CAPability measure (ICECAP) and Adult Social Care Outcome Toolkit (ASCOT) in economic analysis, which have the potential to be used as an alternative to EQ-5D or other measures that allow transformation into QALYs. Two factors prevent their wider use at the moment: measures are still relatively new compared to their counterparts in the health field, and therefore for many topic areas no economic evaluations exist which use those outcome measures, which means that there is a limited evidence base on which NICE’s economic analysis could draw (although this is changing). There is currently no recognised threshold against which to compare cost-per-outcome-unit estimates that would allow deriving conclusions about whether a (positive) change in outcome due to an intervention is large enough to justify their additional costs.
One important parameter for undertaking modelling is the size of change in outcomes in the intervention versus control group. This is often called the effect size, and it strongly depends on population characteristics, service infrastructure, and other contextual factors. For example, effects of interventions that seek to reduce care home admissions might be different depending on how many care homes are available in the locality, or the availability of home care and unpaid care. In the clinical field, there are studies that synthesise findings on effects across different regions or subgroups and carry out analysis to adjust for some of the above factors, but this kind of evidence rarely exists in social care contexts, mainly because there are too few studies. It is thus important to identify an evaluation carried out in a setting that is sufficiently similar to the one relevant to the national context.
Sources of data to inform the analysis. Modelling tends to rely on data already published rather than requiring new data collection. The economist typically carries out pragmatic searches to identify relevant literature. Various types of data are required for modelling; these include data on costs, service use, unpaid care, prevalence or incidence for health or social care conditions or events (e.g., number of people admitted to care homes) and mortality. Data might be taken from: national statistics; national sources for costs of health and social care activity such as the Unit Costs of Health and Social Care (Curtis & Burns, 2019) and NHS Reference Costs (NHS Digital, 2020); national audits or registries; databases of service activity, performance and finance data routinely collected by local authorities. Where data are lacking, expert views, including from the guideline committee, can inform the estimates.
Sensitivity analysis. In economic evaluations, and in particular in decision-analytic modelling, there are various sources of uncertainty, and so additional sensitivity analysis is carried out to investigate the impact of these uncertainties in model inputs on the final results. Different approaches are possible, including deterministic sensitivity analyses (e.g., scenario, threshold, one-way or multi-way sensitivity analysis) and probabilistic sensitivity analysis (Drummond et al., 2015).
Discounting and inflation. Costs and outcomes need to be discounted (e.g., at 3.5%) when the analysis covers time periods longer than one year because the general tendency of individuals (and society as a whole) is to prefer to enjoy benefits sooner rather than later, and to incur costs later rather than sooner. On the cost side, for example, money can be invested and generate interest, which means that it is rational to prefer having a certain amount of money now than being given the exact same amount in the future. In addition, costs need to be inflated if they refer to an earlier price year to reflect increase in salaries and prices over time.
Economic evidence statements are summaries of economic evidence reviewed and of any additional economic analysis carried out. They present the detail of the study findings, together with interpretations based on study strengths, limitations or characteristics. They conclude with whether an intervention is likely to be cost-effective (for a given population). Social care guidelines published before 2019 include economic evidence statements in the full guideline, and from 2019 these are presented in the separate evidence reviews. An example of an evidence statement is shown in Box 1, together with considerations that were made (for the same guideline) by the committee for review questions for which no economic evidence had been identified (Box 2).
Box 1: Example of an economic evidence statement on advance care planning (from NICE social care guideline ‘Decision-making and mental capacity’ NG108, consultation draft, December 2017, p325).
‘There is a large amount of economic evidence that Advance Care Planning for people reaching end of life can reduce the costs of hospital care. The quality of evidence is mixed and refer to a wide range of different settings; most studies were from the US. No conclusions can be drawn about total costs or outcomes from those studies. A systematic review of economic evidence (Dixon et al., 2015 +) summarized findings on identified cost savings in the majority of economic evaluations of advance care planning; they found that those primarily referred to reductions in hospital use; this ranged from USD 64,827 for the terminal hospital stay to USD 56,700 for total healthcare costs over the past 6 months for people with dementia and USD 1,041 in hospital costs over the last week of life for those with cancer; however, neither individual health and wellbeing outcomes nor other costs – including those of community health and social care and those from a societal perspective (unpaid care, out-of-pocket expenditure) – were not captured so that no final conclusions could be drawn about cost effectiveness. A systematic review of economic evidence (Klingler et al, 2016 +) summarized findings on cost savings identified in the majority of economic evaluations; most individual studies measured hospital costs but did not include a comprehensive cost perspective; cost savings ranged from USD 1,041 to USD 64,830; studies which evaluated programme costs were relatively small amounting to 6 to 15% of cost savings. A single cost-effective study (Abel et al 2013, +) found that individuals in a hospice setting who used advance care plans spent considerable less time in hospital in their last year of life (IG 18.1 vs. CG 26.5 days, p < 0.001); mean cost of hospital treatment during the last year of life for those who died in hospital was £11,299, those dying outside of hospital £7,730; MD 3,569; p < 0.001.’
Box 2: Example of economic considerations by the guideline committee (from NICE social care guideline ‘Decision making and mental capacity’ NG108, consultation draft, December 2017; p293).
‘The Guideline committee discussed if there were cases where involvement of an Independent Mental Capacity Advocate would make things more effective and it was suggested by some that there had been a definite improvement in change of accommodation decisions. It was noted that whilst the quality of decision-making might improve it could lead to higher care costs; however, there were also substantial improvement in quality of life. It was suggested that the involvement of Independent Mental Capacity Advocates leads to better decision making and ensures that decision makers have a better understanding of benefits and burdens, and enhances compliance with the best interest process. More generally, the Guideline committee thought that since most recommendations were required by law, monies were already being spent. However, by recommending what they consider good practice, the Committee thought this would this might help to increase cost-effectiveness as outcomes were likely to improve while costs potentially remain at similar levels. In particular they thought by getting it right at every stage of the process (from Decision making, Independent advocacy, Assessing capacity, Best interest decisions, Deprivation of Liberty Safeguards and the Court of Protection), this would ensure that practice adhered to the law and that unlawful actions were prevented. This could prevent costly scrutiny by the Court of Protection if that case would otherwise have been brought to them. The Committee referred to evidence from the Cardiff Report of Welfare Cases that showed that the average cost of a personal welfare case was £13,000 and that this estimate was likely to reflect the lower end. The Committee thought that if the recommendations in the Guideline were used correctly, there was likely to be less reason for complaints processes or legal hearings as a result of disputes and other objections. However, the Committee thought that it was important to emphasise that this must not discourage court applications to determine complex and significant decisions, i.e., those with impact on Article 8 rights or those on end-of-life issues that required judicial decisions. Those were part of good practice and could not be prevented. However, the Committee thought by following the recommendations in the Guideline there would be a reduction in unnecessary and avoidable applications thus creating capacity for the courts to deal with matters that warrant their attention. The best way to achieve such increases in capacity was to ensure that professionals were legally literate and apply the Mental Capacity Act lawfully. The Committee thought that improved practice as a result of practice that followed the recommendations reduced the need for financial ‘end loading’ of expensive complaints and legal processes and led to better health and social care outcomes of people.’
Findings from economic evaluations are often presented as an incremental cost effectiveness ratio (ICER), which is the difference in costs for people who used the intervention (compared with those who did not) divided by the additional gain in a particular outcome (e.g., health- or social care-related quality of life) between the two groups.
If the outcome is health-related quality of life (used in cost-utility analysis), then the ICER is expressed as cost per QALY gained. The advantage of presenting findings in this way is that it can make it easier to make a decision about the cost-effectiveness if the HTA body has already agreed a cost-effectiveness threshold value representing the health opportunity cost (i.e., the QALYs that could have been generated elsewhere from the same resource). NICE has a threshold of between £20,000 and £30,000 per QALY. Interventions below this range are more likely to be recommended by NICE than interventions above the range. NICE does not have a threshold for any other outcome measure.
If the outcome is presented in natural units, the ICER refers to cost per unit change for this particular outcome (e.g., activities of daily living). To decide whether an intervention is cost-effective, a judgement would be required as to whether the decision-maker thinks that the additional outcome associated with the intervention is worth the additional cost (measured on a standardised scale).
An important limitation of using cost-per-QALY thresholds when deciding whether an intervention is cost-effective is that most social care interventions have multiple goals, of which health-related quality of life is only one, or indeed it might be irrelevant. The impact of social care interventions on health-related quality of life can be quite small especially when compared with clinical interventions. In addition to having the threshold range by NICE, it is thus helpful to place findings in a context by comparing them with studies which have evaluated interventions for similar populations and with similar goals.
Caveats and assumptions of the analysis that might influence its findings are explained in the economic report or in the accompanying/supplementary evidence review document. In addition, executable versions of the analysis are available on request during consultation by registered stakeholders. It can be important for stakeholders to look at this when assessing how relevant the evidence is to them, and to inform additional considerations for their local setting. The guideline committee will also have discussed relevance, limitations and applicability of the analysis when formulating recommendations. These might be documented in the guideline or relevant appendices. For example, the guideline ‘Intermediate care including reablement’ includes an economic consideration explaining how recommendations on offering home-based intermediate care might apply differently for different localities depending on their health care infrastructure, and that – especially in rural areas – it might not be cost-effective to offer home-based intermediate care as an alternative to bed-based intermediate care (NICE, 2017 p.192).
The recommendations are the most important outcome of the guideline development. During the review process, the technical team presents the (economic) evidence statements for each review question to the guideline committee. The committee draws recommendations, which are updated iteratively as additional evidence is identified. Recommendations are derived directly from evidence statements, and NICE provides explanations as to which evidence statement(s) supported the recommendations. The ‘Evidence to recommendations’ section of the guideline (since 2018, this is ‘Committee discussion of the evidence’) details some of the Committee’s considerations. Table 2 provides examples of recommendations informed by economic evidence and presents the summarised economic evidence that informed the recommendation as it can be found in the guideline documents.
|Recommendations informed by economic evidence||Economic evidence supporting recommendation|
|Transition between inpatient hospital settings and community or care home settings for adults with social care needs NICE guideline [NG27]|
|1.3.10 Start a comprehensive assessment of older people with complex needs at the point of admission and preferably in a specialist unit for older people.||Economic evidence statement 1 (Page 102)
Evidence from 1 high-quality systematic review and meta-analysis (Ellis et al 2011 ++) suggested that comprehensive geriatric assessment and care provided on specialist units was likely to be cost-effective compared with non-specialist care. Findings from the study showed positive health and wellbeing outcomes for individuals and cost savings from a hospital perspective.
Additional economic analysis was carried out to test the likely impact of the intervention on health and social care and unpaid care costs in a UK context and found that comprehensive geriatric assessment and care provided on specialist units was likely to lead to cost savings from a health and social care perspective and to at least offset costs if costs of unpaid care were included.
|Intermediate care including reablement NICE guideline [NG74]|
|1.4.5 Consider bed-based intermediate care for people who are in an acute but stable condition but not fit for safe transfer home. Be aware that if the move to bed-based intermediate care takes longer than 2 days it is likely to be less successful.||Economic evidence statement EcBB1 (Pages 94–95)
Evidence from 2 economic evaluations (Harris et al. 2005 ++; Walsh et al. 2005 ++), which compared nurse-led units (in hospital or on hospital site) with standard care in medical wards, suggested that the intervention led to the same or better outcomes at possibly higher costs. Both studies evaluated costs and outcomes between baseline and follow-up of 6 months (…).
From both studies it was unclear whether the intervention would offset costs if a follow-up time of more than 6 months and a more comprehensive cost perspective was applied. (…) it was concluded that additional economic analysis was needed in order to derive recommendations about the cost-effectiveness of this type of intervention. In particular, a limited perspective on costs in both studies meant that important resource implications in regards to home care and care home were not included.
Results from our additional economic analysis suggested that mean costs were £610 lower in the intervention group, but this finding was highly sensitive to changes in some of the parameters. In particular, a delay in discharge from the intermediate care unit by a few days turned the cost savings into a negative cost difference (and thus favouring standard care).
Recommendations have different ‘strengths’ reflecting the quality and quantity of the evidence that supports them. This is reflected in their wording: the types of recommendations and examples from social care guidelines are shown in Table 3.
|Strength of recommendation & wording||Strengths of evidence required||Implication||Examples from NICE social care guidelines|
|Very strong (must; must not)||Legislation||There is a legal duty to implement the recommendation.||‘As a minimum, independent advocacy must be offered by local authorities as described in the Care Act 2014, Mental Capacity Act 2005 and Mental Health Act 2007.’ (Rec. 1.1.8; NG108)|
(directive, e.g., offer; do not offer)
|Evidence clearly displays that benefits outweigh harms||The recommendation should be implemented.||‘Develop and use communication protocols and procedures to support admissions.’ (Rec. 1.31; NG27)
‘Offer reablement as a first option to people being considered for home care, if it has been assessed that reablement could improve their independence.’ (Rec. 1.4.4 NG74)
|Evidence shows closer balance between benefits and harms||The recommendation might be implemented.||‘Consider home care support for older people with low to moderate needs to avoid, delay or reduce future dependency on health and social care services.’ (Rec. 1.3.2; NG21)|
Evidence statements can be strengthened or supplemented by committee consensus agreement. In some circumstances, the committee can also make recommendations in the absence of evidence if it can achieve consensus and provide a clear rationale for the expected benefit for the population of interest. This occurs more often in social care than in most areas of healthcare. Recommendations expected to have resource implications need to be supported by legislation (e.g., Health and Social Care Act 2014; Equality Act 2010; Mental Capacity Act 2005) or by existing recognised national guidelines. Social care guidelines can also refer directly to, or include specific recommendations from, existing guidelines accredited by NICE. For those not yet accredited, an assessment needs to be conducted using relevant tools such as AGREEII (Brouwers et al., 2010). This approach was used to consider guidance on tools and ways of working to support effective or accurate recognition and reporting of safeguarding concerns in care homes (NICE, 2020f). Evidence might be supplemented by expert testimony in an area where evidence is limited. In such cases, the committee identifies experts to invite to a meeting and asks them to provide evidence on particular review questions. Finally, research recommendations are developed in areas in which the review process discovered important gaps in (economic) evidence.
We sought to provide a comprehensive overview of the process by which economic evidence informs national social care guidelines in England. Social care guidance by NICE plays an important role in driving evidence-based cost-effective social care practice (NICE, 2019b). It also provides a potential vehicle for developing innovative social care economic evaluation methods, although NICE has not always managed to keep updated with recent methodological challenges and developments (Sculpher & Palmer, 2020). In addition, the work by NICE helps to identify gaps in (economic) evidence, thereby identifying potential research priorities that could be considered by the National Institute for Health Research (NIHR), UK Research and Innovation and other research-funding bodies. This is part of a virtuous circle between research, practice and policy: newly generated evidence from research funded by such national bodies is considered as part of the guideline review process and, in turn, can lead to new or updated recommendations (Turner, Bhurke, & Cook, 2015).
Economic evidence in social care in England – although much more developed than in most other countries – is still scarce in comparison to economic evidence in clinical and public health fields due to challenges in collecting high-quality data (Weatherly et al., 2020) and a general under-investment in adult social care research (Knapp et al., 2010; Tinelli et al., 2020). There could be further development and use of methods for when other evidence is sparse, such as use of decision analytic modelling (Squires & Tappenden, 2011), expert elicitation (Bojke et al., In press) and value of information (Fenwick et al., 2020). The use of digital technologies or of artificial intelligence and machine learning linked to social care devices might allow data collection in real time whilst reducing human data collection burden. As economic evaluations become more prevalent in social care, it is important that decision makers are aware of the evidence and know how best to use it.
Since the outcomes of the guideline development process potentially influence future social care practice and research priorities, it is important that stakeholders understand how and why guidelines are developed. With this knowledge, health and social care practitioners, managers and commissioners can better interpret the evidence and apply recommendations to practice, while people who use services and carers can better understand the type and quality of care they might expect to receive. Furthermore, there is opportunity for ongoing sector improvement if diverse groups of stakeholders are involved in guideline development and dissemination in future.
When HTAs of drugs or clinical interventions consider cost-effectiveness in their review process, it is often one of the most important criteria in determining whether a new drug is recommended (Cerri, Knapp, & Fernández, 2014; Dakin et al., 2015). It would be useful to understand the influence of economic evidence on recommendations (also in relation to other criteria such as effectiveness, acceptability, equity and needs) across social care guidelines. It would also be useful to understand how recommendations informed by economic evidence have been implemented in practice, and whether they led to the expected costs and benefits. Another need is to understand which guidance areas cannot be covered by economic evidence, how those are reflected in research recommendations, and compare those across guidelines.
AB is former economist for the NICE Collaborating Centre for Social Care (NCCSC). BA is former Associate Director for NCCSC, Head of Research & Information for SCIE, and lead partner in NCCSC. She is currently employed by York Health Economics Consortium (YHEC). BL is former technical adviser at the National Institute for Health and Care Excellence (NICE). BN is senior technical adviser at NICE. RK is technical adviser at NICE. MK is former economic lead in the NCCSC partnership. HW and MT have no conflict of interest.
Anderson, BE, Singh, SP, Stansfield, C and Liabo, K. 2018. The International Perspective on Healthcare Transition. In Hergenroeder, AC and Wiemann, CM (eds.), Health Care Transition: Building a Program for Adolescents and Young Adults with Chronic Illness and Disability, 269–278. Cham: Springer International Publishing. DOI: https://doi.org/10.1007/978-3-319-72868-1_28
Bauer, A, Wittenberg, R, Ly, A, Gustavsson, A, Bexelius, C, Tochel, C, Sudlow, C, et al. 2020. Valuing Alzheimer’s disease drugs: A health technology assessment perspective on outcomes. International Journal of Technology Assessment in Health Care, 36: 297–303. DOI: https://doi.org/10.1017/S0266462320000574
Beauchamp, SD and Dionne, C. 2015. The Development of Practice Guidelines in the Social and Human Sciences. Canadian Psychology/Psychologie canadienne, 56(4): 357–367. DOI: https://doi.org/10.1037/a0038811
Brimblecombe, N, Fernandez, J-L, Knapp, M, Rehill, A and Wittenberg, R. 2018. Review of the international evidence on support for unpaid carers. Journal of Long-term Care (September), 25–40. DOI: https://doi.org/10.31389/jltc.3
Brouwers, MC, Kho, ME, Browman, GP, Burgers, JS, Cluzeau, F, Feder, G, Consortium, ANS, et al. 2010. Development of the AGREE II, part 2: Assessment of validity of items and tools to support application. CMAJ: Canadian Medical Association journal = journal de l’Association medicale canadienne, 182(10): E472–478. DOI: https://doi.org/10.1503/cmaj.091716
Cerri, KH, Knapp, M and Fernández, JL. 2014. Decision making by NICE: Examining the influences of evidence, process and context. Health Econ Policy Law, 9(2): 119–141. DOI: https://doi.org/10.1017/S1744133113000030
Ciani, O and Jommi, C. 2014. The role of health technology assessment bodies in shaping drug development. Drug design, development and therapy, 8: 2273–2281. DOI: https://doi.org/10.2147/DDDT.S49935
Dakin, H, Devlin, N, Feng, Y, Rice, N, O’Neill, P and Parkin, D. 2015. The Influence of Cost-Effectiveness and Other Factors on Nice Decisions. Health Economics, 24(10): 1256–1271. DOI: https://doi.org/10.1002/hec.3086
Farrar, D, Tuffnell, D and Sheldon, TA. 2020. An evaluation of the influence of the publication of the UK National Institute for health and Care Excellence’s guidance on hypertension in pregnancy: A retrospective analysis of clinical practice. BMC Pregnancy and Childbirth, 20(1): 101. DOI: https://doi.org/10.1186/s12884-020-2780-y
Fenwick, E, Steuten, L, Knies, S, Ghabri, S, Basu, A, Murray, JF, Rothery, C, et al. 2020. Value of Information Analysis for Research Decisions-An Introduction: Report 1 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value Health, 23(2): 139–150. DOI: https://doi.org/10.1016/j.jval.2020.01.001
Gould, N. 2010. Integrating Qualitative Evidence in Practice Guideline Development: Meeting the Challenge of Evidence-based Practice for Social Work. Qualitative Social Work, 9(1): 93–109. DOI: https://doi.org/10.1177/1473325009355623
Gould, N and Kendall, T. 2007. Developing the NICE/SCIE Guidelines for Dementia Care: The Challenges of Enhancing the Evidence Base for Social and Health Care. The British Journal of Social Work, 37(3): 475–490. DOI: https://doi.org/10.1093/bjsw/bcm020
Grimshaw, JM, Thomas, RE, MacLennan, G, Fraser, C, Ramsay, CR, Vale, L, Donaldson, C, et al. 2004. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess, 8(6): iii–iv, 1–72. DOI: https://doi.org/10.3310/hta8060
Hassan, Z, Smith, M, Littlewood, S, Bouamra, O, Hughes, D, Biggin, C, Lecky, F, et al. 2005. Head injuries: A study evaluating the impact of the NICE head injury guidelines. Emerg Med J, 22(12): 845–849. DOI: https://doi.org/10.1136/emj.2004.021717
Hawley, S, Leal, J, Delmestri, A, Prieto-Alhambra, D, Arden, NK, Cooper, C, Group, RS, et al. 2016. Anti-Osteoporosis Medication Prescriptions and Incidence of Subsequent Fracture Among Primary Hip Fracture Patients in England and Wales: An Interrupted Time-Series Analysis. Journal of bone and mineral research: The official journal of the American Society for Bone and Mineral Research, 31(11): 2008–2015. DOI: https://doi.org/10.1002/jbmr.2882
Knapp, M, Manthorpe, J, Mehta, A, Challis, D, Glendinning, C, Hastings, G, Netten, A, et al. 2010. Developing the evidence base for adult social care practice: The NIHR School for Social Care Research. Journal of Care Services Management, 4(2): 167–179. DOI: https://doi.org/10.1179/175016810X12670238442101
Makady, A, van Veelen, A, Jonsson, P, Moseley, O, D’Andon, A, de Boer, A, Goettsch, W, et al. 2018. Using Real-World Data in Health Technology Assessment (HTA) Practice: A Comparative Study of Five HTA Agencies. PharmacoEconomics, 36(3): 359–368. DOI: https://doi.org/10.1007/s40273-017-0596-z
Matthew Owen, H and Jeffrey, MJ. 2003. 5. Clinical Guidelines and Evidence-Based Practice in Medicine, Psychology, and Allied Professions. In (pp. 83–107). New York: Columbia University Press. DOI: https://doi.org/10.7312/rose12310-005
NHS Digital. 2020. 2019-20 Reference Costs Collection. Reference Costs (Ref Costs). Retrieved from https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-collections/reference-costs.
NICE. 2015a. Dementia, disability and frailty in later life – mid-life approaches to delay or prevent onset. NICE guideline [NG16]. Retrieved from https://www.nice.org.uk/guidance/ng16.
NICE. 2015b. Home care: Delivering personal care and practical support to older people living in their own homes. NICE guideline [NG21]. Retrieved from https://www.nice.org.uk/guidance/ng21.
NICE. 2015c. Older people with social care needs and multiple long-term conditions. NICE guideline [NG22]. Retrieved from https://www.nice.org.uk/guidance/ng22.
NICE. 2015d. Older people: independence and mental wellbeing. NICE guideline [NG32]. Retrieved from https://www.nice.org.uk/guidance/ng32.
NICE. 2015e. Transition between inpatient hospital settings and community or care home settings for adults with social care needs. NICE guideline [NG27]. Retrieved from https://www.nice.org.uk/guidance/ng27.
NICE. 2016a. Autism spectrum disorder in adults: diagnosis and management. Clinical guideline [CG142]. Retrieved from https://www.nice.org.uk/guidance/cg142/.
NICE. 2016b. Mental health problems in people with learning disabilities: Prevention, assessment and management. NICE guideline [NG54]. Retrieved from https://www.nice.org.uk/guidance/ng54.
NICE. 2017. Intermediate care including reablement. NICE guideline [NG74]. Retrieved from https://www.nice.org.uk/guidance/ng74.
NICE. 2018a. Care and support of people growing older with learning disabilities. NICE guideline [NG96]. Retrieved from https://www.nice.org.uk/guidance/ng96.
NICE. 2018b. Decision-making and mental capacity. NICE guideline [NG108]. Retrieved from https://www.nice.org.uk/guidance/ng108.
NICE. 2018c. Dementia: assessment, management and support for people living with dementia and their carers. NICE guideline [NG97]. Retrieved from https://www.nice.org.uk/guidance/ng97.
NICE. 2018d. Learning disabilities and behaviour that challenges: service design and delivery. NICE guideline [NG93]. Retrieved from https://www.nice.org.uk/guidance/ng93.
NICE. 2019a. End of life care for adults: Service delivery. NICE guideline [NG142]. Retrieved from https://www.nice.org.uk/guidance/NG142/.
NICE. 2019b. NICE impact adult social care. Retrieved from https://www.nice.org.uk/Media/Default/About/what-we-do/Into-practice/measuring-uptake/nice-impact-adult-social-care.pdf.
NICE. 2020a. Adults with complex needs: social work interventions including assessment, care management and support [GID-NG10145]. Retrieved from https://www.nice.org.uk/guidance/indevelopment/gid-ng10145.
NICE. 2020b. Advocacy services for adults with health and social care needs. In development [GID-NG10156]. Retrieved from https://www.nice.org.uk/guidance/indevelopment/gid-ng10156/documents.
NICE. 2020c. Developing NICE guidelines: the manual. Retrieved from https://www.nice.org.uk/process/pmg20/chapter/introduction-and-overview.
NICE. 2020d. Integrated health and care for people who are homeless through being roofless. In development [GID-NG10170]. Retrieved from https://www.nice.org.uk/guidance/indevelopment/gid-ng10170/consultation/html-content.
NICE. 2020e. NICE publishes updated principles. Retrieved from https://www.nice.org.uk/news/article/nice-publishes-updated-principles.
NICE. 2020f. Safeguarding adults in care homes. [C] Tools to support recognition and reporting of safeguarding concerns. Draft for consultation. Retrieved from https://www.nice.org.uk/guidance/gid-ng10107/documents/evidence-review-3.
NICE. 2020g. Safeguarding adults in care homes. In development [GID-NG10107]. Retrieved from https://www.nice.org.uk/guidance/indevelopment/gid-ng10107/.
NICE. 2020h. Supporting adult carers. NICE guideline [NG150]. Retrieved from https://www.nice.org.uk/guidance/NG150.
NICE. 2020i. Who can register as a stakeholder? Retrieved from https://www.nice.org.uk/get-involved/register-as-a-stakeholder/who-can-register-as-a-stakeholder.
Sculpher, M and Palmer, S. 2020. After 20 Years of Using Economic Evaluation, Should NICE be Considered a Methods Innovator? Pharmaco Economics, 38(3): 247–257. DOI: https://doi.org/10.1007/s40273-019-00882-6
Specht, JK. 2013. Evidence based practice in long term care settings. J Korean Acad Nurs, 43(2): 145–153. DOI: https://doi.org/10.4040/jkan.2013.43.2.145
Squires, H and Tappenden, P. 2011. Mathematical modelling and its application to social care. Retrieved from London: https://www.sscr.nihr.ac.uk/wp-content/uploads/SSCR-methods-review_MR007.pdf.
Taylor, RS, Drummond, MF, Salkeld, G and Sullivan, SD. 2004. Inclusion of cost effectiveness in licensing requirements of new drugs: the fourth hurdle. BMJ (Clinical research ed.), 329(7472): 972–975. DOI: https://doi.org/10.1136/bmj.329.7472.972
Tinelli, M, Bauer, A, Knapp, M, Kettunen, A and Guy, D. 2020. Making the economic case for adult social care: The ESSENCE project. Journal of Long-term Care, 174–184. DOI: https://doi.org/10.31389/jltc.30
Turner, S, Bhurke, S and Cook, A. 2015. Impact of NIHR HTA Programme funded research on NICE clinical guidelines: A retrospective cohort. Health Research Policy and Systems, 13: 37. DOI: https://doi.org/10.1186/s12961-015-0025-8
Weatherly, H, Faria, R, Van den Berg, B and Sculpher, M. 2020. Economic evaluation methods in social care: A scoping review. In Curtis, L and Burns, A (eds.), Unit Costs of Health and Social Care Report 2020. Canterbury: PSSRU, University of Kent., 18–26.