The cost of implementing and sustaining an evidence-based, behavioral-health electronic screening system in probation departments

Abstract

Background

Roughly 50%-75% of youths who have had contact with the juvenile justice system have a mental-health disorder. In 2019, a northeastern state required probation departments to implement an evidence-based behavioral health (BH) screen. e-Connect is a digital clinical decisional support system designed to identify suicide thoughts and behaviors and related BH risk and triage youths based on BH need, then facilitate linkage to care.

Objective

To identify the resources and estimate the costs required to implement and sustain e-Connect from probation-department and policymaker perspectives.

Methods

Prospective micro-costing analysis conducted concurrently with a rigorous evaluation of e-Connect. Data were collected for 622 youths ages 10–18 via administrative records, study instruments, and semi-structured interviews. Resources/costs were categorized as “fixed”, “time-dependent”, or “variable”. Mean annual costs (per-county and per-screen, by county) were calculated for two intervention phases, “implementation” and “sustainment”. All costs are in 2019 USD.

Results

The policymaker-relevant, annualized, mean, per-county start-up and sustainment costs were $18,704 (SD = $14,320) and $13,374 (SD = $13,317), respectively. The per-screen sustainment cost was $115 (SD = $113) across counties, with variation attributed to a combination of a county’s behavioral-health needs, and differences in the types of resources utilized as part of their post-screening clinical response.

Conclusion

The results of this analysis will inform the decisions of probation departments and their stakeholders, who are interested in implementing an evidence-based behavioral-health screen for youths on probation. Site-level figures will provide important details regarding the resources/costs associated with various implementation and management strategies. Cross-site, per-person averages will provide crucial inputs into budget impact models and cost-effectiveness analyses.

Introduction

Youths with mental and behavioral health disorders form a disproportionately large percentage of those in contact with the United States juvenile justice system (Administration, 2021; Development Services Group, 2017; Meservey & Skowyra, 2015; Shufelt & Cocozza, 2006; Teplin et al., 2015). Studies show roughly 34%−75% of youth who have had contact with the juvenile justice system meet criteria for a mental health or substance use disorder (Teplin et al., 2015; Underwood & Washington, 2016; Wasserman et al., 2010), with monthly and lifetime suicide attempts among pre-adjudicated youth ranging from 1.4%—2.9% and 9.9%—13.2% (Nolen et al., 2008; Wasserman & McReynolds, 2006; Wasserman et al., 2010). Rates of suicidal behavior are elevated in justice involved youth due to a confluence of risk factors such as higher rates of traumatic exposure, PTSD, substance use, impulsivity (Abram et al., 2013), with few protective factors to offset these risks. Yet despite documented high rates of mental health and substance use disorders among youth in the justice system, many are left untreated, in part due to low rates of referral, treatment linkage and service uptake (Abram et al., 2013).

Most youths who are involved in the juvenile justice system are under community supervision through probation or parole (Medicaid and CHIP Payment and Access Commission (MACPAC), 2018). Identification of mental health and substance use disorders among juveniles within the justice system is generally ascertained via screenings and assessments (Development Services Group, 2017). Screenings are typically conducted first and are designed to triage youths, such that those who may be in crisis receive mental health services immediately, and all who appear to be in need of help receive a more comprehensive assessment, followed by treatment tailored to their mental health needs (Development Services Group, 2017). However, data from a large multi-site study focused on increasing access to evidence-based behavioral health treatment for youth on community supervision (Juvenile Justice-Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS)) found that of all youths on juvenile community supervision who had a substance use treatment need, less than 10% initiated treatment (Wasserman et al., 2021b).

Although, not as much existing evidence speaks to the cost savings of behavioral health screens and intervention for suicidal ideation specifically among youths, a 2017 study (ED-SAFE) (Miller et al., 2017) evaluating the implementation of a universal screening tool and intervention for individuals at risk for suicide in emergency departments, found a reduction of 30% in suicide attempt among participants in the intervention, compared to participants in the treatment as usual phase (Miller et al., 2017). Furthermore, a 2019 cost-effectiveness analysis of the same intervention reported universal screening plus intervention led to an estimated $5,020 per averted suicide attempt, compared to a calculated $13,522 per suicide attempt, producing a cost savings of $8,502 per averted suicidal attempts from the societal perspective (Dunlap et al., 2019).

A system that successfully identifies and provides linkage to treatment for suicidal behavior and behavioral health problems in youth on probation has the potential to produce considerable cost-offsets. Several studies have demonstrated significant economic costs associated with suicide among youths ages 15–24. The reported economic cost per suicide in 2013 among individuals between the ages of 5–14 and 15–24 was approximately $1.8 and $2 million, respectively (Shepard, Gurewich, Lwin, Reed Jr, & Silverman, 2016). These costs accounted for the direct and indirect costs of fatal injuries, net present values of future wages/salaries, fringe benefits, and household productivity losses (Shepard et al., 2016). In total, the years of life and average earnings lost from suicides among 15–24 year old in the United States in 2014 alone was $4.6 billion (Doran & Kinchin, 2020). In terms of behavioral health problems, a study by the Chief Probation Officers of California and the California Mental Health Directors Association found that an incarcerated youth with a mental health disorder costs $18,800 more than other youth (Cohen & Pfeifer, 2008). Furthermore, Murphy et al., found that the total annual opioid use disorder costs to the criminal-legal system in 2018, for youths ages 12–17 was more than $1 billion, rising to ~ 7.8 billion after accounting for excess healthcare expenditures, lost productivity, and premature mortality (Murphy, 2020).

An additional key driver in overcoming system-level obstacles to treatment is new or revised policy that directs practice change. In October 2018, a northeastern state passed legislation requiring county probation departments to screen all Adjusted Juvenile Delinquents (i.e., youths who have been arrested and placed on probation without formal adjudication) for behavioral health problems, via an evidence-based mental health screening tool ((DCJS), 2019). In response, several counties in the northeastern state opted to use e-Connect) to fulfill this new policy requirement. The e-Connect system is designed to educate probation officers in identifying suicidal behavior and associated behavioral health risk, facilitate cooperation between juvenile justice and behavioral health agencies, and create straightforward referral plans for behavioral health services, specific to each county (Elkington et al., 2023). e-Connect utilizes an efficacious digital clinical decisional support system (Elkington et al., 2023). Following identification of STB and behavioral health problems, it classifies individuals into clinical risk categories, which in turn informs a county-specific set of follow-up steps for linkage to care (i.e., referral pathway), according to the risk category identified. The "pathway" is established by county probation and behavioral health leaders, with an overarching objective of maintaining uniformity across counties, while accounting for differences in services available. Following the incorporation of the pathway framework, the level of inter-agency agreement on the number of referral options increased significantly, indicating that behavioral health and juvenile justice systems were able to make joint decisions about referral pathways (Wasserman et al., 2021a).

Furthermore, a recent study by (Elkington et al., 2023) was conducted examining the system level outcomes of the e-Connect system (intervention) to standard probation officer practices (treatment as usual), among juveniles on probation in 10 county probation department in the aforementioned northeastern state (Elkington et al., 2023). The primary outcomes of the study were screening and identification of suicidal ideation and behaviors and related behavioral health needs, as well as referral to and initiation of behavioral health services (Elkington et al., 2023). The e-Connect intervention was evaluated using a pre-post comparison, by comparing the performance of care-as-usual in youth probation departments at baseline vs. the performance of the e-Connect intervention. Results of the study found that probation officers who used e-Connect were 5 times more likely to identify youths with suicidal thoughts and behaviors, 11 times more likely to refer youths to behavioral health services, and youth referred through the e-Connect system were 17 times more likely to initiate behavioral health services (Elkington et al., 2023).

The broader adoption or widespread implementation of the e-Connect system, and similar systems (Dennis, Feeney, Stevens, & Bedoya, 2008; Grisso et al., 2001), will require additional resources to be funded by public agencies, such as county probation departments (e.g., staff, trainings, etc.), and Medicaid, in the form of direct costs associated with the clinical response elicited by the behavioral health screen (e.g., behavioral healthcare, an emergency department [ED] visit, etc.). Therefore, information on the resources and associated costs required to implement and sustain the screening system, and how they vary by site, is highly relevant to counties whose probation departments are planning to incorporate such a tool, as well as to policymakers evaluating the budget impact of requiring such a system.

The absence of information on the cost of healthcare interventions is frequently cited as an obstacle to implementation (Gold et al., 2022; Wagner et al., 2020). To overcome this barrier, methods such as micro-costing are often used to estimate the cost of implementation of an intervention (Gold et al., 2022; Wagner et al., 2020). More specifically, micro-costing is a ‘cost estimation methodology employing detailed resource utilization and unit cost data to generate precise estimates of economic costs’ (Xu et al., 2021). Measuring costs can allow stakeholders and decision-makers to evaluate the cost of an intervention from alternate perspectives and time horizons, as well as the resources necessary for replicating an intervention elsewhere (Gold et al., 2022; Wagner et al., 2020).

The objective of this study was to conduct a site-specific micro-costing analysis of the e-Connect) system; the results of which will inform probation departments of the resources and costs associated with various implementation and management strategies, while the cross-site, per-person averages could subsequently serve as crucial inputs into comprehensive budget impact models and cost-effectiveness analyses.

Methods

e-Connect overview

The e-Connect system (described briefly above and elsewhere; (Elkington et al., 2023) is an evidence-based suicide risk identification and related behavioral problems screen, referral, and cross-system linkage program for youth on probation. e-Connect) uses the Gateway Provider Model (Stiffman et al., 2004). Thus, using a web-based digital clinical decision support application, the e-Connect system combines evidence-based screening, classification of clinical need (detailed below), and streamlined linkage to treatment (Elkington et al., 2023).

The e-Connect screen is self-administered by the youth during an intake interview with a juvenile probation officer, and once completed and automatically scored, the e-Connect system classifies youth responses into one of three clinical risk categories. The three e-Connect classifications are Class I, Class II, Class III. A screening result of Class I or Class II indicates that the youth is in-crisis and needs further assessment by a behavioral health professional either immediately (Class I), or within 72 h (Class II). Class III indicates that the youth needs further assessment, but not urgently (i.e., a standard behavioral health referral). Classifications I-III have predetermined, classification-specific, linkage steps (i.e. referral pathways) created by the county probation department and their behavioral health partners, to be followed by the juvenile probation officer (Elkington et al., 2023; Wasserman et al., 2021a). The linkage steps may include county resources such as mobile crisis teams, a group of healthcare professionals that are able to provide mental health services at a variety of locations, including a person’s home (New York City Health, 2021), ED psychiatric hospitalization; resources such as law enforcement can be utilized as a means of transportation for ED psychiatric hospitalization; etc.; and are contingent upon a variety of factors, such as the availability of resources within a county and the e-Connect screening results.

Analytic overview

A micro-costing analysis was conducted concurrently with a National Institute of Mental Health (NIMH)-funded study to evaluate the efficacy of the e-Connect system across 10 County Probation Departments in a northeastern state (Elkington et al., 2023). Micro-costing entails systematically capturing, cataloging, and then valuing changes in resources resulting from the use of an intervention to treat patients (Neumann, 2016). Data on all resources (e.g., labor, materials, supplies) required to implement and sustain the e-Connect program were generated via administrative records, study instruments, and semi-structured interviews with site personnel most familiar with the day-to-day operations of the e-Connect program. Nationally-representative unit-costs were then applied to those resources to estimate the costs of the intervention and adjusted for inflation to 2019 dollars using the Consumer Price Index (Bureau of Labor and Statistics Consumer Price Index, 2022) (See Table 4 in Appendix).

Table 1 Probation department specific costs

Expenditures from resource utilization were calculated from a probation department perspective, as well as from a policymaker perspective. The probation department perspective includes all resources used and associated expenditures accrued by the probation department for the purposes of implementing and sustaining e-Connect. The policymaker perspective includes all resource costs incurred by the probation department, as well as those resulting from the initial clinical response to the e-Connect screening outcome, given that probation department budgets are ultimately derived from public funds, as are the resources utilized during the initial response. All research related costs were excluded from the analyses.

We considered e-Connect intervention costs to be those incurred directly by the probation department during the implementation and sustainment phases of the intervention. Resources/costs were categorized as “fixed” (one-time, upfront resources/costs required by the probation department to get the e-Connect tool ready for use), “time-dependent” (recurring resources/costs that are fixed over a given time period, and are not dependent upon the number of clients seen during that period; e.g., annual licensing fees), and “variable” (resources/costs that are necessary for each client; e.g., staff time). Costs associated with the implementation phase of the intervention included the fixed costs, and time-dependent and variable costs accrued in the first year. We assumed fixed costs would be distributed over the course of the first 12 months following start-up, for budgeting purposes; thus, implementation costs are presented as “Year 1” costs. Sustainment costs consisted of the time-dependent and variable costs that would be incurred in subsequent years.

Fixed costs were calculated using nationally representative unit costs for personnel involved in e-Connect related activities, multiplied by the time spent engaging in the relevant activity. Materials/supplies purchased for the purposes of using the e-Connect system were also included in fixed costs. Variable costs were calculated using the nationally representative unit costs for all personnel involved in each step of the clinical pathway, multiplied by the time spent engaging in the relevant step. Mean annual costs were then calculated for two intervention phases, “implementation” and “sustainment”. All time-dependent and variable costs were annualized to represent the yearly costs incurred from the probation department and policymaker perspective.

Measures

Start-up/preparation for implementing e-Connect

Prior to using the e-Connect system, counties engaged in one-time activities required for start-up, designated as fixed costs (Table 1). Fixed cost expenditures included resources associated with pathway meetings (i.e., meetings held to develop and coordinate the post-screen clinical steps between probation departments and behavioral health agencies) (Elkington et al., 2023; Wasserman et al., 2021a), behavioral health trainings (i.e., web-based trainings developed to educate probation officers on suicide risk among justice-involved youth, the e-Connect system and its use, and how to engage families in the screening and linkage process), and kick-off meetings (i.e., meetings to train/educate probation departments on the e-Connect system and usage etc.). Additional expenditures attributed to fixed costs included materials/supplies (i.e., electronic tablets), information technology services, the cost of training the probation department supervisor on the e-Connect website. Table 4 in Appendix displays all nationally representative unit costs used for this study, personnel involved in e-Connect system related activities, by county and number of personnel involved. Additionally, a technical assistance piece was added to the analysis to take into consideration the costs accrued, and the resources used by the e-Connect team, to provide assistance to the counties during start-up in a real-world scenario (e.g., assistance in behavioral health pathway development, conducting meetings/trainings, travel etc.).

Data collection occurred through electronic tracking and timestamps via learning platforms, electronic expenditure receipts, time-tracking documents, meeting sign-in sheets, and semi-structured interviews, which included periodic short-interviews with probation department leaders to ascertain additional information on resources used. Trainings were conducted in-person and virtually via a learning platform, and electronic notifications were delivered upon training completion. Meetings were conducted in-person, and sign-in sheets were collected with information detailing job title and agency; all meetings abided by a predetermined time frame across counties. Receipts for all e-Connect related resources were tracked electronically, such as for tablets and accessories, time spent setting up tablets, use of the learning platform, and licenses required for usage of the e-Connect system.

Implementing and sustaining e-Connect

Behavioral health screening and clinical response data were collected by county probation departments over the course of 5–8 months following implementation for 622 youths ages 10–18 years. The 5–8 months of behavioral health screening and clinical response data did not include the first month post implementation in an effort to exclude costs attributed to human error when using a new system. Table 6 Appendix represents screening costs for the aforementioned first month post implementation. Behavioral health screening and clinical response data collected between 5–8 months, for 10 counties, were annualized to represent the expenditures the relevant perspective would incur yearly. The resources/costs required for the sustainment of the e-Connect system consisted of staff labor costs (e.g., time spent screening youth and linking them to care) and clinical-response costs (e.g., mobile crisis, hospitalization). Data on the number of e-Connect screens conducted, as well as the behavioral health classifications resulting from the screens, were collected across all counties via automatic electronic tracking through the e-Connect screening platform. Additional information pertaining to the resources used for each linkage step were obtained through semi-structured interviews with probation department site leaders, and Clinical Event forms (for Classes I and II) completed by research staff. The Clinical Event forms were used to document whether the Pathways were followed for Class I and II within a clinically appropriate time frame, as well as any deviations from the predefined linkage steps.

For the purpose of generalizability, nationally-representative mean wages by occupation were obtained from the U.S. Bureau of Labor Statistics. Other unit costs (e.g., for mobile crisis, hospitalization, law enforcement response, and transportation) were obtained from SAMHSA and the NYS Office of Mental Health (SAMHSA, 2020). Table 4 in Appendix lists the sources for each unit cost.

Analysis

Costs were calculated via the resource costing method, which consisted of identifying a unit value that reflects the “real-world” costs faced by the stakeholder of interest for a particular resource, and multiplying it by the number of units utilized. All costs were adjusted to 2019 US Dollars, then annualized, and summed according to stakeholder perspective and resource category (i.e., start-up, time-dependent, or variable), with variable costs further categorized according to the behavioral health classification identified by e-Connect. Implementation costs were estimated by summing all relevant fixed, and Year 1 time-dependent and variable costs, by county. This calculation of implementation costs assumes that the one-time, fixed, start-up costs would be paid over the course of the first year of the program. Annual e-Connect sustainment costs, following Year 1, were estimated by summing the relevant annualized time-dependent and variable costs. Cross-county mean and standard deviation costs were then calculated for each resource category and intervention phase (i.e., implementation, sustainment), as were minimum and maximum values. Finally, the annualized, mean total sustainment cost, per e-Connect screen, was also calculated overall, and by county.

Costs were separated by “policymaker perspective” and “probation department perspective” to distinguish between the costs that were assumed by probation department and the combined costs of the intervention that would be assumed by the county and the probation department, which produce the policymaker perspective. The costs attributed to the probation department excluded costs that would be solely assumed by the county (i.e., mobile crisis services etc.). The costs attributed to the policymaker perspective included costs that were assumed directly by the county (i.e., mobile crisis services etc.) and indirectly through the probation departments, as probation departments are considered local government agencies.

Results

Table 1 contains the cross-site, total, probation department specific implementation and sustainment costs. The annualized, mean total fixed implementation costs across probation departments was $7,216 (SD = $4,038), with a min–max range of $4,483—$18,005. The annualized, mean total sustainment cost across probation departments was $2,086 (SD = $2,914), with a min–max range of $245—$9,821. The annualized, total average cost for administering an e-Connect screen across all counties was $1,304 (SD = 1,781), but it varied widely across counties ($137—$6,045). This variation is largely a function of the number of youths screened in each county, as indicated by the per-screen costs. The mean, per-screen, probation-specific sustainment costs across counties was $17, with a standard deviation of $10. The detailed, county-specific resource costs are presented Table 5 in Appendix. Additionally, Appendix Table 8 displays a 5-year time horizon for time-dependent costs.

Table 2 Annualized, policymaker-relevant, cross-site implementation and sustainment costs

Table 2 contains cross-site, descriptive statistics for each resource by category for costs incurred from the policymaker perspective. The average fixed cost across counties was $5,501 (SD = $1,286), ranging from a minimum of $4,270 to a maximum of $8,724. The annualized, mean, total time-dependent cost across counties was $172 (SD = $100) with a min–max range of $72—$395. The annualized, mean total variable costs across counties was $13,202 (SD = $13,238) with a min–max range of $1,477—$44,450. Annualized, policymaker-related implementation and sustainment costs by county are displayed in Fig. 1; detailed, county-specific resource costs are presented Table 6 in Appendix. Post-screening, clinical-response, resource use, and the associated costs (mean = $11,876, SD = $11,597) also varied widely across counties ($1,219—$38,402). The wide variation in costs associated with post-screening resource use can be attributed to the number of youths screened, the amount of behavioral health need in the county, the types of resources available in a county, and the manner in which those resources were utilized. The total, annualized, post-screening, clinical response resource expenditures across all counties was $118,763, with $84,855 (72%) expended towards mobile crisis related services, $15,492 (13%) expended towards law enforcement transport and psychiatric hospitalization services, and $18,124 (15%) expended towards youths who saw a community behavioral health provider as part of the e-Connect system referral (Table 6 in Appendix).

Fig. 1

Total policymaker-relevant annualized implementation and sustainment costs, by county

Table 3 Annualized, policymaker-relevant, cross-site, per person variable costs

Table 3 displays the policymaker-relevant, annualized, cross-site, per-screen, variable costs according to screening class. The annualized mean variable cost among those categorized as Class I, was $4,703 (SD = $6,077), ranging from $0 to $15,884 across counties (Table 6 in Appendix). The annualized mean variable cost among those categorized to Class II was $6,122 (SD = $7,012), ranging from $192 to $24,020 across counties. The average costs among those in Class III and Below Threshold were $1,560 (SD = $1,508) and $818 (SD = $1,252), respectively. The annualized mean total sustainment cost, per e-Connect screen, across all counties was $116 (SD = 113), ranging from $39 to $422 across counties (Fig. 2).

Fig. 2

Average policymaker-relevant sustainment costs, per e-connect screen

Discussion

The probation department specific average, per-county, e-Connect implementation cost, which included all one-time, fixed, start-up expenditures, as well as time-dependent and variable costs incurred in Year 1 of the program, was $7,216. Fixed costs accounted for approximately 71% ($5,130) of the implementation costs. This pattern differed starkly from the policymaker perspective, explained below, which had approximately 71% ($13,202) of its implementation costs attributed to variable costs. The reasoning behind the difference is that the majority of variable costs, detailed below, were accrued during the post-screening clinical responses, which occur outside of the probation department. Similarly, the average total fixed cost associated with the probation department perspective was less than that for policymakers ($5,501 vs. $5,130), given the additional resources required to ensure a seamless and comprehensive clinical response (e.g., behavioral health representatives from behavioral health agencies who attended e-Connect specific meetings). In both instances, the remainder of the fixed costs was attributed to the cost of technical assistance ($2,955), the baseline additional cost behind the scenes technical assistance and support provided by the e-Connect research team. As shown in Fig. 1, much of the variation in total fixed costs across counties can be attributed to the differences between County A and the other 9 counties, with A exhibiting over a twofold increase in expenditures compared to the county with the second highest start-up cost. Due to County A’s relatively large probation department, there were larger numbers of personnel in meetings/trainings, and additional equipment was required, such as additional electronic tablets and equipment etc. (see Table 6 in Appendix).

On average, just under 90% of the estimated annual sustainment costs for probation departments ($1,913) were variable in nature, and 61% of those were associated with resources expended to administer the e-Connect screen; the remaining 39% was attributed to resources expended by the probation department during the referral and linkage-to-care portion of the e-Connect system (e.g., time spent with a youth waiting for a mobile crisis unit to arrive).

The mean, per-county, annual cost required to sustain the system from a policymaker perspective was estimated to be $13,374, most of which, just over 98% ($13,202), was accounted for by variable costs. As with fixed costs, there were considerable differences in total variable costs between counties, with County A again leading the way by exhibiting almost a threefold increase in variable costs, compared to the County with the second highest variable costs (Fig. 1); however, the relative differences between counties change drastically upon calculation of the mean, per-screen sustainment costs (Fig. 2). This was also true of probation department specific costs (Table 1, Table 5 in Appendix). Variations in the mean, per-screen, sustainment cost, can be attributed to a combination of county behavioral- and mental-health need rates as reflected in the differences in proportions of youths screened to particular Classes, and the differences in the types of resources utilized as part of the county’s post-screening clinical response (see Appendix Table 7). For example, County B and County D both had a similar number of youths screened (75 and 85), similar rates (11% and 16%, respectively) of youths considered to be “in crisis” (Class I + Class II), as well as similar Class I and Class II distributions (County B: Class I = 2%, Class II = 9%; County D: Class I = 2%, Class II = 14%); but, as shown in Fig. 2, the counties varied widely in their mean, per-screen, sustainment cost (County B = $38/screen; County D = $176/per-screen). County B utilized a combination of an on-site clinician and an external community behavioral health provider for youths screened into Classes I-III, while County D utilized a combination of mobile crisis and an external community behavioral health provider. Although County J had the lowest number of annualized youths screened (n = 27), it simultaneously had the highest mean, per-screen, sustainment cost at $419, more than double that of County H, the second highest per-screen cost, at $199. This can be attributed to County J having the highest rate of youths screening as “in crisis” (28%), accompanied by the post-screen clinical resource (law enforcement + psychiatric hospitalization) being the costliest of the clinical resources across counties (SAMHSA, 2020).

Although we do not have information to evaluate the costs from the current study in context with potential downstream cost-offsets (see Strengths and Limitations section below), the data appears to be promising, and the results of the e-Connect intervention shows that youths considered to be “in crisis” are identified and connected to behavioral health services (Elkington et al., 2023). Looking at the relevant findings from a cost-effectiveness study in 2019, examining a universal screening tool and intervention for youths at risk for suicide in emergency departments, results showed that the intervention brought about a cost savings of $8,502 per averted suicide attempts from the societal perspective (Dunlap et al., 2019; Miller et al., 2017). Moreover, the reported costs for suicide in 2013 was $1.8 million for youths aged 5–14 years old and $2 million among adolescents and young adults aged 15–24 years old, accounting for direct and indirect costs (Shepard, Gurewich, Lwin, Reed Jr, & Silverman, 2016). Altogether, it is important to have evidence-based tools integrated within probation departments to successfully identify and link youths on probation with suicidal behavior and behavioral health problems to treatment, while potentially generating significant cost-offsets.

Strengths and limitations

One of the many strengths of this study is it the broad range of county characteristics. Although the counties are all located within a single state, county population and population density distributions varied widely. Two counties had populations below 60,000; 3 counties had between 60,000 and 100,000 residents; 2 counties had a population between 100,000 and 200,000; and 3 counties contained greater than 200,000 residents. Furthermore, 3 counties were above the state average population density of 239.26 persons/square mile, while 7 counties were below (Health, 2011). Additionally, according to a paper produced by the parent study, detailing the creation of the clinical pathways, the authors provide further information pertaining to the settings of the study sites. According to the authors, the rates of adolescent suicides across counties ranges from an average of 9.1 per 100,000 adolescents (range 1.4 – 17.6), (NYS Department of Health, 2019; Wasserman et al., 2021a), the average rate of urbanicity across the counties studied was 53% (range = 30%—90%) (United States Census Bureau, 2010; Wasserman et al., 2021a), and the average availability of behavioral health service providers across counties was 55 per 100,000 youths (range = 27.5 – 141) (CDC, 2015, Wasserman et al., 2021a). Further information regarding the setting of the study sites can be found in the Wasserman et al. (2021a) research paper. Conducting a micro-costing analysis simultaneously with the e-Connect study, as opposed to retrospectively, was also a strength, as it allowed for real-time data collection, thereby limiting potential recall bias.

The primary limitation of this study is its sole focus on the resources/costs required to implement and sustain the e-Connect system above treatment as usual. When considering the implementation of a new program, it is important to understand the resources and costs associated with it for budgeting and planning purposes. However, to truly understand the new program’s value to a particular stakeholder, its costs and effectiveness must be evaluated in comparison to treatment as usual, or alternative strategies. A comprehensive cost-effectiveness, or cost–benefit, analysis would incorporate the assessment of potential downstream cost-offsets relevant to various stakeholders, as well as the additional effects observed as a result of the intervention (Neumann et al., 2016). We were unable to follow youths beyond the initial clinical response in order to evaluate potential downstream cost-offsets resulting from effective treatment, such as reduced recidivism, and utilization of high-cost healthcare services (e.g., the emergency department). However, as discussed in the Introduction, e-Connect, and similar programs have shown promise in their ability to identify and link at-risk youths to treatment; consequently, they are being introduced or mandated elsewhere around the country, thereby increasing the importance of understanding the potential resource needs, and how they might vary across sites, depending on their existing infrastructure.

Furthermore, the technical assistance costs did not include the costs of any ongoing technical assistance needed by the county during sustainment (i.e., editing the linkage pathway to adapt it to real-world changes across time, development of the web-based BHT that would be required to tailor for given states/counties/settings is unknown etc.). Also, costs were annualized to allow for enhanced clarity regarding interpretation, as well as comparability; however, the extent to which the figures presented reflect the true annual costs will depend on whether the trends observed remain. Additionally, although Class I and Class II referral and initiation information is tracked in real-time by the e-Connect research team, who received an alert at the time of classification, Class III youths are not tracked in real-time. Therefore, Class III information was obtained via administrative data on a monthly basis. Consequently, administrative data obtained may result in unknown costs due missing data/incomplete data and the nature of administrative data as whole. Unknown Class III costs are anticipated to be minor, as most of the variable costs emerge from Class I’s and Class II’s.

Conclusion

There is a dire need to identify suicide risk in youths within the juvenile justice system and link these youth to behavioral health services, given the high prevalence of behavioral health disorders among this population, and the many potential personal and public health consequences associated with lack of, or insufficient, treatment (e.g., suicidal behavior, recidivism, etc.). Probation departments are well-positioned to serve as a critical point of access for youths in need of mental health treatment. This micro-costing analysis will allow probation departments and policymakers to predict costs associated with implementing and sustaining the e-Connect system. Additionally, this micro-costing analysis can serve as a basis for a future cost-effectiveness and cost–benefit analyses comparing behavioral health screening interventions, looking at various perspectives (i.e., healthcare perspective, societal perspective etc.,) in order to inform policy decisions on how best to use resources in order to achieve the largest improvement in health among this high-risk population.

Appendix

Table 4 Nationally Representative Unit-Costs for Utilized Resources ("Bureau of Labor Statistics, U.S. Department of Labor, Occupational Outlook Handbook"; "Bureau of Transportation Statistics, United States Department of Transportation"; "Glassdoor"; "Substance Abuse and Mental Health Services Administration")

Table 5 Cross-site probation department specific costs

Table 6 Cross-site policymaker relevant costs

Table 7 Cross-Site Variable Costs

Table 8 5-Year Horizon of Fixed to Time-Dependent Costs

Table 9 Month 1 e-Connect Screening Classification and Costs