Evaluation

Activity Reporting

Goals of Activity Reporting

Data generated by this initiative will be reported by providers for the following purposes:

In the long term, there is a need for a shared activity tracking platform that enables the documentation and tracking of information related to economic security screening and referrals. This platform should be accessible to both referring providers and receiving providers. While ACS is actively exploring opportunities to develop software that meets this long-term need, the Economic Security Activity Reporting form, hosted on Airtable, is the current platform for submitting this documentation.

What Staff Must Do To Report Economic Security Activities

Child welfare teams and ACS-funded ERNs will record identified economic security needs, actions they take to address those needs, and updates on applications and referrals after each related action. They will submit this data to ACS through the Economic Security Activity Reporting form. This platform will allow data to be captured in structured fields for both case-level analysis by providers and aggregate analysis by ACS and its evaluation partners. It is separate from ACS case/progress notes and updates sent from ERNs to child welfare workers.

ERNs funded by NYCBenefits contracts will report their activities to NYCBenefits through existing reporting mechanisms.

What Data Must Be Reported

ACS-contracted child welfare teams and ERNs will document and update each family’s economic security journey by reporting on the following economic security data elements in the Activity Reporting form.

Context

• ACS Case ID/Stage ID or unique ID like case initials (if no ACS ID)
• Role of staff member whose activities are being reported on (e.g., primary child welfare worker, supporting child welfare worker, supervisor, ERN)
• Date of economic security activity
• Economic security activity conducted to support economic security needs (e.g., needs identification, referral to an ERN)

Activity 1: Screen for Economic Security Needs

• Economic security needs identified for family or young person
• Needs for which the family or young person already has support or has received benefits and services
• Identified needs for which the family or young person agrees to receive support

Activity 2: Refer to an Economic Resource Navigator

• Needs referred to ERN
• Whether ERN is in-house or external
• If external, to which organization
• If external, reason for external referral

Activity 3: Provide Benefits and Resources Support

• Any economic security activity conducted by child welfare team to pursue economic security benefit or service (e.g., application submission)
• Status updates on benefits and resources support pursued by child welfare team -or- ERN, as reported back to child welfare team
• Benefits or services secured by families and young people
• Amount secured for each benefit or service
• Reason if unable to obtain benefit or service

Activity 4: Provide Post-ACS Case Closure Pathways

• If the family or young person was connected to a new ERN after ACS case closure
• Name of new economic navigation provider if the family or young person was referred externally
• Status update of each application/benefit applied for by the child welfare team at the time of ACS case closure, if any

CoQI Vision for Economic Security

ACS is committed to integrating economic security into core child welfare practice beyond the ACS Economic Security program to ensure the quality of economic security services provided and outcomes achieved by ACS-contracted providers. To achieve this long-term vision, the ACS Agency Program Assistance (APA) unit within the Division of Policy, Planning, and Measurement (DPPM) will expand the existing Collaborative Quality Improvement (CoQI) model to incorporate valid and reliable economic security data indicators developed by the ACS Economic Security program. Providers will collect and report this data on an ongoing basis, allowing APA to make informed assessments about economic security-focused improvement priorities and collaboratively develop improvement plans over an annual cycle.

CoQI uses a five-step protocol that follows an annual "Plan-Do-Study-Act" (PDSA) continuous quality improvement process, with periodic status checks between key steps to maintain open communication between APA and providers. CoQI also aligns with the Provider Agency Management System (PAMS) annual review cycle to streamline and optimize the use of data to inform decisions.*

‍

In this way, economic security will become a core component of APA's functions in:

1. Monitoring program performance using quantifiable performance data (e.g., case reviews, interviews, PAMS modules)‍
2. Identifying program-specific strengths and weaknesses through data collected from comprehensive evaluations of economic security case practice and services delivered‍
3. Compiling summary reports comparing performance productivity to stated goals and objectives‍
4. Providing guidance and feedback in quality improvement plans with clear expectations and timeframes for improvement‍
5. Identifying provider needs for assistance and providing the appropriate assistance to meet those needs

CoQI Measurement Framework for Economic Security

While the exact economic security data indicators and data sources for consideration in CoQI are still being refined, they will be drawn from the data reporting requirements developed in this initiative (see Activity Reporting) —specifically, from:

• Child welfare workers’ case/progress notes entered in CNNX documenting economic security activities conducted by child welfare workers (e.g., family engagement, screening, referral) and economic security activities conducted by ERNs (e.g., assessment, application, securing benefits) that provide a holistic picture of case management centered on economic security‍
• Regular reporting submitted to ACS by both child welfare workers and ERNs

In the long term, economic security-focused quality standards and indicators for prevention services, foster care, juvenile justice, and Family Services Unit (FSU) will be developed to provide clear guidance for providers to engage in quality economic security practice. These economic security-focused quality standards and indicators will, in turn, also inform and refine PAMS (e.g., case record review guidelines and questions) accordingly. As a result, economic security will be integrated into CoQI’s annual PDSA cycle and aligned with the PAMS review calendar.

CoQI Expectations for Providers

At this stage, providers will only need to commit to the activity reporting requirements outlined in this playbook. Chapin Hall will coordinate with APA to identify specific parts of the existing CoQI model that will be modified or expanded. More concrete guidance and training will be provided to providers as the program continues toward scaled implementation.

*New York City Administration for Children’s Services. (2018). Agency Program Assistance collaborative quality improvement desk guide. Division of Policy, Planning, and Measurement, New York City Administration for Children’s Services.

Why Collect Feedback from Families and Young People?

Ecnomic Security is a new program. It has been designed based on the first-hand experience and expertise of both service providers and ACS-involved families and young people. Its success relies on the continued input of these system participants and the iteration of its design based on that input. What has their experience been like receiving support around economic security needs? What has made it successful? How could it be more successful?

Priority Areas of Desired Feedback

In order of priority, these are the five topic areas around which ACS-connected family members have expressed the strongest desire to share feedback:

1. Respect and Comfort: Did staff make you feel heard, respected, and not judged when sharing personal details?
2. Accuracy of Needs: Did the provider correctly identify the specific things you actually needed (like food, housing, or child care)?
3. Final Results: Did the services actually help you meet your family's basic needs and feel more confident?
4. Clear Explanations: Was it easy to understand why questions were being asked and what your next steps were?
5. Hands-on Help: Did your Navigator actually help you fill out forms, gather paperwork, or solve application problems?

Methods

ACS-connected family members reported willingness to provide feedback via a short survey. An SMS-based survey was most preferred. ACS provider teams should consider sending the survey as an SMS conversation (not a link) using a platform like one of the following:

• TextIt
• Qualtrics
• SurveyLegend

Here is some sample text for the survey:

Message

Hi, this is [Agency Name]. We'd love to hear how things went with your worker — just 5 quick questions, reply with a number 1-5 (1 = not at all, 5 = completely). Reply STOP anytime to opt out.

1. Did your team understand what you and your family needed?
2. Did your team help you with paperwork or applications?
3. Did your team explain things clearly, including what to expect next?
4. Did the support you got make things easier for your family?
5. Did you feel respected and heard when talking with your team?

Closing

Thanks so much — this really helps us do better.