Building Strong Families
Evaluation and Reporting
(originated 6/2000; revised 1-2004)
and practice skills in understanding lifespan development, positive interpersonal relationships, and improving quality of life.
This reporting structure supports the Building Strong Families
section of the North Carolina Cooperative Extension Family and Consumer
Sciences long
range plan.
The Extension Reporting System asks that you capture programming efforts
as much as possible in the reporting system. However, remember --there
is a difference between reporting and evaluating. We are asking you to
report knowledge gain and adoption of practices. Because of the nature
of family life and parenting education, we cannot take the credit for all
changes that family members make. There are also other institutions and
community factors that influence change.
The planning team has designated several evaluation tools to
use for your ease in data collection through out the year so that those
reporting can transfer tallies to the Extension Reporting System (ERS).
Record Keeping Sheets: Parenting
education | Parenting educators | Child
care providers | Other
How and what to report | Adoption of
Practices | Impact Indicators | More
in depth evaluation | Glossary
Knowledge
Evaluation Tool (pdf) | Attendance
Log-parents | Attendance Log sample | Attendance
Log-General
Dollar value- parenting education |
To report the number of participants by audience type, maintain an attendance log then transfer the numbers to the Extension Reporting System (ERS). Although this type of information has been asked for in Section A of ERS, it was not collected by type of audience (ie., parents, families, grandparents, etc).
Reproduce record keeping sheets . These will
be used to transfer data from them for the ERS system for this section
of the Long Range Plan. Where the sheet says "Session 1 title/date" insert
the actual title of the learning session and date. Maintain this same list
throughout the reporting year. Add more rows or columns as needed. This
sheet is for your records only. The participants never see this record
sheet. Simply add names for (first time) new participants. For those who
have attended other sessions, check the name of the current session. Each
name will be listed only once. The idea is to use some system so that Joe
Smith who attends 10 session gets counted for INCREASED KNOWLEDGE only
once, not 10 times. The transfer the figures from your record keeping sheets
to the Extension Reporting System (ERS).
To measure knowledge increase - you may use the generic post
evaluation. Note that the knowledge gain question is item number 7 on the Knowledge
Evaluation Tool. This is the most important question to use to report
an increase in knowledge. To be more specific, you can reword the question
to use the specific subject you taught-for example:
The training provided me with new knowledge about child discipline;
or The educational session provided me with new knowledge about communicating
with my family members.
You could essentially only ask this question but have each person code
their response form with initials or names to track and not duplicate numbers.
or
to roughly gauge 100% knowledge gain, you can record the responses
from one of these sample activities:
-Stand in a circle and tell what you learned -Raise your hand and tell something that was really valuable -In dyads, use an index card and list 3 things you have learned and discuss them -Number off a large audience 1, 2, 3 ,4 and ask all the 2's to stay afterwards for 5 minutes and use this as a sample. Ask them to share what they learned. Give them an incentive/gift is available. -Mark the floor off into a tic-tac-toe board and play a team game to ask questions about what you have taught. Record results based on correct responses. |
How and What to Report
The reference chart will provide a reference
for reporting. But be sure to use the Record keeping sheets that
are referenced for ease in maintaining data and ease of reporting.
For some Impact Indicators you
will need to plan ahead. For example if your goal is to increase the amount
of time parents spend with children, you need a baseline for comparison
before you teach then a measure of their time with children (and perhaps
how they spend it) after you teach. Here are some suggestions:
To measure time with children parents can record the amount
of time they spend with their children for a week (or another given time)
at the beginning of the program. Use this as a baseline. Then ask them
to chart a similar time line following the educational intervention.
One way to collect this data is to ask parents to check the
activities they do with their child from a checklist (see below) or ask
them to list the ways they spend time with their child or simply ask them
to keep a log of time they spend with their child.
Here are some examples. You can probably think of more. Then
count up the items and tally them pre and post.
Examples:
| Check the activities you have done with your child in the past month: | |
| Played a game | |
| Read together | |
| Spent quiet time together | |
| Had a conversation about something more than chores or rules | |
| Made something together | |
| Visited someone together | |
| Worked on homework together |
| Check the activities you have done in the past year: (in community) | |
| Had a conference with my child's teacher | |
| Had a phone conversation with my child's teacher | |
| Volunteered at my child's school | |
| Donated time or goods to my child's school | |
| Volunteered in my child's activity (4-H, Scouts, Church) |
Another way to measure time is to ask parents to keep a log:
Example:
For the next week, record the amount of time you spend with
your child. List the activity and record the approximate time:
| Monday | Tuesday | Wednesday | Thursday | Friday | Sat. | Sun. | |
| Activity | |||||||
| Time spent | |||||||
| Activity | |||||||
| Time Spent etc. | |||||||
Still another way to measure time spent is to ask parents
to estimate of amount of time they spend with their child (per day or per
week) prior to the learning sessions, during and after the learning sessions.
Don't show them the prior responses for comparison each time. Record those
who are reporting significant more time (15-30 minutes more).
To measure parental involvement in educational and community
functions, target the school of parents or the community organization in
which you are striving for greater involvement. Ask parents and staff of
the programs to estimate amount of time spent in the program pre and then
after training to educate parents about how to effectively be involved
in their child's activities.
To measure adoption of practices you
will need to follow-up with the people you trained a few months after the
session or series with another data collection strategy. Here are some
ideas:
- Call a sample of parents from enrollment lists (every 5th, 10th or 25th name). Make phone calls 6-10 weeks after the session. Ask them 2-3 prepared questions. Record the percentage of who report they have changed their interactions with children or as parents. This is adoption of practices. If you used a sampling method, you can generalize the % who reported to the whole audience to arrive at the reported figure. Sample questions may include : Did you use information presented from the learning session (date, title, location), What specifically have you used more than once? What technique have you shared with someone else?
- An example of results would be that you counted 36/40 who indicate they have adopted practices. Then you can report that 90% of those interviewed adopted practices (36) or if this was a sample of a larger group, obtain a # by reporting 90 (out of 100).
- Use a written sample survey (similar to phone survey but mailed or delivered). Provide a gift or incentive to increase returns. Write up results. For a summary of how to write reports, see this web site and click on Reporting Outcomes - http://www.ces.ncsu.edu/depts/fcs/nnfr
- Hold an end of session focus group. This should be conducted by someone other than the primary instructor. For more information about conducting focus groups, see http://www.ces.ncsu.edu/depts/fcs/nnfr. Information is located under Planning Evaluation (Qualitative).
Record keeping sheets are offered
for your use.
Parents - You will need to assign a number
or a set of initials for each parent so that you know if the same parent
comes to multiple sessions and gains knowledge more than once. Then when
you report knowledge gained, you are only reporting the number of actual
people gaining knowledge and not duplicating this by the number of times
they attended. This is called Recod keeping sheet 1.
Parenting Educators - When you provide educational
outreach for parenting educators please report the numbers, knowledge gain
as well as those who applied for their credential as
a result of your telling them about it. This is called Record keetping
sheet 2.
Child Care Provider - There are nine
areas in which training in the state is categorized. Use Record
keeping sheet 3. Please code by the area trained as well as the
number of children they serve and reports for knowledge gained using the
generic evaluation tool.
All other audiences - other audiences (record
on Record keeping sheet - General) such as
Workable Wellness, Caregivers, Marriage and Couples, Grandparents or others
each have specific questions to report against. Some of the evaluation
forms are found within specific curriculum.
If you use a generic reporting form (see Evaluation Tool) , for reporting purposes, you need the parent's name or initials and the most critical question for knowledge gain is their self-report in question 7. This is the one you will tally and report. (You could essentially eliminate all other questions.)
For reporting adoption of practices, the program should be intense and
occur over a period of time. The teaching may have been a series or some
other intense instructional delivery. If the learner practices a skill
and it is observed, (for example through home visitor or child on site
programs) then the more likely you can be assured they have adopted practices.
The record keeping tools are for ease when transferring data
to the online Extension Reporting System.
Attendance Log
Maintain a separate attendance log from the reporting tools.
The reason you will keep the attendance log separate is that is generally
the "sign-in" sheet for participants with address and phone
number on it. You will need this for your personal records. For this
report, you will need a log that also provides you a record of some sort
of code for each participant. These will be used to tally your final
record keeping sheets. Then you can use these to complete the online
information to submit to ERS. A sample attendance log appears below.
Attendance Log (sample)
| Participant Name | Number of children in family | Address | Phone (or other) | For Office Use Only ( assign a code to each parent - a # or initials) |
| Example (some suggested coding schemes) | ||||
| Sally Hanson | 4 | 123 Oak St. | 555-1234 | SH |
| Sam Hart | 2 | 345 Elm St. | 544-4567 | SH2 |
| OR | ||||
| Molly Beard | 1 | 125 Oar St. | 555-2345 | 01 |
| Maxwell Owen | 2 | 655 Mud St. | 444-2345 | 02 |
Dollar Value to Targeted Audience (may be used at a later date)
Family Life and Parenting Education is in demand by many social agencies, courts and schools. Finding qualified educators is difficult. Extension's contribution is valuable! Please use this record sheet to quantify our contributions. The value of $20 is to just provide internal consistency in this report. Cumulative training hours X $20 per hour.
(Report this in section B on ERS as " Dollar Value to Targeted Audience " but do not attempt to report "Benefit to Society")
More in depth Evaluation
If you are considering doing a more in-depth learning program
and would like to assess the learner's behavior, attitude,
or skill level in a more concise way, you may consult with
a State Specialist, a family evaluator, or read more about
evaluation. To read more on your own, here are a few web
sites:
Program
Development and Evaluation
Evaluation
made easy
CYFERnet
evaluation resources
Targeting
Evaluation
Interactive
Parenting Evaluation site
Glossary
Advocating for children - collaboration on
behalf of families, parent-teacher conference, special support
services, special needs children.
Child Care Training areas -The
North Carolina state legislature passed legislation that required
child care providers to receive training in nine different
areas. This allows providers to balance their professional
development. You will need to categorize yor training by these
areas:
Functional Areas for child care training
1. Safe, healthy Environment
2. Physical & intellectual Development
3. Social and emotional development
4. Productive Relationships with families
5. Effective program operation
6. Commitment to professionalism
7. Observing and recording children’s behavior
8. Principles of child growth and development
9. Learning activities that promote inclusion of children with
special needs
Cumulative training hours -
to avoid duplication, please keep a record of your learning
sessions with the number of participants in each one and the
length of the teaching. Multiple the number of people times
the length of the learning session to arrive at cumulative
training hours for that session.
Family education -
CEMP9 reporting that includes topics related to family management,
family relationships, and personal development (see definitions
below)
Family management -
teaching about household responsibility, managing household
chores, balancing tasks
Family relationships -
changing families, conflict resolution, marriage enrichment,
family meetings, communication, sibling rivalry, fatherhood
GRG - Grandparents raising grandchildren where
they have all or most of the (financial, legal, educational)
responsibility for the children.
Limited Resource Parent - meets the 1971 guidelines
that direct 1890 Extension programs to provide educational
assistance to individuals, families and communities with limited
financial and educational resources. Criteria:
Income level - Household income below 80 percent of the median
income for the country or US Census area in which they live.
Educational Level - individuals and families
whose educational level is below the median educational level
for the country or area in which they live
Social Factors - individuals and families
who live in substandard housing, are welfare dependent, unemployed,
or find their lives extensively influenced by other social
stigmas commonly associated with poverty.
Motivating and guiding children - modeling
appropriate behavior, know how to set limits, stimulating children's
curiosity, and learning. Topics include discipline, problem
solving, motivating your teen, homework help, career choices,
responsibility, acceptance of others, violence, anger management.
Nurturing children- self-esteem communication,
self-respect, health and safety, spiritual development, values,
character education, family fun, death & dying, school
readiness & success.
Parent Education - information and education
disseminated to parents to improve their knowledge about raising
children and practices in rearing children.
Parents - broadly defined as adults caring
for their children including single, foster, grand, step, biological,
teen, etc.
Personal development -
stress management, time management, self-esteem, communication
skills
Understanding child development - knows about
predictable child development stages. This would include such
educational topics as child stages, brain development, physical
development, cognitive development, esteem, sexuality development,
bedtime routines, toileting.
Value to targeted audience -
for this area, we recommend using the value to targeted audience
only and not the cost benefit calculation. The value of $25
has been established as a marker to represent a consistent
figure for all people reporting to use. Use the record keeping
sheet to obtain cumulative training hours as the multiplier.
Workable Wellness - A curriculum introduced
to FCS Extension in October, 2003 that is used on worksites
to increase personal wellness practices. The notebook and a
CD include evaluation requirements and recommendations.
