Outcome-Network.org

An International Database and eJournal for Outcome-Evaluation and Research

Paper

Strategies for improving the quality and effectiveness of services for troubled children and their families: experiences of youth care in Amsterdam

abstract

Background and goal of the project. Making youth care interventions evidence based is the ambition of several organizations in the Netherlands. Veerman & Van Yperen (2007) have made a sequence of stages on how to reach the status of evidence based:

  • describe the youth care that is carried out in the daily practice and the problems of the children and their parents that are aimed by the program (the intervention is then called 'potentially effective');
  • give the program or intervention a thorough theoretical basis, formulate a hypothesis on why the intervention should work for children and parents with the described kind of problems (the intervention is 'promising');
  • the intervention has a positive outcome measured by problem reduction, client satisfaction, low drop out rate, high rate of goal attainment (the intervention is 'efficacious');
  • in a random controlled trial one finds positive results of the program; thr intervention has better results than when doing nothing or carrying out an alternative intervention (the intervention is 'effective').

In those four stages one can climb the ladder to an evidence based intervention or program. These stages are used to design research and innovatory activities to improve the quality of youth care programs. Three strategies were used for improving the quality of the programs. First, we searched in research data for specific ingredients to improve the effectiveness for the children and parents with problems on which the program is aiming. Secondly, we implement common factors like good professional training, a good match between the needs of the clients and the program, structured supervision, carrying the program out in the way it is meant to be and so on. Programs with those factors give better results than programs that lack these, regardless of the problems of the clients. Thirdly, one should monitor the outcome of the program continuously, evaluate the results, discuss changes that improve the quality and the outcome and implement them.

At Spirit, a youth care organization in Amsterdam and surroundings, a research is done for the degree of effectiveness of some of the intervention programs. The intervention programs already existed for a certain amount of time and were developed in collaboration with professional organizations. They were not known as 'evidence based' programs, but had already proven to be successful in practice. Spirit asked the Netherlands Youth Institute to set out a research program to find out how to improve those practice-based programs to be practice- ánd evidence based in the end. On the basis of these researches Spirit was advised how to improve the intervention programs. Today Spirit is developing a way to further improve these programs. Spirit is implementing some of the advises and has developed, together with the Netherlands Youth Institute, a way of continuously monitoring the results and program integrity of the interventions.

Methods. First was analyzed which (ingredients of) interventions are necessary to get the best results for the target group of the investigated programs. A search was done for these ingredients in scientific research data on intention effectiveness. The comparison of those evidence based ingredients and the concrete ingredients that were described in the program manuals, delivered some advice for improvement. Secondly, client information was analysed to conclude whether the program is aiming at the clients who profit the most from the program. Thirdly, interviews were done with some social workers to find out whether they work according to the 'common factors to all approaches', and whether they carry out the program as described, because that should give the best results.

The case of non-residential social work. The non-residential social work of Spirit focuses on a wide target group: families, often one-parent families, with children between 0 - 18 years with problems on several areas. It involves problems with raising children, housing, finances, relational problems between parents, keeping a social network and keeping a social position. Some parents are in (need of) psychiatric care or suffer from an addict. We often speak of multiproblem families. Children often have specific problems as behavioural problems, oppositional behaviour, Adhd, etc. About 50% of the target group has a different ethnic background. The non-residential intervention program takes about 9 months and consists of three stages. In the first motivational stage, the network of the family is explored as well as the problems children and parents experience. Together with the family, the social worker makes a first action plan. In the stage of change, which usually takes about 6 months, the social worker carries out different intervention techniques, like video home training, behavioural training, empowering the social network of the family etc. In the third stage, the social worker supports the family on maintaining the new learned skills and network contacts.

Findings: is the intervention program effective? Research showed that the official target group of the program matched the group of clients in practice. The literature search showed that the theoretical basis of the program was positively matching the main points of the literature-analyses on necessary ingredients to get the best results for the target group. These are:

  • the need for empowering the positive forces in the families (empowerment-theory);
  • learning to negotiate about situations (communication-theory);
  • reinforcing the authority of the parents (systemic theory);
  • change of behaviour by instruction, practice, reward of correct behaviour modelling (learning-theory).

So the 'program theory' is sufficiently embedded in the intervention program of the non-residential social workers or in other words: the described interventions in the program manual are in general the best interventions for the target group. But research also suggested to develop more strategies specifically focused on children, to learn them to cope with some of their behavioural problems. Researchers also suggested to be more specific in diagnosing the problems of the families, where Spirit tends to focus mainly on the strengths of the families. Questionnaires could be used more often. Another suggestion was to use more psycho-education, for instance for parents about Adhd. Also cognitive restructuring should be used more often to change children's behaviour. Children with oppositional behaviour often have a distorted vision of reality which leads to explosions of anger; in the Netherlands we can make use of several training programs for anger management for children. Next, the program integrity was researched. Interviews with social workers learned that they incorporated many elements of the intervention programs. The intervention manual however did not function enough for them as the main point of focus. The choice of specific intervention techniques often depended more on the expertise and experience of the social worker, than on the problems of the family. There are only a few results of the intervention available. The degree of dropout was 8 - 9%, which is an average result. Clients judged the intervention program to be a 8.6 on a scale of 0 - 10. Overall, the research showed the program to be 'potentially effective'. The program is described well. The theoretical background matches the findings from the literature search. Is the intervention program 'promising'? Almost. When the intervention program is extended with some specific interventions for children for instance, the program will fit better for the target group. Also, the social workers have to learn to work more according to the methodical guidelines to be more effective in their work. Only a thorough research program, like a randomized clinical trial, could tell us if the intervention program is really 'evidence based effective'.

 Implementing change. The findings motivated Spirit to start implementing some changes. One of the most important advises the Netherlands Youth Institute gave, was on starting a method of continuously monitoring the results and program integrity of the interventions. This advise was favourably received. Spirit has organized a pilot project on this subject. In this project Spirit asked all non-residential teams to look back on about ten of their last clients, and ask themselves three questions:

  • do we get the right target group, according to our intervention program;
  • which interventions worked well, which worked less, in our opinion;
  • what worked well for which clients.

The main goal of this 'method evaluation' as Spirit calls it, is to monitor on a more systematic base if our interventions still match with the needs of our clients. Of course, this is not a scientific approach. The strength however is that all social workers are motivated to look back and discuss the pro's and con's of their intervention techniques. In our experience, many of the teams have a strong desire to evaluate their work, and this method evaluation gives them a good framework to do this. Last may 2007, the teams have made their first method evaluation. This way of monitoring the results and program integrity of the interventions in the teams will be repeated twice a year and will also be implemented for other programs of Spirit.

At the moment Spirit is comparing different evidence based interventions for coping with aggression by children with the purpose to eventually select one of these interventions to use within all programs of Spirit (if necessary). The same is done for different social skills trainings. Method evaluation is an instrument to constantly improve the quality of the intervention programs on mainly practice based evidence. In collaboration with universities a more scientific search on effectiveness of a few intervention programs of Spirit will also be done.

Key references

Veerman, J. W. & Yperen, T. van (2007). Degrees of freedom and degrees of certainty. A developmental model for the establishment of evidence-based youth care. Evaluation and Program Planning, 30, 212-221.

Contacts: Ellen Schulze, Spirit Fred. Roeskestraat 73, 1076 EC Amsterdam, the Netherlands, E-mail: e.schulze@spirit.nl, Phone +31 (0) 20 54 00 544.

© copyright 2024 Outcome-Network.org all rights reserved, in partnership with FondazioneZancan | iaOBERfcs | read the legal notice.