What is it, and why is it important for the entire process of working with welfare technology?
Effect monitoring can be seen as your insurance policy when working with welfare technology. If set up properly it gives you feed-back on how well your implemented solutions are working in everyday life. This is primarily important because delivering social services is an extremely complex task involving a wide range of parameters (staff, organization, citizens, management geographical districts etc).
If any number of those parameters change over time, the service delivery changes and when that happens you want to make sure that the implemented technology is still performing as needed and as planned.
So having a good system for monitoring the effect of the implemented solutions guarantee that both the municipality and the end-users continue to get the quality that you payed for.
Not many Nordic municipalities have a specific effect monitoring plan, when working with welfare technology. Many see this a part of a good implementation process – to set up a system that checks that the implementation was successful.
In CONNECT we have chosen to make "Effect Monitoring" an independent step, to force municipalities to consider what happens after a successful implementation and to help municipalities ensure they do indeed get a positive return on their investment – both in terms of provided quality but also financially.
Remember, a benefit has not been truly realized until you have successfully implemented a sustainable change. This is much harder than it sounds as giving up old habits and workflows is difficult and often takes time.
As effect monitoring requires extra resources ensure political and/or management backing. This can be done by utilizing the collected data as a tool to justify and promote further investment in welfare technology.
Remember that effect monitoring includes a time perspective. From a quality perspective, the time frame for effect monitoring should be the entire life expectancy of the given technology. But, we understand individual limitation may apply, shortening the period you do the effect monitoring (such as economic or strategy choices). Please note that your "effect goals" for the technology may also include a time perspective which can influence your effect monitoring.
- Example: When implementing a new technology, it will most likely take time before the staff and end-user is fully comfortable with the technology/new service and similarly it will likely take time before the new workflows or organization works efficiently. So, it is likely that a new technology is not working to its full potential from day one and this should be considered when doing effect monitoring.
Effect monitoring provides you with extra data on the everyday use of the technology. Consider how this data can be used going forward – perhaps patterns of use can provide information that benefits future evaluation processes or perhaps it can even help optimize your general service delivery (use aggregated non individualized data to avoid legislative challenges).
Back to top
Guidelines based on experience
Effect monitoring should be a part of both your business case and your implementation plan. It should be clear to everyone how and why this step is necessary and it should be visible early in the process of working with the given technology.
Continues effect monitoring will cost both time and resources, so consider the scale of your investment and adjust the effect monitoring accordingly. You can reuse the evaluation model methodology and scale up or down according to individual requirements.
When collecting data during effect monitoring remember to explore and include existing data sources such as electronic patient journals for cheaper and easier effect monitoring. Similarly, when collecting data for effect monitoring make sure to identify and utilize existing workflows rather than wasting resources creating new ones specific for this process.
Selecting the effect monitoring parameters is very important. We recommend you start by looking at your effect goals for the technology and the already completed technology evaluation. This can be combined with input from end-users as well as the staff responsible for carrying out the actual effect monitoring.
- Example: Effect goals could include a focus on quality of life, work environment, economic gains or similar – use these as starting points for selection of parameters.
If you have procured a service, rather than simply a physical device, remember to include effect monitoring in that service (see step 7 for procurement plan). Furthermore, make sure to specify that delivering effect monitoring means delivering measurable and usable data to the municipality– not simply stating whether or not the service is still working.
Back to top
What has having an effect monitoring plan meant to the CONNECT participants?
Effect monitoring serves as a guideline to help visualize to what extent we have achieved the desired effect of the technology. It gives an indication to whether the goals have been met and continues to be met.
Further, it gives an indication to what extend the resources have been prioritized in a clever way.
Back to top
Methods and tools
Back to top
Examples of effect monitoring from the CONNECT participants
Uses the Sitra handbook on social and health care management. South Karelia finds the handbook helpful both in relation to evaluation and effect monitoring. You can browse the handbook or contact South Karelia for practical examples (see Participants).
Back to top