8.1

The evaluation design is both quantitative and qualitative.

Qualitative data will be acquired in a participatory evaluation. Implementation of the evaluation will be based on lessons learned from the Communities In Schools participatory evaluation underway since 1997. The evaluation has already begun with the development of goals, objectives and success indicators for the program.

The purpose of the participatory evaluation is to improve the implementation of this project and to provide information for others who might want to replicate this process.

8.1.2

The projected evaluation committee includes the director of the Community Education Network; the chair of the Community Studies department at the College of the North Atlantic; the regional director of Communities In Schools; the executive director of the Long Range Regional Economic Development Board, and the consultant.

Reporting and facilitating the evaluation is the responsibility of the consultant. However, as a participatory evaluation reflects the observations and opinions of all stakeholders, each community also has responsibility for implementation.

As in the CIS evaluation, we use video for data collection so our resource requirements are videotapes (as well as ENG and editing equipment supplied by Ryakuga).

8.1.3

During the CIS evaluation, it was found that the video data collection process surfaced issues which did not arise in the normal reporting process. It was possible to respond immediately with the implementation of positive change in the process. The evaluation committee, in its monthly meeting, references the data collected to the established goals, objectives and success indicators of the project.

8.1.4

1. By April 2000, the Sharing Our Future overall steering committee will meet to establish a meeting schedule and approve collaboration agreements with local committees as well as the hiring of local community communication facilitators.

2. By April 2000, the Sharing Our Future local steering committees will meet to establish a meeting schedule and approve collaboration agreements with the overall steering committee as well as the hiring of a local community communication facilitator.

3. By May 2000, the Sharing Our Future community communication facilitators will begin working in the designated communities.

4. By May 2000, the new and established community communication facilitators will have been offered training programs.

5. By May 2000, the participatory evaluation team will have been formed with local representation.

6. By May 2000, the participatory evaluation process, utilizing video as a data collection tool, will begin.

7. By July 2000, the SOF summer program will begin.

8. By September 2000, the community communication facilitators will have recruited and trained teams of active community communication volunteers. This process will be ongoing.

9. By September 2000, Sharing Our Future will reach out to expatriate Newfoundlanders by means of advertising in provincial and national media - and internet communication, principally internet broadcasting of local community television events.

10. By October 2000, the community communication facilitators will have initiated regular weekly programming on community television and/or radio.

11. By October 2000, the Long Range Regional Economic Development Board and the Community Education Network (with the support of Sharing Our Future) will have initiated a program of networking (utilizing internet communication tools such as video conferencing, list serves and web boards) with professionals and groups in Atlantic Canada. The networking will focus on community economic development, community education and participatory community communication.

12. By November 2000, community communication facilitators will be working:
(A) To help promote existing community access program centres.
(B) To help organize community access program centres where they don't exist.

13. By March, 2001, the overall steering committee will have identified three other communities which want to replicate the Sharing Our Future process.

14. By April 2001, collaboration agreements will have been signed between the overall steering committee and the local committees of the replication communities.

15. By May 2001, community communication facilitators will be working in the replication communities.

17. By July 2001, the Sharing Our Future summer programs will begin in the communities.

18. By September 2001, prototype learning guides will be available.

19. By December 2001, the replicant communities will have all produced community forum/webcasts.

20. By March 2002, the overall steering committee will have found alternate sources (including local support) to funding the community communication facilitator positions.

21. By March 2002, Sharing Our Future will have produced a replication video/guide and written report on the project.

8.1.5

Quantitative data will be presented in the monthly reports of the facilitators and the consultant.

Facilitators' reports will include hours devoted to Sharing Our Future activities; number of active volunteers ; number of interactions on community forums (and issues raised); descriptions of emails/requests from expatriates (and questions asked); meetings of local committees. Another data source is the community forum program log which is included with the facilitator's monthly report. Finally the facilitator is requested to each month update the community communications needs, skills and resources assessment.

Further quantitative data (number of interactions, issues discussed) will be available from the web board set up for facilitator networking. The board is moderated by CEN representatives and the consultant with results included in the consultant's monthly report. Other data which may be extrapolated from the consultant's report include number of meetings of the overall steering committee and the evaluation committee; number of video conferences with Atlantic communities; advertising; webcasts; training events, and community visits.

The actual quantitative requirements will have to be worked out in initial meetings of the overall steering committee and the evaluation committee. But it would seem reasonable to project - the facilitators will devote 20 hours a week to SOF activities; a community forum will be held once a month in each community; there will be one nationally advertised webcast each month; the local committees and the evaluation committee will meet monthly and the overall steering committee will meet twice a year; facilitators and the consultant will report monthly; the evaluation committee will report twice a year; facilitators will be required to post to the web board each week, and there will be two training events each year.

(At the time of this writing, we are discussing the participation of Sir Wilfred Grenfell College in coordinating surveys of the public to assess the effectiveness of the project. Dr. Ivan Emke - a professor at the college and member of the SOF developmental stage advisory group - has already surveyed the public in Burgeo and Ramea concerning community television programming.)

Qualitative data is collected on videotape during interviews with the stakeholders. (See the participatory evaluation guide attached to the developmental stage report.) In this project, potential stakeholders include the community communication facilitators, volunteers, members of local committees and the community at large.

The interviews are recorded on digital videotape; dubbed to Hi-8 for offline viewing, and then to VHS with visible timecode for analysis by the evaluation committee.

8.1.6

During the CIS evaluation process, the interviews surfaced information which allowed coordinators to make immediate changes to the implementation of the program in the community. Also in the SOF evaluation process, it is the responsibility of the evaluation committee to meet monthly to analyse the data with reference to the success indicators of the project.

And, of course, the standard evaluation questions of - What worked best for you? What should be done differently? What advice would you have for others wishing to implement a similar program? - produces information which demands adjustments to be made.

8.2

The evaluation is designed to allow for continuous and incremental change in the program.

This participatory, grassroots evaluation process typically includes two reports annually - in writing and on video. The video report in itself is a useful method of letting others see how the project is viewed by its stakeholders. It becomes a tool for replication in the second year.