SPARK has recognizes the importance and need for continuous research, monitoring and evaluation and is committed to learning from past projects. SPARK utilizes a Results Based Monitoring (RBM) approach coupled with innovative monitoring tools and sector leading transparency initiatives (e.g. Transparkency). We firmly believe this approach helps us continuously analyse, improve and further develop our capacity to deliver services and implement SPARK’s interventions so they are more effective and achieve better results. To support this aim SPARK employs a web based management information system (MIS) to facilitate its monitoring and evaluating activities. Through MIS, our programme management teams have quick access to the latest program data allowing them to proactively monitor their projects, including identifying outcome targets that demand more attention. Additionally, programme management teams can avail of the latest chart technologies built into MIS, for meetings with beneficiaries, local partners and donors ensuring that discussions on monitoring and learning are that much more informed and based on the latest data.
Externally, SPARK relies on mid-term and end-of-program reviews conducted by expert external evaluators (only the best are chosen). Notably, upon completion, all large, multi-year projects are externally evaluated. Findings from these important and rigorous reviews are a central part of the continuous learning loop that is vital to our organization. Evaluation reports are shared throughout the organization so that each department has an understanding of how a project, and their role in the project, has performed. Indeed, such findings are the driving force behind SPARK’s sector leading innovations. New proposals and programs are built upon evaluation findings ensuring SPARK absolutely does not rest on its laurels but remains dynamic and at the forefront.
SPARK’s monitoring and evaluation (M&E) system utilises a standard results monitoring approach coupled with innovative web-based monitoring tools and sector leading transparency initiatives. This comprehensive approach allows for a more dynamic view by our programme management team on project progress up to completion and donors and stakeholders to have a clear view to how our project funds are spent.
Results monitoring is the core of SPARK’s monitoring system. The projects logical framework is translated into the final Monitoring Plan that then serves as performance measurement framework for the duration of a project. The final Monitoring Plan lists outputs and outcomes targets to be attained at key milestones and are grouped by strategy/objective. The plan includes, per SMART indicator the means of verification and the frequency of data-collection.
In measuring results, SPARK is guided by the Donor Committee on Enterprise Development (DCED) Standard on Measuring Achievements in Private Sector Development and its compendium for Conflicted-Affected Environments. Where relevant, indicator definitions are adopted that are universally accepted in this standard.
Following a Results Monitoring programme baseline values for planned results at the outcome level are determined and incorporated in the Monitoring Plan. Often project specific surveys are required and when so, the chosen survey methodology is documented and a database maintained to allow for post project surveys. To enhance the rigor and validity of such single-difference measurements, baseline and post project surveys are also conducted with a comparison group enabling a double-difference measurement to be made and therefore allowing SPARK insight into the impact of a project.
SPARK’s reporting procedures include project report format that closely mirrors the projects Monitoring Plan. Reporting frequencies are set and adhered to, based on agreements with donors with narrative and financial reporting supplied to donors synchronously via SPARK’s online expenditure tracking and financial management system, TransSPARKency.
Upon completion all large projects are externally evaluated by an independent evaluator. SPARK favours mixed methods approaches for end of project evaluations. As such we have made use of video based Most Significant Change methodology to compliment quantitative outcome data produced by our Management Information System (MIS).
Management Information System (MIS)
SPARK has developed a proprietary web-based project monitoring system, in which we enter our project monitoring plans, output indicators, outcome indicators, targets and indicator definitions. This allows various levels of programme management to enter and review target data provided by our partner organizations. The system has been designed to be easily accessible by all of our field offices via our intranet.
Built into the system is a review and verification process that begins with gathering and entering indicator data from partner organizations by our local project officers, this is then reviewed and verified by the Country Manager and ends with the final verification of data by the Programme Manager. The process flow ensures we adhere to a ‘four eyes principle’.
Once data has been reviewed and approved it is used to feed charts designed to keep SPARK’s Management further up to date via easy to read charts.
In parallel to MIS SPARK began using Salesforce in 2013 as a parallel web-based monitoring system. Built on the Salesforce platform the system is designed to help our partners track the businesses they have supported over several projects. The system focuses on SPARK core indicators i.e. jobs created and businesses supported and the World Bank Doing Business indicators.
Additionally, partners are able to keep better track of training and coaching services businesses have participated. To take advantage of the rich data captured in Salesforce, numerous dynamic reports have been created. Utilising these reports SPARK, partners and businesses supported are able to keep track of the performance of a business (e.g. cash flow, profit and turnover) and also their progress in upgrading their administrative systems with the aim of being able to better identify training and coaching needs.
Another SPARK proprietary system designed so that project financial performance can be tracked in a result based way and value for money of projects can be calculated. The system provides both inputs (mostly financial data) and outputs and outcomes as well as additional cross references, e.g. the inputs necessary to produce related outputs and outcomes for example cost per SME started.
SPARK was one of the first organizations to adopt the global transparency standard making public all of our project budgets and how those budgets have been spent. Over 300 organizations now participate in IATI, including the Bill & Melinda Gates Foundation, Oxfam GB, UNDP and the World Bank. Participating IATI takes our own TranSPARKency system to another level allowing the public insight how our projects budgets have been managed.