Skip Navigation
France

Open Access to Public Policy Evaluations (FR0006)

Overview

At-a-Glance

Action Plan: France, First Action Plan, 2015-17

Action Plan Cycle: 2015

Status: Inactive

Institutions

Lead Institution: Ministry of State for State Reform and Simplification attached to the Prime Minister

Support Institution(s): NA

Policy Areas

E-Government

IRM Review

IRM Report: France End-of-Term Report 2015-2017, France Mid-Term Progress Report 2015-2017

Starred: No

Early Results: Marginal

Design i

Verifiable: No

Relevant to OGP Values: Access to Information Civic Participation

Potential Impact:

Implementation i

Completion:

Description

STAKES Evaluation of public policies helps decision-makers make public action more relevant, effective and efficient and contributes to informing citizens. The stake of evaluation goes beyond those of its direct protagonists and concerns all citizens. Nevertheless, access to these various works remains complex due to the large number of players involved and the multitude of distribution media used.

CONTEXT & AIM The Prime Minister’s circular dated on February 23th, 1989, (10 May 1988/15 May 1991), on the renewal of public service, was an important step in the affirmation of public policy evaluations as a tool for government modernization. Evaluating a public policy means judging its value with regard to a set of criteria, in order to improve and inform decision-making. According to the decree of 18 November 1998, it means "assessing, within an interministerial framework, the effectiveness of this policy by comparing its results to objectives assigned and resources used". Today, public policy landscape is marked by numerous stakeholders: Parliament, Ministries, the Cour des comptes, the Economic, Social and Environmental Council (CESE), the Regional Economic, Social and Environmental Councils, local and regional authorities, State agencies, etc. The diversity of actors involved in evaluations and of media used for publication now raises questions about the legibility and impact of these evaluations in public decisionmaking. Furthermore, citizens aspire to get more involved in the evaluation and design of public policies. The government will undertake a process involving all these actors so that, gradually, citizens will have more visibility on the work carried out, reports will increasingly be made public and contribution of evaluations in public decision-making will be more legible. To do this, the government primarily relies on the best practices of its own approach. 59 evaluations have been launched since the end of 2012, covering numerous public policy fields (social affairs, national education, justice, ecology, housing, etc.). In June 2015, 43 reports were published on the web site of the Secretary-General for Government Modernization12 (SGMAP)(some evaluations are still ongoing). In the dedicated area, a gauge indicates the progress of each evaluation (by large milestones: launch, diagnostics, transformation scenarios, report, implementation), along with the related documentation. The methodology promoted by the SGMAP recommends to systematically involve users benefiting from the evaluated policy (through consultations, polls, ad hoc surveys and workshops), and for stakeholders to be invited to monitor the evaluation work as part of an "evaluation committee". The SGMAP also provided its support and contributed to funding the project run by the French Evaluation Society (SFE), which has been developing, since 2012, a Public Policies Evaluation Observatory. The SFE gathers 300 members (230 individual members and 70 institutional members: State services, local and regional authorities, public institutions, associations and consulting firms). This observatory has already listed more than 1600 references for evaluations made since 2000, and published summaries of corresponding reports for half of them. This initiative is a first step towards the creation of a resource center that will offer an easy access to all evaluation work, in order to help members of Parliament in their mission of drafting bills and controlling the government, as well as decision-makers, public administrators and citizens.

ROADMAP
• Encourage all actors involved in evaluations to make their work available in the Public Policies Evaluation Observatory, to improve its comprehensiveness and facilitate research through its database (access by keyword, etc.)
- Make the Public Policies Evaluation Observatory available to all, as the reference database for public policies evaluations
- Systematize the publication of public policies evaluations launched in the context of government modernization programs (when not prevented by legal considerations)
• Systematize and improve citizens' participation in evaluations coordinated by the SGMAP, for example through opinion surveys, workshops for citizens' juries, etc.
• Improve the traceability of impact on public action of public policies evaluations launched under government modernization programs

IRM End of Term Status Summary

4. Open access to evaluations of public policies and to their conclusions

Commitment Text:

ROADMAP

• Encourage all actors involved in evaluations to make their work available in the Public Policies Evaluation Observatory, to improve its comprehensiveness and facilitate research through its database (access by keyword, etc.)

Make the Public Policies Evaluation Observatory available to all, as the reference database for public policies evaluations

Systematize the publication of public policies evaluations launched in the context of government modernization programs (when not prevented by legal considerations)

• Systematize and improve citizens' participation in evaluations coordinated by the SGMAP, for example through opinion surveys, workshops for citizens' juries, etc.

• Improve the traceability of impact on public action of public policies evaluations launched under government modernization programs

Editorial Note:This is a partial version of the commitment text. For the full commitment text please see France's national action plan: https://bit.ly/2MTYhsR.

Responsible Institution: Secretary-General for Government Modernisation (SGMAP) attached to the Prime Minister

Supporting Institution(s): N/A

Start Date: Not Specified 

End Date: Not Specified

Commitment Aim

This commitment sought to make public policy evaluation reports more accessible to the public. Public policy evaluations facilitate relevant and efficient public action, and contribute to better-informed citizens. However, access to evaluations is limited due to the number of actors involved and the different methods used. This commitment aimed to make evaluations accessible to the public, encourage the use of evaluation by public officials, and involve citizens in the evaluation of public policies.

To this end, the Secretary-General for Government Modernization (SGMAP) has set up a partnership with the French Evaluation Society (Société Française de l'Évaluation, or SFE), an association which maintains a repository for public policy evaluations. The repository (called the “Observatoire” in French) is a database of public policy evaluations from agencies at different government levels (state, local authorities, etc.). It also gives users the opportunity to submit a report themselves.

Status

Midterm: Limited

Completion of this commitment was limited at the midterm. There was a significant difference in progress between the commitment activities; the public policies evaluation repository was complete while the other two activities, involving citizens in public policy evaluation and encouraging officials' use of the evaluation repository, had not started.

The Public Policies Evaluation Repository was made accessible online, free of charge. An increasing number of evaluations were published during the first year of implementation. The midterm IRM report notes that the search filters were inadequate given the number of evaluations. Importantly, stakeholders signalled that the technology used in the repository was not open source as the software was proprietary. Lastly, the repository did not have a dashboard to follow the number of downloads and contributions.

The midterm IRM report indicates that there was no public evidence that the other two activities had started. For more information, please see the IRM midterm report.[Note48: Independent Reporting Mechanism, France Rapport D'ètape (OGP, 2017), https://www.opengovpartnership.org/sites/default/files/France_Progress-Report_2015-2017.pdf.%5D

End of Term: Limited

The government, with SGMAP in the lead, took some steps forward regarding the production and publication of public policy evaluations. However, the government self-assessment does not list any new developments since the midterm assessment. The overall objectives of the commitment were not achieved by the end of the implementation period. Of particular note is the fact that there is no evidence that citizens were given a chance to participate in public policy evaluations nor that the impact of public policy evaluations is more measurable as a result of this commitment.

Based on desk research, the IRM researcher notes that it is now possible to use multiple filters to search through evaluations. However, a significant number of evaluations are not linked to any document or report, making the listing of evaluations less relevant and useful.

Regarding the evaluations launched by the government in 2012 as part of its efforts to modernise public action, 80 evaluations are listed on the dedicated page on the SGMAP's website[Note49: Available at http://www.modernisation.gouv.fr/laction-publique-se-transforme/en-evaluant-ses-politiques-publiques/toutes-les-evaluations-de-politiques-publiques.%5D and 76 feature a public report. The webpage also contains an explanation of the methodology used to evaluate policies, as well as examples of documents and instruments used during evaluations.

The government self-assessment indicates that an independent evaluation of SGMAP's methodology and assessment of public policies has been undertaken by the auditing firm KPMG and Quadrant Conseil between July 2016 and January 2017. The report is available on the SGMAP's website.[Note50: Available at http://www.modernisation.gouv.fr/laction-publique-se-transforme/en-evaluant-ses-politiques-publiques/levaluation-des-politiques-publiques-evaluee-a-son-tour (accessed 9 Jan. 2018).] The report recommends that:

· transparency and stakeholder participation be reinforced;

· the methodology be more selective in the evaluation themes as not all policies can be evaluated the same way;

· the methodology become more flexible and less time-consuming; and

· that evaluations be combined with other methodologies.

The evaluation highlights the absence of a monitoring mechanism that would allow for an improved follow-up to policy evaluations. It also notes that innovative data collection techniques (e.g. big data, predictive analysis, or citizen consultations) are not used.[Note51: SGMAP, L'évaluation des politiques publiques évaluée à son tour : enseignements et perspectives, (SGMAP, 2017), http://www.modernisation.gouv.fr/laction-publique-se-transforme/en-evaluant-ses-politiques-publiques/levaluation-des-politiques-publiques-evaluee-a-son-tour (accessed 22 Sept. 2017); KMPG, Quandrant. Evaluation de la démarche globale d'évaluation des politiques publiques menée dans le cadre de la modernisation de l'action publique, (KMPG, 2017), http://www.modernisation.gouv.fr/sites/default/files/fichiers-attaches/evaluation_epp-map_rapport.pdf.%5D

Finally, the IRM researcher did not find any public information that points to the completion of the activity seeking to make the impact of evaluations on policy more traceable.

Did It Open Government?

Access to Information: Marginal

Civic Participation: Did Not Change

The midterm IRM report considered the milestones of this commitment to have a negligible potential impact because a free library of public policy evaluation reports already exists (La documentation Française[Note52: Stéphane Saurel, “What is the Budget for the European Union after 2020?” (La Documentation Francaise, accessed 25 Mar. 2018), http://www.ladocumentationfrancaise.fr/.%5D) and the activities for increasing citizen participation in SGMAP evaluations and monitoring the evaluations' impact are vague and lack measurable actions. SGMAP has published over 70 public policy evaluations on its website and has started a discussion on how to improve the traceability and impact of public policy evaluations. As such, the implementation of this commitment marginally opened government regarding access to information, given the availability of new information and improvements in the search functions of the Observatory during the second year of implementation.

However, civic participation showed no improvement. The evaluation produced by KPMG and Quadrant highlights the absence of public consultations. During Etalab's forum, civil society noted that public evaluations should gather feedback from different stakeholders using a variety of tools, while a participative evaluation should involve stakeholders starting from the very conception of the evaluation in a steering committee.[Note53: “Thomas” (comment), “Commitment 4: Open access to public policy evaluations and their conclusions: Mid-term self-evaluation report of the Action Plan for France 2015-2017: ‘For a transparent and collaborative public action,'' (Etalab, accessed 25 Mar. 2018), https://forum.etalab.gouv.fr/t/engagement-4-ouvrir-lacces-aux-evaluations-de-politiques-publiques-et-a-leurs-conclusions-rapport-dautoevaluation-a-mi-parcours-du-plan-daction-pour-la-france-2015-2017-pour-une-action-publique-transparente-et-collaborative/1905/3.%5D He mentions Région Rhône-Alpes, Rennes Métropole and Nantes Métropole as local governments having implemented such methods. Civil society also recommends that evaluations not only be commissioned by public authorities but also by civil society, that evaluators be trained in public consultations to make sure both intermediary bodies and beneficiaries are involved in the process, that a digital evaluation method be piloted at the local level, and that all the data surrounding evaluations be opened.

Carried Forward?

This commitment was not carried over to the new action plan.


France's Commitments

  1. Transparency of Public Services

    FR0030, 2018, E-Government

  2. Transparency of Public Procurement

    FR0031, 2018, E-Government

  3. Transparency of Development Aid

    FR0032, 2018, Aid

  4. Expand Open Data

    FR0033, 2018, E-Government

  5. Improved Data Policies and Administration

    FR0034, 2018, Capacity Building

  6. Transparency of Public Algorithms

    FR0035, 2018, E-Government

  7. Open Data at Sub-National Level

    FR0036, 2018, Capacity Building

  8. State AI Lab

    FR0037, 2018, Capacity Building

  9. Administrative Capacity-Building

    FR0038, 2018, Capacity Building

  10. Public Service Incubators

    FR0039, 2018, Capacity Building

  11. Streamline Data Flows

    FR0040, 2018, E-Government

  12. Open Etat Forum

    FR0041, 2018, E-Government

  13. Online Procedures Dashboard

    FR0042, 2018, E-Government

  14. Govtech Summit

    FR0043, 2018, Capacity Building

  15. Imrove Public Consultation Mechanisms

    FR0044, 2018, E-Government

  16. International Transparency and Citizen Participation

    FR0045, 2018, Aid

  17. Public Pariticipation in Sustainable Development

    FR0046, 2018, Capacity Building

  18. Open Science

    FR0047, 2018, E-Government

  19. Citizen Involvement in Cour Des Comptes

    FR0048, 2018, Capacity Building

  20. Private Sector Transparency

    FR0049, 2018, Asset Disclosure

  21. Access to Information on Public Officials

    FR0050, 2018, Asset Disclosure

  22. Open Regional and Local Authorities' Data

    FR0001, 2015, Fiscal Transparency

  23. Publish Municipal Council Decisions and Reports Online

    FR0002, 2015, E-Government

  24. Publish Building Permits in Open Data Format

    FR0003, 2015, Open Data

  25. Starred commitment Increase Transparency in Public Procurement

    FR0004, 2015, Open Contracting and Procurement

  26. Improve Transparency in International Development Aid

    FR0005, 2015, Aid

  27. Open Access to Public Policy Evaluations

    FR0006, 2015, E-Government

  28. Involve Citizens in Cour Des Comptes Work

    FR0007, 2015, Fiscal Transparency

  29. Access to Public Officials Transparency Obligations

    FR0008, 2015, E-Government

  30. Starred commitment Beneficial Ownership

    FR0009, 2015, Beneficial Ownership

  31. Transparency in Extractive Industries

    FR0010, 2015, Extractive Industries

  32. Transparency in International Trade Commercial Negotiations

    FR0011, 2015, Labor

  33. Fix My Neighborhood

    FR0012, 2015, E-Government

  34. Digital Fix-It

    FR0013, 2015, Open Data

  35. Co-Produce Data Infrastructure with Civil Society

    FR0014, 2015, Open Data

  36. Starred commitment Open Legal Resources

    FR0015, 2015, Legislation & Regulation

  37. Reform Participatory Mechanisms

    FR0016, 2015, Public Participation

  38. Mediation and Justice

    FR0017, 2015, Judiciary

  39. Starred commitment Open and Circulate Data

    FR0018, 2015, Land & Spatial Planning

  40. Open Calculation Models and Simulators

    FR0019, 2015, Open Data

  41. Open Platform for Government Resources

    FR0020, 2015, E-Government

  42. Improve Public Services Through E-Government and User Interaction

    FR0021, 2015, E-Government

  43. Empower Civil Society to Support Schools

    FR0022, 2015, E-Government

  44. Diversify Recruitment Within Public Institutions

    FR0023, 2015, Capacity Building

  45. Culture Change

    FR0024, 2015, Capacity Building

  46. Spread Public Innovation

    FR0025, 2015, Capacity Building

  47. Starred commitment Protect Against Conflicts of Interest

    FR0026, 2015, Conflicts of Interest

  48. Civil Society & Transparency in COP21 Conference Planning

    FR0027, 2015, Environment and Climate

  49. Open Data and Climate/Sustainable Development

    FR0028, 2015, Open Data

  50. Collaborate with Civil Society on Climate and Sustainable Development

    FR0029, 2015, Environment and Climate