Evaluate Open Data (RO0064)
Action Plan: Romania Action Plan 2018-2020
Action Plan Cycle: 2018
Lead Institution: Secretariat General of the Government (SGG) — Directorate for Information Technology
Support Institution(s): The Organisation for Economic Co-operation and Development (OECD)
Policy AreasAccess to Information, Capacity Building, E-Government, Open Data
Measuring the impact of the re-use of open data
2018-2020 Lead implementing agency/actor Secretariat General of the Government (SGG) — Directorate for Information Technology Other actors involved others The Organisation for Economic Co-operation and Development (OECD) CSOs, private sector, multilaterals, working groups What is the public problem that the commitment will address? At the European Union level, open data maturity in member stated is assessed yearly (since 2015) by the European Commission. For the 2015-2017 reports, 2 main issues were assessed: open Data Readiness level, which is closely linked to the existence and implementation of the necessary policies and the maturity of the Open Data Portal. Starting with the 2018 analysis, 4 evaluation topics are being assessed: Open Data Policy, Open Data Impact, Open Data Portals and Open Data Quality. From the 2018 Report it resulted that Romania is lagging behind on open data re-use and measurement of OD re-use and impact. Commitment description What is the commitment? Under the project Capacity building in the field of public governance of Romania, re-use and impact indicators of open data will be developed. How will the commitment contribute to solve the public problem? Development of impact indicators on the publication of open data Why is this commitment relevant to OGP values? Increasing the re-use of open data from public administration Milestone activity with a verifiable deliverable Responsible agency / partner Start Date: End Date: Development of indicators and methodology for measuring open data reuse and impact SGG/OECD 2019 2019 39 Start monitoring and evaluation of open data re-use and impact SGG 2019 2020 Activities to promote the re-use of open data SGG 2019 2020 Additional information Budget EEA / Norwegian Financial Mechanism 2014-2021 Correlation with other government programs/strategies National Strategy for the Digital Agenda
IRM Midterm Status Summary
17. Measuring the impact of the re-use of open data
Commitment Text: "Under the project Capacity building in the field of public governance of Romania, re-use and impact indicators of open data will be developed."
- Development of indicators and methodology for measuring open data re-use and impact
- Start monitoring and evaluation of open data re-use and impact
- Activities to promote the re-use of open data
Start Date: 2019
End Date: 2020
Editorial Note: The commitment text is abridged. The full text can be found in the OGP 2018-2020 national action plan.
Context and Objectives
Since 2018, the European Commission assesses open data maturity in Europe using four dimensions: (1) open data policy, (2) open data impact, (3) open data portals, and (4) open data quality. According to the European Commission’s 2018 findings, Romania lags on open data reuse and on the measurement of open data reuse and impact.  Moreover, according to an interviewed OGP representative, the lack of a methodology for and measurements of the (re)utilization and impact of open data is a problem that many countries are confronted with.  This commitment aims to develop impact and (re)utilization indicators, monitor and evaluate the (re)use and impact of open data, and promote its (re)use among civil society and the business sector. Through its activities, this commitment furthers access to information and civic participation.
The solutions proposed by this commitment are threefold:
- The Secretariat General of the Government (SGG) will develop indicators and a methodology for measuring open data reuse and impact, in cooperation with the Organization for Economic Cooperation and Development (OECD), through the project funded through the European Economic Area (EEA) and Norwegian Financial Mechanism 2014-2021. According to the OGP representative, the unofficial agreement and main concepts exist, whereas all else are still being defined. 
(2) SGG will start monitoring and evaluation of open data reuse and impact.
(3) SGG will promote the reuse of open data.
The milestones as written are specific enough to be verifiable, but lack key details. The OGP representative when interviewed, provided additional detail, such as that the OECD project "Capacity building - Data Centers of Government" will provide indicators and methodology to measure the (re)utilization of open data, and will tailor them to Romania. According to the SGG representative, the SGG plans to consult civil society on how to apply the indicators to the Romanian case, although tailoring details are missing.  Monitoring and evaluation will reportedly be done annually, and will be used to show the public administration the commercial and administrative benefits, but there are no details on how the monitoring will be done, on which methodology will be used, or the target group. Finally, SGG will measure the satisfaction of open data creators and users with the open data portal and ask them to identify high-value datasets;  collect examples of open data (re)utilization;  and add new examples of open data reutilization to a dedicated section "Activities" within the open data portal data.gov.ro, among other activities. Given this, while the commitment as elaborated will clearly help enable SGG to measure the (re)utilization of open data, the commitment as written has a minor potential impact.
This commitment addresses an important policy area, but could benefit from a higher degree of specificity when carried forward:
- SGG could detail what is meant by the tailoring of the OECD indicators to the Romanian context and how this will take place—e.g., though a survey, discussions with users, and data generators etc.
- SGG could provide a detailed plan on how the monitoring of the open data reuse and impact will take place—e.g., define a monitoring methodology, a calendar, form testing hypotheses, and detail how the results of the monitoring will be published and how the recommendations and conclusions will be implemented.
- Finally, in future versions of OGP commitments, SGG should specifically detail what is meant by ‘activities,’ so that the IRM reports can measure the true potential of what is planned.
 Interview with Larisa Panait, General Secretariat of the Government - Open Government Partnership point of contact (OGP contact), 15 April 2019.
 Interview with Larisa Panait, OGP contact, 15 April 2019.
IRM End of Term Status Summary
17. Measure the impact of the re-use of open data
According to an SGG representative, the OECD project that would have provided the indicators and methodology to measure the (re)use of open data did not progress according to plan.  Consequently, the first two milestones of this commitment, which were tied to this project, were not started during the action plan period. Instead SGG applied for public funding to develop the methodology and the indicators in-house. 
SGG carried out a variety of promotional activities on the re-use of open data, including showcasing good practices on the open data portal  on social media,  organizing webinars and work groups,  attending app launch events,  organizing an Open Data Day,  and creating a dedicated corner on Romania’s open data portal data.gov.ro for developers wishing to work with open data.  Moreover, according to the OGP repository, SGG adopted the open license for the open data CC-BY 4.0 to stimulate the (re)use of open data.  There are currently 10 applications that utilize datasets opened on the data.gov.ro platform, compared to three before the action plan period.