Opening Data Collected from the National Health System (RO0023)
Action Plan: Romania, Second Action Plan, 2014-2016
Action Plan Cycle: 2014
Lead Institution: Ministry of Health
Support Institution(s): CSOs: Open Data Coalition
Policy AreasAccess to Information, Anti-Corruption, Health, Open Data, Public Procurement, Public Service Delivery
Early Results: Major
Relevant to OGP Values: Access to Information , Technology
The Ministry of Health has shown great interest in the OGP related activities, has taken part in discussions with the civil society and has published open datasets produced at the central level or by subordinate units.
Opening the data collected from the public health system represents an important step in enhancing transparency and fight against corruption, and last but not least, in improving public trust in the system.
Starting September 2014, the ministry will develop the models for two information platforms: one monitoring the procurements of public health units and one monitoring the activity of the Ethical Councils in public health units.
All stakeholders will be actively consulted during implementation and the monitoring results will be published in open formats.
IRM End of Term Status Summary
Commitments 5 and 6: Open Data and Monitoring Anti-Corruption in the Health System
5. Commitment Text:
Opening data collected from the National Health System:
A. Development of the legislative framework. The Ministry will draft the legislative proposals for regulating the monitoring the procurement in public health units and for monitoring the activity of Ethical Councils in public health units.
Approval of the legislative proposals:
B. Development of the Platforms. The ministry will develop the models for two information platforms: one monitoring the procurement of public health units and one monitoring the activity of the Ethical Councils in public health units. The ministry will pilot the platforms, in collaboration with civil society, in order to improve their functionalities based on the results.
Completion of the two platforms - Start implementation in all public health units:
C. Initiate the opening of collected data
Responsible institution: Ministry of Health
Supporting institution: Open Data Coalition
Start date: 2014 End date: 2016
6. Commitment text:
Opening data collected from the monitoring of preventive measures as part of the National Anti-Corruption Strategy 2012-2015
1. The ministry will develop the model information platform for monitoring the results of anti-corruption preventive measures
2. The ministry will pilot the platforms, in collaboration with civil society, in order to improve their functionalities based on the results
3. Completion of the platform
4. Initiate platform large-scale use
5. Opening collected data
Responsible institution: Ministry of Health
Supporting institutions: None specified
Start date: 2014 End date: 2016
The health system in Romania faces significant financial, administrative, and corruption challenges.[Note 16: An EU report published in January 2016 (SWD (2016) 16 final) stated: “Corruption in the Romanian health system has been recognized as a significant problem many Romanian citizens are facing. The main problems are related to public procurement, to fraud and passive corruption in the context of obtaining medical certificates, and to the practice of informal payments.”] These factors have decreased public trust in a health sector characterized by a culture of secrecy. These commitments aim to improve transparency and access to information regarding Romania’s 350 hospitals and the health system administration. This could also be a positive step for increasing public trust in the system.
Commitment 5 aims to open data collected by the national health system. To reach this goal, the government will take several preparatory steps to monitor the health system and gather data. Ethical councils will be established in hospitals and clinics across the country, and the Ministry of Health will develop a legislative framework to create two web platforms. One platform will monitor health-sector procurement activities, and the other will publish reports on the ethical councils’ performance in health units.
Commitment 6 proposes to develop anti-corruption measures, as well as to monitor and publish information on their implementation. To achieve this, the Ministry of Health will create a separate online health platform for monitoring the results of corruption prevention measures. The ministry will collect patient feedback to identify problems such as quality of care and corruption in the health system.
By the midterm evaluation, the Ministry of Health had adopted the legislative framework needed to regulate and monitor health procurement and the ethical councils’ activities. The ministry had substantially completed the two web platforms; however, some lingering challenges remained. The health procurement platform replicates some features of a pre-existing system, the Public Procurement Electronic System (SEAP), run by the Digital Agenda Agency of Romania. To avoid redundancy, the SEAP portal will house the health procurement platform. This somewhat deviates from the original intent of the commitment. Though the procurement information will still be publicly released, the form it will take diminishes the relevance and value to stakeholders. The original plans for a health platform included features and tools for the public to actively monitor the procurement process. The SEAP portal, however, will mainly be a repository of reports about the health sector.
Likewise, the platform to monitor the activities of ethical councils has been developed, but several problems remain. When collecting information on ethical practices, the councils often failed to scrub identifying patient data or other legally private information. To prevent this information from being released, the Ministry of Health decided to analyze raw data from the ethical councils and use the platform to present aggregated reports about trends in findings. The ministry had yet to open existing data. For more information, please see the 2014–2015 IRM midterm report.
End of term: Substantial
This commitment remains substantially complete. Parliament passed the required legislation, the two platforms were developed, and the public health units started supplying information to the platforms. The Ministry of Health is publishing data online regarding the ethical councils, health procurement, and public health budgets. However, there have been delays in opening access to the new platforms. At the end of term, the two portals are not yet fully functional, but development is ongoing.[Note 17: See https://portalvpn.ms.ro/remote/login?lang=en. ] After the conclusion of the implementation period, the ministry uploaded procurement data on 8 September 2016 and information about the ethical councils’ activities on 29 October 2016.
On the pilot procurement platform, the Ministry of Health’s website included approximately 2,800 data visualizations by the end of 2016.[Note 18: Public procurement data taken from the public procurement data portal. The visualization is available online at http://transparenta.ms.ro/#.] In addition to the public procurement data, public health unit budgets are also available via ministries’ webpages.[Note 19: The budget data for hospitals are available at http://monitorizarecheltuieli.ms.ro/centralizator. ] The platform for the ethical councils is available online, but only health administrators responsible for reporting raw data from the public health units have access to the platform.[Note 20: This platform contains data on the ethical councils. It is available online at http://infrastructura-sanatate.ms.ro/. However, the platform is password protected, and only health administrators can access this information.] The Ministry of Health can then aggregate and analyze this data and publish results on the data.gov.ro platform.
Despite the advances in the implementation of this commitment, CSO representatives have criticized the way the ministry chose to carry out implementation. Some CSOs veiwed these projects, which were already under development prior to the action plan, as a way to to publicize their progress without actually increasing cooperation with civil society counterparts. Civil society had very little input or responsibility in overseeing or monitoring the implementation.
At the midterm, the Ministry of Health had developed the platform to monitor anti-corruption measures and had taken preparatory steps to test the platform in five major hospitals. The platform was only accessible to public health workers, and the Ministry of Health assisted all 350 public health units in Romania to develop their own input for the platform. The ministry had not yet developed corruption prevention measures. For more information, please see the 2014–2015 IRM midterm report.
End of term: Limited
Since the midterm report, stakeholders have made some progress on creating a patient feedback mechanism and developing a pilot platform, but the ministry has not started the implementation of anti-corruption measures. This commitment was intended to complement the Ministry of Justice’s new National Anti-Corruption Strategy. However, creation of the anti-corruption strategy framework was delayed, and the ministry adopted it after the conclusion of the commitment period. Through this commitment, the Romanian government included goals to reduce health system corruption in a national policy for the first time. In the IRM researcher’s view, the delay in establishing the national framework may explain the lack of progress in implementing this commitment.
Hospitals have adopted the pilot anti-corruption platform, but it is not intended for public use or access. Once patient identification information has been scrubbed, health institutions are able to collect patient feedback and data through the platform, analyze it for trends, and then publish statistics and findings on Romania’s open data portal.
Implementation of the patient feedback system suffered from administrative shortcomings and improperly trained staff. In addition, the results of the patient feedback survey were compromised by inappropriate administration methods, such as failing to ensure anonymity and not controlling for selection bias. Specifically, the questionnaires:
• Lacked a mechanism for ensuring patients could complete only one survey
• Were too lengthy and complex for patients to understand
• Were not anonymous (completed in the presence of medical personnel)
• Were not representative samples due to self-selecting methodology
After the commitment period, the Ministry of Health proposed a new survey methodology, published on 21 November 2016 on its website for public consultation. These updates address some of the issues listed above but still do not include a mechanism to control for bias.
As of late 2016, only summaries of the patient questionnaires are publicly available. The summary data has been published on the open data portal, but the raw data containing individual patient responses is not available. It should also be noted that summaries were published for September and October 2016, with plans to publish the results from the patient feedback mechanism every three months. Discussions with CSO representatives have indicated that there was an open disagreement within the ministry regarding how to best implement this commitment, which may explain the limited completion and substantive issues with patient feedback methods.
Did it open government?
Access to information: Major
The achievements of this commitment should be considered a major step forward in opening government with respect to access to information. The platform monitoring public procurement in the national health system is a somewhat minor advance, considering that health procurement data was already accessible in a similar form on the SEAP portal. However, the platform monitoring the ethical councils represents a major change in government practice. It has lead to opening in one of the least transparent public systems in Romania. The data offered through this platform allows citizens to access previously unavailable information, such as the number and the type of complaints that the ethical councils have resolved.[Note 21: The data on the activity of the ethical councils is available at http://data.gov.ro/dataset/activitatea-consiliilor-etice-din-unitatile-sanitare-publice.] Accessing information on complaints raised in specific hospitals is particularly important because it allows the public to make more informed decisions when choosing hospitals for specific types of treatment, and enables citizens to assess different aspects of health unit performance.
Although substantially completed, there are still aspects related to the commitment that could be improved to open government in a transformative way:
1. As indicated by the self-assessment report, the public and other interested stakeholders should be more involved in assessing the two platforms. Since the data was published on the open data portal, there was only one public comment on the dataset on procurement, indicating a low number of users for the two datasets.
2. The quality of datasets published on the open data portal could be improved. The current versions do not have metadata, nor do they include clear explanations and descriptions for the variables in the datasets.[Note 22: http://data.gov.ro/dataset/activitatea-consiliilor-etice-din-unitatile-sanitare-publice.]
3. A monitoring system is needed to ensure that hospitals will continue collecting and updating information on the platforms regularly.
Access to information: Marginal
This commitment marginally opened government. Health institutions have published data from the patient feedback mechanism after several delays, and significant methodological problems had affected the data. The Ministry of Health has modified both the methodology and the patient questionnaire to correct issues identified during the piloting phase, but the government has done little to consult or involve civil society in the activities of this commitment. However, providing for these shortcomings in implementation, this commitment resulted in an overall change in government practice within the health system. Prior to the commitment period, it was highly uncommon, even in private clinics and hospitals, for patients to request additional explanations on procedures or treatment. In this context, implementing a questionnaire for patients to provide feedback and evaluate services they received represents an improvement and offers access to previously unavailable data in a structured manner. Overall, this commitment could offer public access to relevant and useful information, but so far the mechanism is still underdeveloped.
Since commitment 5 was completed soon after the end of the second national action plan, it is no longer included in the third national action plan.
Commitment 6 remains incomplete, and the government did not include it in the third national action plan. However, the legislation required for completing this commitment has been adopted, and the Ministry of Health is working on the methodology for collecting patient feedback, suggesting that this project will continue. The IRM researcher recommends consulting with civil society actors to improve the patient feedback monitoring and reporting methodologies. A monitoring system should also be put in place to ensure that health institutions are regularly updating and publishing collected feedback in a transparent way.
Standardize Public Consultation Practices
RO0048, 2018, E-Government
Open Local Government
RO0049, 2018, Capacity Building
RO0050, 2018, Capacity Building
RO0051, 2018, Capacity Building
Register of Civil Society Proposals
RO0052, 2018, E-Government
Access to Information – Local
RO0053, 2018, Capacity Building
Online Business Sector Information
RO0054, 2018, Capacity Building
Digital Consular Services
RO0055, 2018, Capacity Building
Transparency in the Funding of Political Parties
RO0056, 2018, Access to Information
National Investment Fund Transparency
RO0057, 2018, Access to Information
Civil Servant Training
RO0058, 2018, Capacity Building
Raise Awareness About Corruption
RO0059, 2018, Capacity Building
Transparency of Seized Assets
RO0060, 2018, Access to Information
Access to Social Services
RO0061, 2018, E-Government
Open Access to Research
RO0062, 2018, Access to Information
RO0063, 2018, Access to Information
Evaluate Open Data
RO0064, 2018, Access to Information
RO0065, 2018, Access to Information
Improving the Legal Framework and Practices Regarding Access to Public Interest Information
RO0030, 2016, Access to Information
Centralized Publishing of Public Interest Information on the Single Gateway Transparenta.Gov.Ro
RO0031, 2016, Capacity Building
Promoting Open Parliament Principles
RO0032, 2016, Capacity Building
Improved Management of the Applications Submitted for Granting Citizenship
RO0033, 2016, Capacity Building
Standardization of Transparency Practices in the Decision-Making Procedures
RO0034, 2016, Capacity Building
Centralised Publication of Legislative Projects on the Single Gateway Consultare.Gov.Ro
RO0035, 2016, Capacity Building
RO0036, 2016, Capacity Building
Improve Youth Consultation and Public Participation
RO0037, 2016, Capacity Building
Subnational Open Government
RO0038, 2016, Capacity Building
Promoting Transparency in the Decision-Making Process By Setting Up a Transparency Register (RUTI)
RO0039, 2016, Anti-Corruption
Access to Performance Indicators Monitored in the Implementation of the National Anticorruption Strategy (SNA)
RO0040, 2016, Access to Information
Improve Transparency in the Management of Seized Assets
RO0041, 2016, Access to Information
Annual Mandatory Training of Civil Servants on Integrity Matters
RO0042, 2016, Anti-Corruption
Improving Access to Cultural Heritage
RO0043, 2016, Capacity Building
Open Data and Transparency in Education
RO0044, 2016, Access to Information
Virtual School Library and Open Educational Resources
RO0045, 2016, Capacity Building
RO0046, 2016, Anti-Corruption
Increasing the Quality and Quantity of Published Open Data
RO0047, 2016, Access to Information
Publishing the Public Interest Information on a Single Government Portal: Transparenta.Gov.Ro
RO0019, 2014, Access to Information
Making an Inventory of the Datasets Produced by the Ministries and Subordinate Agencies
RO0020, 2014, Access to Information
Ensuring the Free Online Access to National Legislation
RO0021, 2014, E-Government
Amending Law 109/2007 on the Re-Use of Public Sector Information
RO0022, 2014, Access to Information
Opening Data Collected from the National Health System
RO0023, 2014, Access to Information
Opening Data Collected from the Monitoring of Preventive Measures as Part of the National Anticorruption Strategy 2012-2015
RO0024, 2014, Access to Information
RO0025, 2014, Anti-Corruption
Opening up Data Resulted from Publicly-Funded Research Projects
RO0026, 2014, E-Government
Increasing the Quality and Quantity of Published Open Data
RO0027, 2014, Access to Information
Human Resource Training in the Field of Open Data
RO0028, 2014, Access to Information
Disseminating Information on the OGP Principles and Promoting the Open Data Concept in an Accessible Manner
Designating a Person Responsible for Publishing Open Data in Each Public Institution
RO0001, 2012, Access to Information
Identifying Regulatory Needs, Logistical and Technical Solutions
RO0002, 2012, Access to Information
Making an Inventory of Available (High-Value) Data-Sets
RO0003, 2012, Access to Information
Priority Publishing on the Web Pages of Public Institutions of Specific Data-Sets
RO0004, 2012, Access to Information
Initiating Pilot-Projects, in Partnerships
RO0005, 2012, Access to Information
Organizing Public Debates on the Utility of Open Data, in Partnerships
RO0006, 2012, Access to Information
Uniform, Machine-Readable Publishing Format for Open Data
RO0007, 2012, Access to Information
Procedures for Publication of Data-Sets Based on Civil Society Recommendations
RO0008, 2012, Access to Information
Procedures for Citizen Complaints Pertaining to Open Data
RO0009, 2012, Access to Information
Consultation Mechanism Between Suppliers and Beneficiaries of Open Data
RO0010, 2012, Access to Information
Creating a Rating System for the Assessment of High-Value Data-Sets
RO0011, 2012, Access to Information
Routinely Publishing Specific Data-Sets on Web Pages of Public Institutions
RO0012, 2012, Access to Information
Integrating Open Data from Public Institutions in a Single National Platform
RO0013, 2012, Access to Information
Inventories of Data, in Order to Facilitate Public Access
RO0014, 2012, Access to Information
Institute a Monitoring Mechanism of Compliance for Open Data
RO0015, 2012, Access to Information
Stimulating the Market for Innovative Use of Open Data
RO0016, 2012, Access to Information
Routinely Publishing Data-Sets on the National Platform, 25% High-Value
RO0017, 2012, Access to Information
The Public Procurement Electronic System (SEAP). the Electronic Allocation System for Transports (SAET)/B.1 C) Expanding the On-Line Submission of Fiscal Forms. Ensuring the Free On-Line Access to National Legislation. Developing Electronic Tools to Manage Subpoenas and Facilitate Access Toinformation Regarding Legal Proceedings. Developing Electronic Tools to Manage the Procedures Related to Obtaining the Romanian Citizenship. Developing Electronic Tools to Manage the Procedures Related to the Creation of Non-Profit Legal Persons. the Integrated System for Electronic Access to Justice (SIIAEJ)
RO0018, 2012, Access to Justice