Concept Note on Publishing Data (LV0018)

Data and information held by public administration institutions is a resource that offers unexplored economic and social potential. The value of data increases when making them available for the use in creating new commercial products and services, in research, the analysis of public processes. This will have a positive impact, in terms of GDP growth and tax yield, not only on national economies but also directly on the budget revenues, a part of which can be used for sustaining and funding this direction in the activities of public administration. To achieve that public administration related data are technically and legally accessible, are published in a proactive format, and their use is facilitated in the creation of new solutions.
In order to support and facilitate making of public data available for re-use, the following measures will be supported:
● technical solutions for preparing and publishing data in a publicly accessible, transparent, harmonised and automatically processable form, where possible, while ensuring the protection of personal data;
● establishment of the ICT infrastructure required for sharing to make the current data on national data registers available for reuse;
● creation of a unified, centralised data catalogue, where data structures and interfaces are described following a harmonised model and available in a centralised catalogue;
● working out a solution for centralised data distribution, including decentralised solutions, where expedient.
● support for activities required to ensure the functionality of data sources with the aim of re-use and converting into a reusable format of the data held in those sources, including anonymisation measures;
● alongside the solutions, the necessary policies should be devised and legal framework put in place:
○ for implementation into national law of Directive 2003/98/EC of the of the European Parliament and of the Council on the re-use of public sector information (PSI Directive), incl. charging and licencing provisions, e.g. in the area of geospatial information;
○ for changing the model of financing public administration institutions, in order to promote the re-use and sharing of data held by the state, thereby reducing, as much as possible, direct dependency of the core activities of an institution on revenues gained from offering information for re-use;
● measures promoting the use of open data in the creation of new and innovative products (application software, solutions contests, educational seminars and workshops).
Infrastructure measures to be supported:
● shared use solutions for the processing, publishing and previewing of the open data;
● creating open data applications (incl. dataset aggregation and integration).

1. Concept paper on publishing the public sector data in a machine readable format (open data)

Commitment Text:

Data and information held by public administration institutions is a resource that offers unexplored economic and social potential. The value of data increases when making them available for the use in creating new commercial products and services, in research, the analysis of public processes. This will have a positive impact, in terms of GDP growth and tax yield, not only on national economies but also directly on the budget revenues, a part of which can be used for sustaining and funding this direction in the activities of public administration. To achieve that public administration related data are technically and legally accessible, are published in a proactive format, and their use is facilitated in the creation of new solutions.

In order to support and facilitate making of public data available for re-use, the following measures will be supported:

• technical solutions for preparing and publishing data in a publicly accessible, transparent, harmonised and automatically processable form, where possible, while ensuring the protection of personal data;
• establishment of the ICT infrastructure required for sharing to make the current data on national data registers available for reuse;
• creation of a unified, centralised data catalogue, where data structures and interfaces are described following a harmonised model and available in a centralised catalogue;
• working out a solution for centralised data distribution, including decentralised solutions, where expedient.
• support for activities required to ensure the functionality of data sources with the aim of re-use and converting into a reusable format of the data held in those sources, including anonymisation measures;
• alongside the (technical) solutions, the necessary policies should be devised and legal framework put in place:
  • for implementation into national law of Directive 2003/98/EC of the European Parliament and of the Council on the re-use of public sector information (PSI Directive), incl. charging and licencing provisions, e.g. in the area of geospatial information;
  • for changing the model of financing public administration institutions, in order to promote the re-use and sharing of data held by the state, thereby reducing, as much as possible, direct dependency of the core activities of an institution on revenues gained from offering information for re-use;
• measures promoting the use of open data in the creation of new and innovative products (application software, solutions contests, educational seminars and workshops).

Infrastructure measures to be supported:

• shared use solutions for the processing, publishing and previewing of the open data;
• creating open data applications (incl. dataset aggregation and integration).

Responsible institution: Ministry of Environmental Protection and Regional Development (VARAM)

Supporting institution(s): N/A

Start date: 2014  End date: 2020

Context and Objectives

Publicly gathered data allow opportunities to create new commercial products and conduct policy research when open for third party use. At the end of 2013, open data were accessible in Latvia mainly upon request. There were no technical mechanisms for releasing data for reuse, nor regulations to prohibit institutions from selling the data at commercial prices. Global Open Data survey results suggest that available data are now offered mostly free of charge. ^{Global Open Data index, http://global.census.okfn.org/place/lv.}

https://arxiv.org/ftp/arxiv/papers/1406/1406.5052.pdf  Scarce data also made it hard for policy analysts and journalists to evaluate various policy areas. The Ministry of Environmental Protection and Regional Development, which is responsible for this commitment, prepared a policy planning document highlighting many of these data shortcomings. ^{Database of policy documents, http://tap.mk.gov.lv/lv/mk/tap/?dateFrom=2013-01-01&dateTo=2013-09-30&text=VSS-548&org=0&area=0&type=0.}  

The objective of the commitment is to ensure that public administration data are technically and legally accessible, published proactively, and are reusable. It is to be achieved by way of a three-pronged approach: developing technical mechanisms, such as an online data portal; setting legal requirements and standards; and implementing promotional activities following launch of the system. The commitment entails a broad long-term program, which also includes a large investment project (the development of the open data portal). The commitment is relevant to access to information and innovation and technology because it focuses on making public data more open and usable through improved technology infrastructure.

There is a medium level of specificity in the commitment. As written, the technical solutions and promotional activities’ milestones lack details, such as the scale and expected audience of the activities and the features of the proposed ICT infrastructure. Both milestones require expert input during later stages to determine needs and to adjust implementation accordingly.

If implemented as described, the commitment could have a transformative effect. Free, accessible data in a centralized location would be a significant departure from the status quo. Currently, data are available only upon request and government institutions are allowed to charge for access to data. The data portal and policy measures that facilitate reuse would simplify interactions between the government and those who provide and use data. On the demand side, there is a growing multi-stakeholder community of investigative journalists, academics, and civil society organisations — not to mention government institutions and local governments themselves — that would benefit from open data for better policy planning and assessments. Open data enthusiasts have developed a small and steadily growing community, and have created an open data portal where interested persons can share their data needs, technical obstacles in obtaining data, and findings on new data sets. ^{Open data portal, http://data.opendata.lv/.}  If fully implemented, the commitment would improve the quantity and quality of data available for both these communities and the public at large. However, success will depend on the specific data sets chosen for publication and the commitment’s ability to expand the community that will benefit from use of the data for commercial purposes, policy research, and journalism.

Completion

Interviews with officers at the ministry ^{Interviews were conducted with Toms Ceļmillers, Jānis Glazkovs, and Inese Gaile, Ministry of Environmental Protection and Regional Development, 15 August 2016.}  show that most milestones are in progress. The Ministry of Environmental Protection and Regional Development has developed the legal basis for the investment project to develop an open data portal (milestone 1). This will be funded by the European Fund for Regional Development. The project is among the funding priorities approved by the Cabinet of Ministers. While financing is pending, the ministry is conducting a feasibility study for the project. It was estimated, at the time of writing, that the study would be available and technical specification for procurement developed by the end of 2016. The ministry also conducted a survey of end users on the expected functionalities of the portal. According to the ministry, ^{This is according to an interview with Toms Ceļmillers, Ministry of Environmental Protection and Regional Development, 15 August 2016.}  the first trial version of the portal could be ready by mid-2017.

New policy developments (milestone 2) include amendments to the Freedom of Information Law passed on 3 September 2015. ^{Official database of Law, http://likumi.lv/ta/id/276655-grozijumi-informacijas-atklatibas-likuma.} The amendments implemented into national law Directive 2003/98/EC of the European Parliament and of the Council concerned the re-use of public sector information (PSI Directive). The amendments provide definitions for concepts such as ‘reuse’, ‘open data’, and ‘meta data’. The law also establishes limits on data produced by the government to avoid an unnecessary burden on public resources. For example, an institution is not required to collect and disclose data that are not necessary for fulfilling its own public functions. It also specifies pricing guidelines if data are not free.

The annotation to the amendments reveals that NGOs did not participate and were not consulted during the development of the law. ^Ibid.  The ministry ^{Interview with Toms Ceļmillers, Ministry of Environmental Protection and Regional Development, 15 August 2016.}  stated that this was a “highly technical” project, which would not have been of interest to citizens. The annotation states that the draft was sent to professional associations working on information technology for their opinions. These associations — LIKTA (Latvian Association of Technologies for information and Communication) ^{Home page of the association, LIKTA, https://www.likta.lv/EN/Pages/home.aspx.}  and LATA (Latvian Association for Open Technologies) — did not object. Nor did other NGO experts object to the new open data amendments or how the ministry is proceeding with implementation of the commitment. ^{Data providers and suppliers made these comments to the IRM researcher at an informal dinner organized by the Data School on issues of open data policy, 25 August 2016.}  

The ministry has not begun promotional activities (milestone 3), which are planned for later stages. However, ministry officials are active participants at data users’ forums, such as the NGO Data School, its Facebook group, ^{https://www.facebook.com/groups/560163084143465/. } and an informal Google discussion group on open data.

It is difficult to judge whether a commitment is on time according to the action plan, which does not give a clear time schedule. From interviews with the ministry, it appears that all activities are on time and there are no delays or obstacles preventing further implementation. ^{Interview with Toms Ceļmillers, Jānis Glazkovs, and Inese Gaile, Ministry of Environmental Protection and Regional Development, 15 August 2016.}  Other stakeholders^{Comments were made by data providers and suppliers at an informal dinner organized by the Data School on issues of open data policy, 25 August 2016.}  also had no concerns about the delay of expected outcomes. As a result, the IRM researcher considers the commitment to be on time.  

Early Results (if any)

The implementation of the commitment is in its early stages. The laws adopted are an important basis for developing the open data portal and publishing data in the future. However, it is too early to judge results. The NGO “Data School” organized a discussion on data needs and availability for data providers and users. ^{For a list of participants at the meeting, see https://docs.google.com/document/d/18n-INtznUVJvsuFSV5lXbVzLoIsrREyi4EBNlUqOkyU/edit.}  The discussion revealed that some ministries, institutions, and local governments are already opening their data outside the OGP framework. For example, the Riga City Council started publishing data on available territories for gardening; city-owned spaces for rent; and statistics on taxpayers, marriages, and citizen communications to the Council. These data are renewed monthly. The Council also offers services and tools for downloading machine-readable data from its server. ^{Home page of the Riga City Council, https://opendata.riga.lv/.}  It is now working on offering data in more engaging and attractive ways to foster public interest and usage.

The State Revenue Service, too, consulted journalists, policy researchers, and data technology specialists on the functionalities of its web page. According to data experts, the agency has made it easier to gather machine-readable data from the income declarations of government officials. ^{Comments by data providers and suppliers were made to the IRM researcher at an informal dinner organized by the Data School on issues of open data policy, 25 August 2016.}  

Next Steps

IT professionals, researchers, journalists, and policy analysts all want data in formats that are comparable. Participants at the data discussion mentioned above made several observations about the current state of data in Latvia:

• There is a lack of awareness in the community about this commitment and the government’s plans to open data sets.
• There is a poor understanding of the differences between the “technical” and “human” perceptions of open data. Formats for researchers and journalists are not the same as they are for IT specialists, though there is a demonstrated interest in learning more about the needs and opportunities of both sides.
• IT companies experiment and offer platforms for publishing open data, such as SIA 'ZZDats,' the company that developed and donated its software to the Riga City Council for its data initiative.
• IT professional associations are strong players in lobbying for e-governance solutions, though open data end users are less organized and less knowledgeable.
• There is a lack of expertise on issues such as data privacy to protect sensitive data and prevent inferring personality traits or public transportation habits, for example.

Given these observations, the IRM researcher recommends moving forward on all three milestones of the commitment and introducing promotional activities before the launch of the portal to develop a more informed community of open data users. In addition, open data must be free of charge. ^{Open Data barometer, http://opendatabarometer.org/open-data/.}  The IRM researcher recommends preventing institutions from setting prices on reusable data sets in cases where a price is set. For the second milestone, the government could explore the possibility of not selling data to reimburse production costs. The commitment calls for “reducing, as much as possible, direct dependency of the core activities of an institution on revenues gained from offering information for re-use.” However, this principle is not implemented in law and will require continued engagement from stakeholders to move forward.

Stakeholders recommend that the next action plan focus on opening data sets of particular interest to the public. These include budget data, lists of lobbyists, and lists of people who have visited ministries or parliamentary committees (through data on entry passes). The government could introduce a consultation system with regard to data sets of interest to journalists and researchers to ascertain the demand for data and explore possibilities for data releases. The functionalities for consulting data users should be considered whenever improving existing data portals or developing new ones.

✪ Commitment 1. Open Data

Commitment Text:

Promote access to the public administration data in form of open data

Data and information held by public administration institutions is a resource that offers unexplored economic and social potential. The value of data increases when making them available for the use in creating new commercial products and services, in science and research, the analysis of public processes that will have a positive impact, in terms of GDP growth and tax yield, not only on national economy but also directly on the budget revenues, a part of which can be used for sustaining and funding of further development, creating the opportunity to increasingly turn the potential of national data in economic potential.

The Information Society Development Guidelines for 2014-2020 (hereinafter – the Guidelines) foresee a range of measures to promote proactive publishing of the public administration data and facilitate their use for creating new solutions.

With a view to support and facilitate the transfer of public data for re-use, the Guidelines support:

● Development of technical solutions for preparing and publishing data in a publicly accessible, transparent, harmonised and automatically processable form, where possible, while ensuring the protection of personal data;

● Establishment of the common ICT infrastructure required for making the data in national registers available for re-use;

● Creation of a unified, centralised data catalogue, where data structures and interfaces have to described following a harmonised model and available in a centralised catalogue;

● Solution for centralised data distribution, envisaging also decentralised solutions, where expedient.

● Activities required to ensure the functionality of data sources with the aim to re-use and convert into a reusable format the data held in those sources, including anonymization measures;

● Alongside the solutions, the necessary policies should be developed and legal framework implemented:

○ for implementation into national law of Directive 2003/98/EC of the European Parliament and of the Council on the re-use of public sector information, incl. implementation of charging and licencing provisions, e.g. in the area of geospatial information the requirements are set for licensing;

○ for changing the model of financing public administration institutions, in order to promote the re-use and shared use of data held by the state, thereby reducing, as much as possible, direct dependency of the core activities of an institution on revenues gained from transmitting the information for re-use;

● Measures which encourage the use of open data for new and innovative product development (applications, competitions for solutions, educational seminars and workshops). …

Infrastructure measures to be supported:

● Shared solutions for the processing, publishing and previewing of the open data;

● Creating open data applications (incl. dataset aggregation and integration).

Responsible Institution: Ministry of Environmental Protection and Regional Development (VARAM)

Supporting Institution: N/A

Start Date: 2014.................... ...................... End Date: 2020

Editorial Note: The commitment text above is drawn from the updated version of the action plan, published in October 2016 and available at http://bit.ly/2EK34dH. The original version of the action plan is available at http://bit.ly/2ptZ0sq. To see the changes between the two versions, please visit http://bit.ly/2FPvK4r.

Commitment Aim

The objective of the commitment is to ensure that public administration data are technically and legally accessible, published proactively, and reusable. When this commitment was adopted, there were no (1) technical guidelines for government-institution release of data for reuse, (2) regulations that set technical standards for publishing and licencing them, or (3) prohibitions against institutions selling data at commercial prices. The Ministry of Environmental Protection and Regional Development (VARAM), the agency responsible for this commitment, prepared a policy planning document that highlighted many of these data shortcomings. ^{“The Draft Guidelines ‘Information Society Development Guidelines for 2014-2020,’” Legislative Proposals, Draft Legislation of the Cabinet of Ministers of the Republic of Latvia, last updated 5 March 2018, http://tap.mk.gov.lv/lv/mk/tap/?dateFrom=2013-01-01&dateTo=2013-09-30&text=VSS-548&org=0&area=0&type=0.}

To address these issues, the commitment includes three main lines of work: (1) developing technical solutions, such as an online data portal; (2) setting legal requirements and standards; and (3) implementing promotional activities after the launch of the system. The commitment entails a broad long-term programme that includes the large investment project of developing the open data portal.

The end date for the commitment’s implementation is 2020. Nonetheless, according to ministerial plans for investment projects, ^{“Specific Objective of Support 2.2.1: Ensuring an Increase in the Re-use of Public Data and Effective Interaction between Public Administration and the Private Sector,” EU Funding 2014-2020 per Year, Ministry of Environmental Protection and Regional Development, http://www.varam.gov.lv/lat/fondi/kohez/2014_2020/?doc=18634.} the planned activities for the period of the second action plan (2015–2017) included the development of a beta version of the open data portal and legal regulations, and data support for institutions and portal users. The revised version of the action plan (published in October 2016) likewise specifies that the creation of legal amendments and the development of the open data portal were milestones expected to be completed by late 2017. This specification about the expected timeline was the only substantive change between the original and updated versions of the action plan.

Status

Midterm: Limited

Most activities were in progress at the time of the midterm report. VARAM developed the legal basis for the open data portal investment project, which was among the funding priorities approved by the Cabinet of Ministers. The government was also developing a feasibility study for the project and technical specifications for procurement. In addition, VARAM conducted a survey among end users on expected functionalities of the portal. At the time, an initial beta version of the portal was expected to be ready by mid-2017 (the government launched the portal during the second year of implementation, as described in the following section).

The parliament also passed amendments to the Freedom of Information Law that provide definitions for concepts such as “reuse,” “open data,” and “metadata.” The amendments established limits on the amount of data produced by the government to avoid an unnecessary burden on public resources. For example, an institution is not required to collect and disclose data that are not necessary for the fulfillment of its own public functions. Lastly, the amendments established pricing guidelines for data that are not made available for free.

The government did not carry out any promotional activities, as those related to later stages of the project’s implementation. For more information, see the 2015–2016 IRM midterm report. ^{Open Government Partnership, Independent Reporting Mechanism (IRM): Latvia Progress Report 2015–2016, https://www.opengovpartnership.org/wp-content/uploads/2001/01/Latvia_Progress-Report_2015-2017_for-public-comment_0.pdf.}

End of term: Substantial

The second year of implementation has brought tangible results. The open data portal is now fully operational ^{“Welcome to the Open Data Portal of Latvia!”, https://data.gov.lv/lv} and open for government institutions to release their publicly available data in a reusable format. In accordance with findings of the working group at the ministry, the open licences of Creative Commons will be used for the portal. VARAM translated the licences in late 2016 and is now negotiating the terms with Creative Commons. The ministry has already applied the first licence to the beta version of the portal. ^{“Target Declaration,” Creative Commons, https://creativecommons.org/publicdomain/zero/1.0/legalcode.lv.}

VARAM also developed five open data guidelines that are currently published on its web page; ^{“Open Data,” E-administration, Ministry of Environmental Protection and Regional Development, http://www.varam.gov.lv/lat/darbibas_veidi/e_parv/atvertie_dati/?doc=20449.} these will be available on the open data portal. The specifications include guidelines for data publishers, standards for metadata descriptions, guidelines for the development of metadata, and standards and guidelines for descriptions of machine-readable (CSV format) datasets. The guidelines for publishers suggest publishing data in an open format by default and offer recommendations for prioritising datasets for publication on the open data portal.

The government plans to publish existing datasets as well as those that are created as part of a project financed by the European Regional Development Fund. The VARAM website lists the upcoming datasets to be financed by this project. ^{“Specific Objective of Support 2.2.1,” http://www.varam.gov.lv/lat/fondi/kohez/2014_2020/?doc=18634.} VARAM also invites other institutions to publish data on the portal.

Several VARAM activities also involved the public, though not necessarily within the framework of the promotional activities envisioned under the commitment. For example, VARAM experts participated in a conference and discussion on the opening of geospatial data, ^{“The Conference ‘Open Technologies for Growth,’” Latvian Association of Open Technologies, http://lata.org.lv/konference2017_programma/; and “Workshop—Discussion on the Opening of Geospatial Data in Latvia,” Latvian Association of Open Technologies, http://lata.org.lv/seminars-diskusija-par-latvijas-geotelpisko-datu-atversana/.} and in the NGO Data School’s discussions on the use of open data in various areas, such as research and journalism. ^{“Datu Skolas Sezonas Atklasana,” Facebook, 13 September 2016, https://www.facebook.com/events/1784218671847711/.} VARAM also surveyed data users on functionalities of the open data portal, the results of which were considered while developing the beta version. ^{“Anketa par Atverto Datu Portala Lietojamibu,” Facebook, 29 August 2016, https://www.facebook.com/datuskola/posts/323954001286728.}

The implementation of the commitment is considered to be on time, since all of the activities planned for 2015 to 2017 were completed (e.g. launch of the open data portal, legal amendments, and data support according to the revised version of the action plan). However, the commitment as written is only substantially complete because several milestones were not fulfilled. Specifically, the government did not change the financing model of institutions gathering data or complete all of the planned public-awareness-raising activities. These pending activities are expected to be completed by the end of 2020.

Did It Open Government?

Access to Information: Major

By the end of 2013, public data were mostly accessible only upon request in Latvia. There were also no technical guidelines for releasing data for reuse, nor were there regulations that would prohibit institutions from selling data for commercial prices. Although the 2015 Global Open Data Survey noted that government datasets were mostly offered free of charge, ^{“Latvia,” Global Open Data Index: Survey, http://global.census.okfn.org/place/lv; and Uldis Bojars and Renars Liepins, “The State of Open Data in Latvia: 2014,” Baltic Journal of Modern Computing 2, no. 3: 160–70,}

https://arxiv.org/ftp/arxiv/papers/1406/1406.5052.pdf. scarce data made it hard for policy analysts and journalists to carry out evaluations in various policy areas.

The implementation of this commitment has provided the legal basis, practical support, and technical opportunities for government institutions to publish data in an open data format. In addition to new open data guidelines and legal definitions of data terms, the new open data portal, which is open to the public, contained 32 datasets from 12 institutions by the end of the action plan. The datasets are all free for reuse and have a Creative Commons licence. The law also now stipulates that in cases in which datasets are not free, the price must not exceed the price of their collection. Compared to the status quo at the outset of the action plan, when open data guidelines and a culture of open data were largely inexistent, ^{Nika Aleksejeva, “Latvia on Its Way to Open Data,” Datu Skola, 29 August 2016, http://www.datuskola.lv/2016/08/29/latvia-on-its-way-to-open-data/.} this commitment has led to a major change in government practise regarding access to information.

However, several public administration institutions that collect data, such as the State Land Service, are still partially financed by revenue from selling data. For example, geospatial data are mostly only available for a price. ^{“Workshop—Discussion on the Opening of Geospatial Data in Latvia,” http://lata.org.lv/seminars-diskusija-par-latvijas-geotelpisko-datu-atversana/.} Even government institutions, and local governments in some instances, have to acquire the data they need by paying other government entities. ^{“Informacijas Sabiedribas Padomes Protokols,” Cabinet of Ministers, http://www.mk.gov.lv/sites/default/files/editor/isp_13_07_2017_protokols_0.pdf.}

Ambitious goals still left to achieve include mandating that institutions release existing data in an open data format by default; changing the model for financing public administration institutions; and reducing, as much as possible, direct dependence of an institution’s core activities on revenues gained from selling data for reuse. These are issues to be monitored during the implementation of the next action plan.

Carried Forward?

The next steps include fine-tuning the portal. A larger challenge to address during the next action plan is to shift from suggested to legally binding practises for government institutions to release data by default. Another challenge lies in changing the model for financing government institutions that are partially financed by selling data. These actions would facilitate implementation of the principle that government-produced data must be available for reuse for free, thus complying with open data standards and definitions.

In the third action plan, the government commits to setting up a system for prioritising datasets for release in open data formats (free of charge) and to conducting promotional activities about the data available in these formats. The government also proposes activities to increase the number of datasets on the open data portal and to expand the circle of institutions that release data. In this sense, the next plan seems to pave the way toward the publication of data by default, which would better answer the public’s needs.