Policy and Data Management Interface

The Policy and Data Management component provides two different user interfaces: the web interface and the REST API interfaces. Users can select the one that is more suitable/intuitive for them. The REST APIs are also used for the integration of this component with other components in the AI4PublicPolicy architecture, for instance with the VPME.

Web Interface

A web interface was developed to ease the use of the component for non-technical users that upload datasets to the platform or define policies. All interfaces have been implemented using HTML and JavaScript. There are three main interfaces: datasets, policies and the AI models interfaces. Datasets and policies are grouped by area (Pilot): Urban Mobility (pilot leader: Nicosia Municipality), Data-Driven Water Infrastructure Planning and Maintenance (pilot leader: Burgas Municipality), Citizens and Business Services Optimisation (pilot partner: Commune di Genova), and Management and Optimisation of City Resources (pilot leader: Dimos Athinaion Epicheirisi Michanografisis (DAEM) – City of Athens IT Company) and Energy Management (pilot leader: Lisboa E-Nova – Agência Municipal de Energia e Ambiente).

Dataset schema definition

This interface is used to define the schema of a new type of dataset. That is the format and type of the elements entailed into a dataset. This interface is used only when a new pilot is added, or new types of data are generated. The following figure shows the Dataset Definition interface which is a form that internally is converted into a JSON object. The user selects an area, provides the dataset name, selects the type of files in the dataset (csv, json, geo-json, xlsx or image), and if it is an image, the format of the image has to be indicated (jpg, png), the description of the dataset, the owner, the language (English, Italian, Greek, Portuguese and Bulgarian), and the names of the fields of the dataset set if the dataset type is different to image. For each field of the dataset, a name, type, description, and default value have to be provided.

List of Datasets

Once the dataset is defined, its definition can be seen in the Datasets interface as shown in the following figure. The user selects one of the areas and all previously defined datasets are shown with a description and the language of the data. When clicking on the dataset description, the list of available datasets files is shown. A dataset is downloaded by clicking on it.

Uploading a Dataset

The user can upload files in two different ways: using the Dataset Management interface or uploading the dataset through the EGI DataHub interface. The Dataset Management interface in the following figure provides a form where the user selects the area, the dataset definition, and the endpoint where the dataset file to be uploaded is available. Then, clicking on the submit button the Dataset Management component downloads the file and stores it into a EGI DataHub folder that keeps all the files for that dataset.

Policy Definition

The policy interfaces are similar to the dataset ones. The following figure shows the policy definition interface where a new policy is defined. First of all, the area of the policy must be selected. Then, the name of the policy, the owner, the description, the policy objective, KPIs and metadata information must be provided. The datasets to be used in the policy are added in the datasets section of the form. This section contains a drop-down widget with the datasets available for the selected area. By clicking on the submit button the policy is stored into the database.

List of Policies

Once a policy is defined, it is available in the policies interface, shown in the following figure.

Clicking on a given area (Citizens and Business Services Optimisation in the figure) all stored policies with their name, description, and associated datasets are listed. Clicking on the More Information button the objectives, KPIs and metadata are shown.

Policy Update

The following figure shows how to update the information of a policy through the web interface. Selecting the area, a policy drop-down field with the policies stored for that area is shown. Selecting a policy, all the information associated with the policy is shown. The user can update any field and submit the changes.

AI Models Interface

The AI Models interface presents the AI Models defined for the different areas. Selecting the area, a drop-down menu with all the policies stored is shown. When the user selects a given policy, the AI Models associated with that policy are listed.

A video tutorial was recorded showing how to use the Policy and Data Management component web interface, this video is available at https://ai4publicpolicy.eu/ai4publicpolicy-dataset-and-policy-management-demonstration

EGI DataHub Interface

Another way to upload a dataset is using the EGI DataHub file system interface as depicted in the following figure. When the dataset is defined, the data management component creates internally a folder where datasets are stored. This interface is similar to any file system interface and therefore it is very intuitive for any type of user. Below, the figure shows the folder created for the Energy Management Policies Pilot area and the CITY SENSORS dataset schema. A user can log in to AI4PublicPolicy space and access that folder. The user can drag and drop the dataset or use the upload button on the right side of the interface.

REST API

Two different REST API have been developed to interact with the Policy and Data Management component, one for data management and the other one for the policies. These APIs can be used by programs from pilots to automatically upload data and are also used internally by the AI4PublicPolicy platform to manage policies and datasets.

Data Management REST API

The dataset management component provides a REST API with all operations needed for data management. The definition of REST API with the Swagger is shown in the figure below. Seven operations allow users to define, update, delete and read dataset schemas, and store datasets associated with a given schema.