The CRISP-DM Methodology applied in AI4PublicPolicy policy-making

The AI4PublicPolicy Policy Making Process leverages on the Cross-Industry Standard Process for Data Mining (CRISP-DM) methodology to provide a data-driven, AI and evidence-based approach for policy development, while fostering the collaboration between policy makers and AI experts. Through CRISP-DM the project, also, aims to involve citizens and other stakeholders in the policy evaluation and optimization procedures, thus boosting the acceptance of policies and explaining the policy-development outcomes. What is more, AI4PublicPolicy introduces the reusability and sharing of policy development models and datasets across different authorities.

CRISP-DM methodology was introduced in 2000 as an alternative to Knowledge Discovery in Databases (KDD), a previous data mining methodology, thus responding to common issues and needs. The methodology comprises 6 key steps starting from (1) business understanding to gain an understanding of objectives and requirements from a business perspective; (2) data understanding as an initial data collection; (3) data preparation covering activities required to construct the final dataset from the initial raw data; (4) the modelling phase when various modelling techniques are used; (5) the evaluation of the model to ascertain that the deployed models meet the initial objectives and (6) the final deployment phase to enable end-users to use the data as basis for decisions.

Based on the above approach the main steps of the project’s policy making process are structured as follows:

Policy Definition: In the first step the Policy Maker starts creating an Analytical Policy Model describing the Policy problem(s) and associating and describing relevant Datasets for the Policy.

Policy Extraction: Second, an AI Expert, or the Policy Maker himself with Auto Machine Learning support, creates one or more AI Workflows to analyse the datasets and prepare the data to train and test AI Models based on different AI Algorithms, in order to provide responses (insight, recommendations) to the Policy problem(s). The Policy Maker executes the AI Models on new data, analyses the responses and validates the AI Models.

Policy Evaluation: During the third step, the Policy Maker involves the relevant stakeholders (citizens, business and other local actors) in the Policy Evaluation creating one or more surveys on the Policy problems, AI model responses and Policy alternatives. The Policy Maker then evaluates the stakeholders feedbacks, presented with a statistical or sentiment scoring, and decides to complete the Policy with actionable outcomes or optimizing it taking into account the stakeholders feedbacks.

Policy Presentation and Sharing: In this final step the Policy Maker uses eXplainable AI techniques provided by the platform to better understand and explain the rationales under the AI Models responses. Once completed this step the Policy Maker could present the final Analytical Policy Model, which represent the result of the AI-based Policy Making Process, to the relevant stakeholders and publish it in the shared catalogue.