Microdata refers to databases where the micro-unit constitutes the central element of analysis – individuals, families, or businesses. This data is traditionally collected through surveys, censuses, or administrative data and allows users/researchers to analyze a wide range of topics and relationships between subpopulations. The characteristics of the data are generally determined by the guiding objective of its collection. As such, it usually does not cover all dimensions of analysis in depth, which creates the need to collect information through new and expensive surveys or other data collection methods.
In response to this problem, several methods have emerged that seek to utilize existing information dispersed across various databases. These methods aim to integrate multiple databases through a common set of variables. This document presents an analysis of the methods commonly used for integrating dispersed information in microdata, with particular emphasis on identifying the feasibility of integrating the Staffing Survey (QPS) and the European Working Conditions Survey (EWCS). The techniques considered here fall into three distinct categories: (1) parametric; (2) non-parametric; and (3) mixed.
The results of this analysis suggest that the EWCS and QPS can be successfully integrated through statistical matching methods. As expected, there is an integration cost associated with this procedure, which is reflected in the probability distributions of the new synthetic database. In order to successfully integrate the two information sources, an extensive harmonization procedure is necessary, requiring the aggregation of some continuous variables, which translates into an implicit loss of specificity of the information contained in the databases. Finally, due to the associated computational requirements, it was not possible to optimize the matching process. The ideal optimization should be obtained through an algorithm that solves the assignment problem. However, a heuristic approach was used to optimize our problem, minimizing the distances between individuals in the two databases through sequential iteration.
Authors: Luís Manso, Jena Santi
