User-oriented exploration of semi-structured datasets

User-oriented exploration of semi-structured datasets Nelly Barret 3rd year PhD student Supervised by Ioana Manolescu Inria Saclay and Institut Polytechnique de Paris October 9, 2023 Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 1 / 38 Context: data is the new gold (1/3) Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 2 / 38 Context: data is the new gold (1/3) Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 3 / 38 Context: data is the new gold (2/3) Our digital world comes: In various contexts: science, health, political life At various scales: home, city, country, world By different actors: scientists, businesses, policy makers With different needs, constraints, abilities We are overwhelmed by (raw) data, we need: Data-driven applications Data journalism Knowledge graphs Artificial “intelligence” ... Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 4 / 38 Context: data is the new gold (3/3) Very heterogeneous data: Mainly RDF (1K datasets in the LODC) Also: XML, JSON, relational, Property Graph... Detection of entities of interest: People, Place, email, ... With heterogeneous data, users need: 1Auniform integration, view 2Efficient algorithms and applications 3A global understanding,description 4Interesting entity connections Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 5 / 38 Create a unique data graph “A uniform integration, view” Angelos Anadiotis Oana Balalau Ioana Manolescu et al... IPP, EPFL Inria, IPP Inria, IPP INSEC, ... Heterogeneous input Loading Collection graph Abstra PathWays New features Studio Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 6 / 38 Graph construction Ingest any dataset into a directed graph (•,→) Extract named entities, NEs, from the graph values (◦,99K): Temporal: date , time reference Web: URI, email address , hashtag, Twitter citation Complex entities: People , Place, Organization Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 7 / 38 Create a compact representation of the data graph “Efficient algorithms and applications” Heterogeneous input Loading Collection graph Abstra PathWays New features Studio Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 8 / 38 A uniform view of data formats Each data format has its own specificities: RDF (URIs) JSON (labels) XML (ID-IDREF) PG (edge attrs) But, we encode the same logic: Record: piece of data, an object Value: record with no children Same-kind records: schema or intuitive order Relationship: how records relate Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 9 / 38 Collections weights, boundaries and graph updates Collection weight wdesck wleafk wDAG wPageRank wdwPageRank Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 15 / 38 Collections weights, boundaries and graph updates Collection weight wdesck wleafk wDAG wPageRank wdwPageRank Boundary bounddesc boundleaf boundDAG boundflood boundacyclic−flood Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 16 / 38 Collections weights, boundaries and graph updates Collection weight wdesck wleafk wDAG wPageRank wdwPageRank Boundary bounddesc boundleaf boundDAG boundflood boundacyclic−flood Graph update updateboolean updateexact Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 17 / 38 Find relationships between main collections Possible relationships The set of relationships connecting a pair of collections is the set of their paths. paper →wB →author paper →pIn →conf author →hW →paper conf →inv →author Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 18 / 38 The final output in Abstra https://team.inria.fr/cedar/projects/abstra/ Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 19 / 38 Enumerate entity paths “Interesting entity connections” Nelly Barret Antoine Gauquier Jia Jean Law Ioana Manolescu Inria, IPP IMT IPP Inria, IPP Heterogeneous input Loading Collection graph Abstra PathWays New features Studio Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 20 / 38 PathWays: find interesting connections in the data Problem statement How to interactively explore entity connections in heterogeneous datasets? 1No query writing, nor prior knowledge 2Atabular, high-level output, easy to grasp for NTUs 3Do it efficiently even if the data graph is large =⇒Connect named entities (People, Places, ...) in and across datasets. Keyword Graph Reachability PathWays search query query No query writing X× × X Tabular output ∼ ∼ ×X Efficient ×X X X Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 21 / 38 PathWays: find interesting connections in the data Problem statement How to interactively explore entity connections in heterogeneous datasets? 1No query writing, nor prior knowledge 2Atabular, high-level output, easy to grasp for NTUs 3Do it efficiently even if the data graph is large =⇒Connect named entities (People, Places, ...) in and across datasets. Keyword Graph Reachability PathWays search query query No query writing X× × X Tabular output ∼ ∼ ×X Efficient ×X X X Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 21 / 38 Scenario and terminology Adata (entity) path is a path in the data graph Acollection (entity) path is a path in the collection graph The evaluation of a collection path leads to a set of data paths Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 22 / 38 Collection (entity) path enumeration (τ1, τ2); max path length; non-specific connections Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 23 / 38 Quick overview of experiments On widely-used open data formats: JSON, RDF, XML and PG. Abstra PathWays •User study •# paths: 0 to very high •Comparison to schemas •4 path “shapes” Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 30 / 38 Future work, takeaways and open questions Heterogeneous input Loading Collection graph Abstra PathWays New features Studio Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 31 / 38 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data French media are interested (DataJournos, CFI) Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 32 / 38 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 33 / 38 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 34 / 38 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 35 / 38 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 36 / 38 Takeaways and open questions Abstra: a dataset abstraction system for heterogeneous data PathWays: an entity-focused exploration system Studio: a user-oriented data lake for data exploration Abstra PathWays Studio EDBT 2024 ADBIS 2023 CoopIS 2023 Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 37 / 38 Further opportunities Nelly BARRET ¥[email protected] §https://pages.saclay.inria.fr/nelly.barret/ Inria Saclay & Institut Polytechnique de Paris Palaiseau Nelly Barret (Inria) Semi-structured Data Exploration October 9, 2023 38 / 38