Schema Evolution and Table Biographies: Studying the Lives of Tables

Our research on table biographies studies schema evolution at the level of individual relational tables. We investigate how properties such as table birth, deletion, lifetime, size, and update activity are related, and identify recurring patterns that distinguish stable, actively evolving, surviving, and eventually deleted tables. This work also includes methods for visualizing the parallel evolution of tables within a database schema.

Patterns of Table Evolution

In [ER15, IS17], we identified patterns of table evolution. Our findings indicate that (i) most tables live quiet lives; (ii) few top-changers adhere to a profile of long duration, early birth, medium schema size at birth; (iii) tables with large schemata or long duration are quite unlikely to be removed, and, (iv) early periods of the database life demonstrate a higher level of evolutionary activity compared to later ones.

In [CAiSE17, JoDS17] we demonstrate the electrolysis pattern, stating that survivor and ultimately deleted tables live quite different lives: tables typically die shortly after birth, with short durations and mostly no updates, whereas survivors mostly live quiet lives with few updates -- except for a small group of tables with high update ratios that are characterized by high durations and survival.

In [ER23] we introduce Parallel Lives Diagrams, a method that allows the visual representation of the evolution of the tables of a schema via a visual 2D encoding of time, amount of change, birth and death.

Research on Table Evolution

Fanis Giachos, Nikos Pantelidis, Christos Batsilas, Apostolos V. Zarras, Panos Vassiliadis. Parallel lives diagrams for co-evolving communities and their application to schema evolution. Companion Proceedings of the 42nd International Conference on Conceptual Modeling (ER 2023): ER Forum, 7th SCME, Project Exhibitions, Posters and Demos, and Doctoral Consortium. Lisbon, Portugal, November 6-9, 2023.

Focus: Introduces parallel-lives diagrams for visualizing how multiple relational tables co-evolve across the successive versions of a database schema.
Related material: local page for Parallel lives diagrams for co-evolving communities and their application to schema evolution with paper and supplementary material.

Panos Vassiliadis, Apostolos V. Zarras. Schema Evolution Survival Guide for Tables: Avoid Rigid Childhood and You’re En Route to a Quiet Life. Journal on Data Semantics, Volume 6, Issue 4, December 2017, pp 221–241, ISSN: 1861-2040, DOI: 10.1007/s13740-017-0083-x.

Focus: Details the electrolysis pattern of CAiSE 17, establishing the systematic differences of survivor and ultimately deleted tables wrt their birth characteristics, lifetime, and subsequent schema-update behavior.
DOI: 10.1007/s13740-017-0083-x.
Related material: local page for Schema Evolution Survival Guide for Tables with highlights, papers, presentations, data, code, and results.

Panos Vassiliadis, Apostolos Zarras. Survival in schema evolution: putting the lives of survivor and dead tables in counterpoint. 29th International Conference on Advanced Information Systems Engineering (CAiSE 2017), 12-16 June 2017, Essen, Germany.

Focus: Compares survivor and eventually deleted tables and demonstrates the electrolysis pattern: deleted tables typically die shortly after birth, with no updates, whereas survivors mostly live quiet lives with few updates -- except for a small group of tables with high update ratios that are characterized by high durations and survival.
DOI: 10.1007/978-3-319-59536-8_21.
Related material: local page for Survival in Schema Evolution with highlights, paper, presentation, data, code, and results.

Panos Vassiliadis , Apostolos V. Zarras , Ioannis Skoulis. Gravitating to Rigidity: Patterns of Schema Evolution -and its Absence- in the Lives of Tables. Information Systems, Volume 63, January 2017, Pages 24-46, ISSN 0306-4379, doi:10.1016/j.is.2016.06.010.

Focus: Identifies recurring patterns in individual table histories and shows a broad gravitation toward rigidity, with many tables experiencing little or no change after their early life.
DOI: 10.1016/j.is.2016.06.010.
Related material: local page for the table-evolution study with highlights, papers, presentations, data, code, and results.

Panos Vassiliadis, Apostolos Zarras, Ioannis Skoulis. How is Life for a Table in an Evolving Relational Schema? Birth, Death & Everything in Between. 34th International Conference on Conceptual Modeling (ER 2015). 19-22 October 2015, Stockholm, Sweden.

Focus: Studies tables as evolving entities, relating birth, death, lifetime, size, and update activity to reveal characteristic behaviors across a table's life.
DOI: 10.1007/978-3-319-25264-3_34.
Related material: local page for How is Life for a Table in an Evolving Relational Schema? with highlights, paper, presentation, data, code, and results.

Talks and Videos on Table Evolution

Highlights of Information Systems 2017
Highlights of CAiSE 2017: Electrolysis
Keynote talk at DATA 2016


Schema Evolution and Gravitation to Rigidity: a tale of calmness in the lives of structured data: a keynote talk at the 7th International Conference on Model and Data Engineering (MEDI 2017), October 4-6, 2017 - Barcelona. [Click here for a 5-pages paper]