Schematix
An explosive normalization tool
Schematix is a tool for those who face the problem of Data Wrangling: the art, or science, or conforming your data to a desired format. Data analysis tools are extremely high in demand. However, one known issue with data analysis is an inconsistency in formatting between different data sources. Is my data compatible with the analysis tool I’m trying to use? This problem is only exacerbated when data is represented inconsistently between various mediums. Take for example a team of two, each trying to represent the same structured data. Let’s say that one person decided to create an Excel spreadsheet while the other wrote a Python script to populate a CSV file. What if they chose inconsistent names of attributes for enumerated types (i.e. “female” vs. “f” vs. “Female”)? What if one person decided to leave some cells blank, while the other didn’t? What if the person entering into Excel had typos that were corrupting various aggregation results? Suddenly, the team’s simple task of trying to merge and aggregate their data with some analysis tool becomes much more complicated.
Schematic fission is a tool to help users easily and intuitively migrate their data from non-relational, spreadsheet-like data management tools, such as Excel, to relational databases. Our tool is inspired by and heavily uses the Stanford Visualization Group’s DataWrangler, an open source project for “wrangling” semi-structured CSV data into a giant spreadsheet format. Our tool takes this a step further and helps users detect functional dependencies that are definitely present in their data, infer functional dependencies that may be present in the data, and normalize the data based on those functional dependencies into a traditional relational schema. By the end, the user can play around with their normalized schema in our web interface (including full query support) and export the necessary SQL statements to create their own database locally.
