Semantic heterogeneity
From Wikipedia, the free encyclopedia
Semantic heterogeneity is when database schema or datasets for the same domain are developed by independent parties, resulting in differences in meaning and interpretation of data values.[1] Beyond structured data, the problem of semantic heterogeneity is compounded due to the flexibility of semi-structured data and various tagging methods applied to documents or unstructured data. Semantic heterogeneity is one of the more important sources of differences in heterogeneous datasets.
Yet, for multiple data sources to interoperate with one another, it is essential to reconcile these semantic differences. Decomposing the various sources of semantic heterogeneities provides a basis for understanding how to map and transform data to overcome these differences.
History
Conflicts between independently designed database schemas were studied extensively in the 1980s as part of research in schema matching. Batini, Lenzerini, and Navathe compared existing integration methodologies in 1986 and calloused the naming and structural conflicts each had to resolve.[2] William Kent's 1989 paper "The many forms of a single fact" used examples to show how one fact can be represented in many different forms.[3]
Sheth and Larson's 1990 survey of federated database systems defined semantic heterogeneity as disagreement about the meaning, interpretation, or intended use of the same or related data. They separated these conflicts of meaning from differences among database systems and data models.[4]
Later work concentrated on automating the detection and resolution of such conflicts. Rahm and Bernstein reviewed techniques for automatic schema matching in 2001,[5] and Euzenat and Shvaiko's Ontology Matching covers the corresponding problem for ontologies.[6] Halevy argued in 2005 that semantic heterogeneity is inevitable because schemas will always be built by independent parties.[7] Machine learning and, more recently, large language models have been applied to schema matching, ontology alignment and entity matching.[8]
Classification
One of the first known classification schemes applied to data semantics is from William Kent in the late 80s.[9] Kent's approach dealt more with structural mapping issues than differences in meaning, which he pointed to data dictionaries as potentially solving.
One of the most comprehensive classifications is from Pluempitiwiriyawej and Hammer, "Classification Scheme for Semantic and Schematic Heterogeneities in XML Data Sources".[10] They classify heterogeneities into three broad classes:
- Structural conflicts arise when the schema of the sources representing related or overlapping data exhibit discrepancies. Structural conflicts can be detected when comparing the underlying schema. The class of structural conflicts includes generalization conflicts, aggregation conflicts, internal path discrepancy, missing items, element ordering, constraint and type mismatch, and naming conflicts between the element types and attribute names.
- Domain conflicts arise when the semantics of the data sources that will be integrated exhibit discrepancies. Domain conflicts can be detected by looking at the information contained in the schema and using knowledge about the underlying data domains. The class of domain conflicts includes schematic discrepancy, scale or unit, precision, and data representation conflicts.
- Data conflicts refer to discrepancies among similar or related data values across multiple sources. Data conflicts can only be detected by comparing the underlying sources. The class of data conflicts includes ID-value, missing data, incorrect spelling, and naming conflicts between the element contents and the attribute values.
Moreover, mismatches or conflicts can occur between set elements (a "population" mismatch) or attributes (a "description" mismatch).
Michael Bergman expanded upon this schema by adding a fourth major explicit category of language, and also added some examples of each kind of semantic heterogeneity, resulting in about 40 distinct potential categories [11] .[12] This table shows the combined 40 possible sources of semantic heterogeneities across sources:
| Class | Category | Subcategory | Examples |
| Ingest Encoding Mismatch | |||
| Ingest Encoding Lacking | Mis-recognition of tokens because not being parsed with the proper encoding | ||
| Query Encoding Mismatch | For example, ASCII v UTF-8 in search | ||
| Query Encoding Lacking | Mis-recognition of search tokens because not being parsed with the proper encoding | ||
| Languages | Script Mismatch | Variations in how parsers handle, say, stemming, white spaces or hyphens | |
| Parsing / Morphological Analysis Errors (many) | Arabic languages (right-to-left) v Romance languages (left-to-right) | ||
| Syntactical Errors (many) |
Ambiguous sentence references, such as I'm glad I'm a man, and so is Lola (Lola by Ray Davies and the Kinks) | ||
| Semantics Errors (many) | River bank v money bank v billiards bank shot | ||
| Conceptual | Naming | Case Sensitivity | Uppercase v lower case v Camel case |
| United States v USA v America v Uncle Sam v Great Satan | |||
| United States v USA v US | |||
| Such as when the same name refers to more than one concept, such as Name referring to a person v Name referring to a book | |||
| Misspellings | As stated | ||
| Generalization / Specialization | When single items in one schema are related to multiple items in another schema, or vice versa. For example, one schema may refer to "phone" but the other schema has multiple elements such as "home phone", "work phone" and "cell phone" | ||
| Aggregation | Intra-aggregation | When the same population is divided differently (such as, Census v Federal regions for states, England v Great Britain v United Kingdom, or full person names v first-middle-last) | |
| Inter-aggregation | May occur when sums or counts are included as set members | ||
| Internal Path Discrepancy | Can arise from different source-target retrieval paths in two different schemas (for example, hierarchical structures where the elements are different levels of remove) | ||
| Missing Item | Content Discrepancy | Differences in set enumerations or including items or not (say, US territories) in a listing of US states | |
| Missing Content | Differences in scope coverage between two or more datasets for the same concept | ||
| Attribute List Discrepancy | Differences in attribute completeness between two or more datasets | ||
| Missing Attribute | Differences in scope coverage between two or more datasets for the same attribute | ||
| Item Equivalence |
When two types (classes or sets) are asserted as being the same when the scope and reference are not (for example, Berlin the city v Berlin the official city-state) | ||
|
When two individuals are asserted as being the same when they are actually distinct (for example, John F. Kennedy the president v John F. Kennedy the aircraft carrier) | |||
| Type Mismatch | When the same item is characterized by different types, such as a person being typed as an animal v human being v person | ||
| Constraint Mismatch | When attributes referring to the same thing have different cardinalities or disjointedness assertions | ||
| Schematic Discrepancy | Element-value to Element-label Mapping | One of four errors that may occur when attribute names (say, Hair v Fur) may refer to the same attribute, or when same attribute names (say, Hair v Hair) may refer to different attribute scopes (say, Hair v Fur) or where values for these attributes may be the same but refer to different actual attributes or where values may differ but be for the same attribute and putative value. Many of the other semantic heterogeneities herein also contribute to schema discrepancies | |
| Attribute-value to Element-label Mapping | |||
| Element-value to Attribute-label Mapping | |||
| Attribute-value to Attribute-label Mapping | |||
| Scale or Units | Measurement Type | Differences, say, in the metric v English measurement systems, or currencies | |
| Units | Differences, say, in meters v centimeters v millimeters | ||
| Precision | For example, a value of 4.1 inches in one dataset v 4.106 in another dataset | ||
| Primitive Data Type |
Confusion often arises in the use of literals v URIs v object types | ||
| Data Format | Delimiting decimals by period v commas; various date formats; using exponents or aggregate units (such as thousands or millions) | ||
| Naming | Case Sensitivity | Uppercase v lower case v Camel case | |
| Synonyms | For example, centimeters v cm | ||
| Acronyms | For example, currency symbols v currency names | ||
| Homonyms | Such as when the same name refers to more than one attribute, such as Name referring to a person v Name referring to a book | ||
| Misspellings | As stated | ||
| ID Mismatch or Missing ID | URIs can be a particular problem here, due to actual mismatches but also use of name spaces or not and truncated URIs | ||
| Missing Data |
A common problem, more acute with closed world approaches than with open world ones | ||
| Element Ordering | Set members can be ordered or unordered, and if ordered, the sequences of individual members or values can differ | ||
A different approach toward classifying semantics and integration approaches is taken by Sheth et al.[13] Under their concept, they split semantics into three forms: implicit, formal and powerful. Implicit semantics are what is either largely present or can easily be extracted; formal languages, though relatively scarce, occur in the form of ontologies or other description logics; and powerful (soft) semantics are fuzzy and not limited to rigid set-based assignments. Sheth et al.'s main point is that first-order logic (FOL) or description logic is inadequate alone to properly capture the needed semantics.
Resolution approaches
The traditional approach to resolution is mapping where a practitioner studies the sources, writes rules to translate between them, and revises the rules when a schema changes.[14] Wiederhold proposed "mediators" for this: software modules that sit between applications and data sources and hold the knowledge needed to translate.[15] Later data integration systems accept queries against a single mediated schema and answer them through stored mappings to each source.[16]
The core problem to this approach is that writing mappings by hand is slow, error-prone, and can have biases based on the practitioner. Schema matchers compare element names, structures and data values to propose likely correspondences.[5] The same idea extends to ontologies.[6] Entity matching (also called record linkage) works on the data itself, deciding whether two records refer to the same real-world entity.[17]
Neural networks changed the matchers. Trained on labeled pairs, they learn the similarity judgments that older systems encoded in rules.[18] Pre-trained language models represent records as embeddings, so a matcher can tell that "HTN" and "high blood pressure" name the same condition.[8] Large language models judge matches from a prompt, with a few labeled examples or none.[8] Knowledge graphs supply what the records leave out, such as synonyms and type hierarchies.[8]
Standards address the problem before integration begins. The FAIR principles, published in 2016, ask data publishers to describe their data with shared, formal vocabularies.[19] Datalake systems store data in its original form and add semantics afterward with a metadata layer that links stored files to knowledge graphs, and ontology-based data access rewriting queries over an ontology into queries over the sources.[20]
Relevant applications
Besides data interoperability, relevant areas in information technology that depend on reconciling semantic heterogeneities include data mapping, semantic integration, and enterprise information integration, among many others. From the conceptual to actual data, there are differences in perspective, vocabularies, measures and conventions once any two data sources are brought together. Explicit attention to these semantic heterogeneities is one means to get the information to integrate or interoperate.
In practice, such conflicts arise when independent groups record the same facts.[7] The same product can appear under its full retail name in one catalog and under an internal stock code in another.[8] Geographic information systems built by different agencies describe the same terrain in different terms.[21] NASA lost the Mars Climate Orbiter in 1999 because ground software reported thruster impulse in pound-force seconds while the navigation software expected newton-seconds.[22] In medicine, where one condition can go by many names, the Unified Medical Language System links more than two million names from over 60 families of biomedical vocabularies to some 900,000 common concepts.[23]