Schema Validation

Note

This feature is still experimental and in active development. Your feedback is highly appreciated!

All of the below actions and examples can also be executed from the Semantic Objects’ web-based administration interface, the Workbench.

The information on this page is intended for anyone who wants to write a schema from scratch or to improve a schema generated from an ontology using the OWL2SOML tool.

Overview

Ontotext Semantic Objects 3.6 introduces an extension to the existing schema validation that allows validation of the SOML schema against RDF data.

The following validation checks are supported by the Semantic Objects:

The functionality is not intended for validating the data against a particular schema, but rather the other way round. Thus, it does not guarantee data consistency. For this purpose, use the SHACL validation support instead.

Property Range Validation

The validation will check for any discrepancies in the data compared to the one defined in the model. The result of the validation is a report that will list the number of values that have data types different from the accepted ones.

Here is an example of such discrepancies for literal properties.

Consider the following schema snippet:

id: /soml/swapi-validations
specialPrefixes:
  base_iri: https://swapi.co/resource/
  vocab_iri: https://swapi.co/vocabulary/

objects:
  Character:
    kind: abstract
    name: rdfs:label
    props:
       birthYear: {range: int}

  Human:
    inherits: Character

And the database containing the following data:

@base <https://swapi.co/resource/> .
@prefix voc:   <https://swapi.co/vocabulary/> .
@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .

<human/67>  a      voc:Human , voc:Character ;
    rdfs:label     "Dooku" ;
    voc:birthYear  "102BBY" .

<human/68>  a      voc:Character , voc:Human ;
    rdfs:label     "Bail Prestor Organa" ;
    voc:birthYear  "67BBY" .

<human/69>  a      voc:Human , voc:Character ;
    rdfs:label     "Jango Fett"@en ;
    voc:birthYear  "66BBY" .

<human/62>  a      voc:Human , voc:Character ;
    rdfs:label     "Cliegg Lars" ;
    voc:birthYear  "4082"^^xsd:gYear .

The following warning will be returned:

  • “The property ‘Human.birthYear’ is defined as ‘range: int’, but found 3 instances of values with types: ‘string’ and 1 instance of ‘xsd:gYear’”

And here is an example of discrepancies for object properties.

Consider the following schema:

id: /soml/swapi-validations
specialPrefixes:
  base_iri: https://swapi.co/resource/
  vocab_iri: https://swapi.co/vocabulary/

objects:
  Character:
    kind: abstract
    name: rdfs:label

  Human:
    inherits: Character
    props:
      homeworld: {range: Planet}

  Planet:
    name: rdfs:label

And the database with the following data:

@base <https://swapi.co/resource/> .
@prefix voc:   <https://swapi.co/vocabulary/> .
@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .

<human/67>  a      voc:Human , voc:Character ;
    rdfs:label     "Dooku" ;
    voc:homeworld  <planet/52> .

<human/68>  a      voc:Character , voc:Human ;
    rdfs:label     "Bail Prestor Organa" ;
    voc:homeworld  <planet/2> .

<human/69>  a      voc:Human , voc:Character ;
    rdfs:label     "Jango Fett"@en ;
    voc:homeworld  "Concord Dawn" .

<human/62>  a      voc:Human , voc:Character ;
    rdfs:label     "Cliegg Lars" ;
    voc:homeworld  <planet/Tatooine> .

<planet/52>  a     voc:Planet ;
    rdfs:label     "Serenno" .

<planet/2>  a      voc:Planet ;
    rdfs:label     "Alderaan" .

This would produce the following warning:

  • “The property ‘Human.rdfs:label’ is defined as ‘range: string’, but found 1 instance of values with types: ‘rdf:langString’”
  • “The property ‘Human.homeworld’ is defined as ‘range: Character’, but found 1 instance of values with types: ‘iri’, 1 instance of ‘xsd:string’ and 2 instances of ‘voc:Planet’”

The first warning can be resolved by defining the rdfs:label explicitly and setting a proper range for it and the second by setting the range as Planet:

Character:
  kind: abstract
  name: rdfs:label
  props:
    rdfs:label: {range: stringOrLangString, min: 1, max: inf, lang: "en"}

Human:
  inherits: Character
  props:
    homeworld: {range: Planet}

Property Cardinality Validation

The validation will check if the defined cardinality constraints are already violated according to the data found in the linked database.

The following checks are performed:

  • Properties defined as single-valued (max: 1) will be checked if they are objects with more than one value.
  • Types with required properties that affect the GraphQL query response. A property is considered defined as required when it is either defined as min: 1 or non nullable single valued (nonNullable: true,  max: "1"). The matching entities will be reported if they do not have value for any of the required properties.
  • Types with multivalued properties (max: 2, max: 3, …, max: inf) will be checked if all of the entity values are single-valued and if so it will issue an information message suggesting to make them single-valued by setting max: 1.

Consider the following schema:

id: /soml/swapi-validations
specialPrefixes:
  base_iri: https://swapi.co/resource/
  vocab_iri: https://swapi.co/vocabulary/

objects:
  Character:
    kind: abstract
    name: rdfs:label
    props:
      rdfs:label: {range: stringOrLangString, min: 1, lang: "en"}

  Human:
    inherits: Character
    props:
      homeworld: {range: Planet, max: 2}

  Planet:
    name: rdfs:label
    props:
      habitable: {range: boolean, nonNullable: true, max: 1}

Run against the following data:

@base <https://swapi.co/resource/> .
@prefix voc:   <https://swapi.co/vocabulary/> .
@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .

<human/67>  a      voc:Human , voc:Character ;
    rdfs:label     "Dooku" ;
    voc:homeworld  <planet/52> .

<human/68>  a      voc:Character , voc:Human ;
    voc:homeworld  <planet/2> .

<human/69>  a      voc:Human , voc:Character ;
    rdfs:label     "Jango Fett"@en , "Джанго Фет"@bg , "ジャンゴ・フェット"@jp .

<planet/52>  a     voc:Planet ;
    rdfs:label     "Serenno" ;
    voc:habitable  "true"^^xsd:boolean .

<planet/2>  a      voc:Planet ;
    rdfs:label     "Alderaan" .

After running the validation of the schema against the given data, we get the following information and warning messages:

  • “The property ‘Human.rdfs:label’ is defined as ‘max: inf’, but could not find an object with multiple values”
  • “The property ‘Human.homeworld’ is defined as ‘max: 2’, but could not find an object with multiple values”
  • “The property ‘Human.rdfs:label’ is defined as ‘max: 1’, but found multiple values for one or more objects”
  • “The property ‘Human.rdfs:label’ is defined as ‘max: inf’, but could not find an object with multiple values”
  • “The property ‘Human.homeworld’ is defined as ‘max: 2’, but could not find an object with multiple values”
  • “The property ‘Planet.habitable’ is defined as ‘max: inf’, but could not find an object with multiple values”
  • “The property ‘Human.name’ is defined as ‘min: 1’, but some objects do not have any value for this property”
  • “The property ‘Human.rdfs:label’ is defined as ‘min: 1’, but some objects do not have any value for this property”
  • “The property ‘Planet.habitable’ is defined as ‘nonNullable: true’, but some objects do not have any value for this property”

Object Type Validation

The validation will check if a particular entity matches more than one type defined in the current model.

Let’s look at this example of discrepancies for object types.

Consider the following schema:

id: /soml/swapi-validations
specialPrefixes:
  base_iri: https://swapi.co/resource/
  vocab_iri: https://swapi.co/vocabulary/

objects:
  Character:
    kind: abstract
    name: rdfs:label

  Human:
    inherits: Character

  Pilot:
    name: rdfs:label

  Rebel:
    name: rdfs:label

Run against the following data:

@base <https://swapi.co/resource/> .
@prefix voc:   <https://swapi.co/vocabulary/> .
@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .

<human/67>  a      voc:Human , voc:Character ;
    rdfs:label     "Dooku" .

<human/68>  a      voc:Character , voc:Human ;
    rdfs:label     "Bail Prestor Organa" .

<human/69>  a      voc:Human , voc:Character , voc:Pilot ;
    rdfs:label     "Jango Fett" .

<human/62>  a      voc:Human , voc:Character , voc:Pilot , voc:Rebel;
    rdfs:label     "Cliegg Lars" .

After running the validation for the provided data and schema snippet, the result will be this:

  • “Found object with multiple concrete types: Rebel, Human, Pilot”
  • “Found object with multiple concrete types: Human, Pilot”

Validation API

Only managed schemas can be validated. This results in the fact that the validation would not report any schema errors as these must be fixed before the actual schema is persisted.

See more about schema management in the Schema Administration section.

Start Validation

POST /soml/{id}/validate
POST /soml/{id}/validate?type=[SO type]*
POST /soml/{id}/validate?type=[SO type]* &repository=[target repository]?
# The following is supported from Semantic Objects 4.1
POST /soml/{id}/validate?type=[SO type]* &property=[properties filter]* &validation=[validations]* &repository=[target repository]?

The API supports full validation or partial validation for one or more types and/or one or more properties. In the request, you can also specify which validations are to be run. If the request does not include a type, then all types will be validated. Likewise, if it does not include properties filter, then all properties for each of the types will be validated; if it does not include validation parameters, then all validations will be run.

In addition to specifying which exact properties should be checked, the properties filter also allows specifying the properties with a wildcard *. For example, if the type is not restricted and the properties rdf:type and rdfs:label are present in the schema for the specified type (or for all types), then the property=rdf* argument will match those properties.

The supported values for validation parameter are:

  • propertyMaxCardinality - Validation step that checks if a particular property has a correct max cardinality value according to the database. The check should fail if property is defined as single value yet at least one object with multiple values is found in the data.
  • propertyMinCardinality - Validation step that checks if a particular property has a correct min cardinality value according to the database. The check should fail if property is defined as required or non nullable yet at least one object does not have such a property.
  • objectPropertyRangeMismatch - Validations step that checks if a particular object property has proper range according to the database. For example, if the range is defined as range: Character and there are predicated in the database that point to Starships, then the validation will fail. It will also report which other types are found according to the model. This allows data scientists to decide if they want to change the type.
  • scalarPropertyMismatch - Validations step that checks if a particular scalar property has a proper range according to the database. For example, this validation will report if a property in the schema is defined as an integer using the definition range: int, but the database contains data for that property that is not interger and is of type string.
  • maxCardinalityAdvice - Validation step that checks whether all of the values in given multi-valued properties are single valued. If there are any properties that aren’t single-valued, it suggests the user to define them as as such by setting their max characteristic to 1.
  • typeValidation - Type validation that checks if a particular entity matches any other type defined in the current model. If one of the matched types has a common parent with the current type, then a warning will be issued. The validation supports query visualisation and will list objects that have more than one type.

If there is a validation in progress for the given schema, it will be cancelled and started again.

If there is a completed validation for the given schema, it will be removed and a new one will be started.

Since Semantic Objects 4.0 you can run the validation on a repository different than the primary one by specifying the target repository. The repository could be specified as request parameter as well as value through the HTTP header X-Repository

For example:

curl -X POST "http://localhost:9995/soml/swapi-validations/validate?repository=soaas"

The response of the request will look like this:

{
  "@context": {
    "@vocab": "http://ontotext.com/ontology/validation/",
    "@base": "http://data.ontotext.com/",
    "xsd": "http://www.w3.org/2001/XMLSchema#",
    "hydra": "http://www.w3.org/ns/hydra/core#"
  },
  "@type": "ValidationReport",
  "schemaId": "/soml/swapi-validations",
  "repository": "soaas",
  "createdOn": "2021-09-08T17:51:46.060+03:00",
  "totalTasks": 12,
  "remainingTasks": 12,
  "infoTasks": 0,
  "warningTasks": 0,
  "failedTasks": 0,
  "done": false,
  "id": "3ed31fd6-a740-4bd3-9572-86ba2e415bf0",
  "tasks": [
    {
      "id": "8791a2fb-604e-4140-8dd8-b11bd1a8ec52",
      "validatorName": "typeValidation",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "2415dc08-4057-4973-8bea-91317b24472c",
      "validatorName": "scalarPropertyMismatch",
      "objectType": "Human",
      "propertyName": "type",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "570d37cd-5262-4546-baa8-b8dc2ab20a05",
      "validatorName": "propertyMinCardinality",
      "objectType": "Human",
      "propertyName": "name",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "08cc5eb8-cf3b-41cd-a11f-4ede52d78d5d",
      "validatorName": "propertyMinCardinality",
      "objectType": "Human",
      "propertyName": "rdfs:label",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "d7523760-38aa-495b-abfc-63f95289c736",
      "validatorName": "maxCardinalityAdvice",
      "objectType": "Human",
      "propertyName": "rdfs:label",
      "status": "NOT_RUN",
      "queryable": false
      "dataQueryable": false
    },
    {
      "id": "e7922d7d-e8de-40e9-9d17-6ec264e5fa9f",
      "validatorName": "propertyMaxCardinality",
      "objectType": "Human",
      "propertyName": "homeworld",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "46140d74-a8f6-41eb-a8bf-cb2cf34836ab",
      "validatorName": "objectPropertyRangeMismatch",
      "objectType": "Human",
      "propertyName": "homeworld",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "e2664435-43b2-421d-9c63-77f0c5dc398a",
      "validatorName": "scalarPropertyMismatch",
      "objectType": "Planet",
      "propertyName": "type",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "cc862808-199a-405f-8247-04b3fff06358",
      "validatorName": "propertyMinCardinality",
      "objectType": "Planet",
      "propertyName": "name",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "4a2a79f5-3755-4124-9aaa-3d58da7fc490",
      "validatorName": "propertyMaxCardinality",
      "objectType": "Planet",
      "propertyName": "rdfs:label",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "b010696d-39e6-48d2-bef9-a52ca483b66c",
      "validatorName": "propertyMinCardinality",
      "objectType": "Planet",
      "propertyName": "rdfs:label",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "53166e54-ba26-49ef-9ae5-da729d449652",
      "validatorName": "scalarPropertyMismatch",
      "objectType": "Planet",
      "propertyName": "rdfs:label",
      "status": "NOT_RUN",
      "queryable": true
    }
  ]
}

Here are a single type validation request and response:

Request:

curl -X POST "http://localhost:9995/soml/swapi-validations/validate?type=Human"

Response:

{
  "@context": {
    "@vocab": "http://ontotext.com/ontology/validation/",
    "@base": "http://data.ontotext.com/",
    "xsd": "http://www.w3.org/2001/XMLSchema#",
    "hydra": "http://www.w3.org/ns/hydra/core#"
  },
  "@type": "ValidationReport",
  "schemaId": "/soml/swapi-validations",
  "repository": null,
  "createdOn": "2021-09-09T10:40:45.749+03:00",
  "totalTasks": 7,
  "remainingTasks": 7,
  "infoTasks": 0,
  "warningTasks": 0,
  "failedTasks": 0,
  "done": false,
  "id": "ccb04540-7f20-45db-b5f3-87c827c6fcf0",
  "tasks": [
    {
      "id": "8980945e-30cd-4ce5-b15d-10cdab09c3e1",
      "validatorName": "typeValidation",
      "objectType": "Human",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "e42aa77f-8d6f-40e2-9448-b98d406d08d8",
      "validatorName": "scalarPropertyMismatch",
      "objectType": "Human",
      "propertyName": "type",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "77354f8b-62ea-45fb-bf15-1d84e4b0b2b9",
      "validatorName": "propertyMinCardinality",
      "objectType": "Human",
      "propertyName": "name",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "48ae5ca7-15d0-4fb5-aeb6-e66dfc2c0eaf",
      "validatorName": "propertyMinCardinality",
      "objectType": "Human",
      "propertyName": "rdfs:label",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "00c9e31b-edbb-4ed5-90d6-ed495283ff45",
      "validatorName": "maxCardinalityAdvice",
      "objectType": "Human",
      "propertyName": "rdfs:label",
      "status": "NOT_RUN",
      "queryable": false
      "dataQueryable": false
    },
    {
      "id": "5d512cf2-1b39-48ec-a03b-a9eb8a036e68",
      "validatorName": "propertyMaxCardinality",
      "objectType": "Human",
      "propertyName": "homeworld",
      "status": "NOT_RUN",
      "queryable": true
    },
    {
      "id": "5d1bdef8-73b3-4087-b6c2-23ede93ab194",
      "validatorName": "objectPropertyRangeMismatch",
      "objectType": "Human",
      "propertyName": "homeworld",
      "status": "NOT_RUN",
      "queryable": true
    }
  ]
}

Query Validation Status

GET /soml/{id}/validate
GET /soml/{id}/validate?all=[true | false | failed]
# The following is supported from Semantic Objects 4.1
GET /soml/{id}/validate?all=[true | false | failed] &dataLimit=[int : 0]? &dataOffset=[int : 0]?

Example:

To fetch only the validation summary, send the following request:

curl "http://localhost:9995/soml/swapi-validations/validate"

The response will be:

{
  "@context": {
    "@vocab": "http://ontotext.com/ontology/validation/",
    "@base": "http://data.ontotext.com/",
    "xsd": "http://www.w3.org/2001/XMLSchema#",
    "hydra": "http://www.w3.org/ns/hydra/core#"
  },
  "@type": ":Validation",
  "schemaId": "/soml/swapi-validations",
  "repository": null,
  "createdOn": "2021-09-08T20:51:46.060+03:00",
  "completedOn": "2021-09-08T20:51:46.372+03:00",
  "totalTasks": 12,
  "remainingTasks": 0,
  "infoTasks": 1,
  "warningTasks": 1,
  "failedTasks": 0,
  "done": true,
  "id": "3ed31fd6-a740-4bd3-9572-86ba2e415bf0"
}

The optional argument all=true queries the individual tasks groups:

curl "http://localhost:9995/soml/swapi-validations/validate?all=true"

And the response would be:

{
  "@context": {
    "@vocab": "http://ontotext.com/ontology/validation/",
    "@base": "http://data.ontotext.com/",
    "xsd": "http://www.w3.org/2001/XMLSchema#",
    "hydra": "http://www.w3.org/ns/hydra/core#"
  },
  "@type": "ValidationReport",
  "schemaId": "/soml/swapi-validations",
  "createdOn": "2021-09-08T20:51:46.060+03:00",
  "completedOn": "2021-09-08T20:51:46.372+03:00",
  "totalTasks": 12,
  "remainingTasks": 0,
  "infoTasks": 1,
  "warningTasks": 1,
  "failedTasks": 0,
  "done": true,
  "id": "3ed31fd6-a740-4bd3-9572-86ba2e415bf0",
  "tasks": [
    {
      "id": "8791a2fb-604e-4140-8dd8-b11bd1a8ec52",
      "validatorName": "typeValidation",
      "status": "SUCCESS",
      "queryable": true,
      "durationInMs": 7
    },
    {
      "id": "2415dc08-4057-4973-8bea-91317b24472c",
      "validatorName": "scalarPropertyMismatch",
      "objectType": "Human",
      "propertyName": "type",
      "status": "SUCCESS",
      "queryable": true,
      "durationInMs": 7
    },
    {
      "id": "570d37cd-5262-4546-baa8-b8dc2ab20a05",
      "validatorName": "propertyMinCardinality",
      "objectType": "Human",
      "propertyName": "name",
      "status": "SUCCESS",
      "queryable": true,
      "durationInMs": 7
    },
    {
      "id": "08cc5eb8-cf3b-41cd-a11f-4ede52d78d5d",
      "validatorName": "propertyMinCardinality",
      "objectType": "Human",
      "propertyName": "rdfs:label",
      "status": "SUCCESS",
      "queryable": true,
      "durationInMs": 7
    },
    {
      "id": "d7523760-38aa-495b-abfc-63f95289c736",
      "validatorName": "maxCardinalityAdvice",
      "objectType": "Human",
      "propertyName": "rdfs:label",
      "status": "INFO",
      "queryable": false,
      "durationInMs": 5,
      "result": [
        "The property 'Human.rdfs:label' is defined as 'max: inf', but could not find an object with multiple values"
      ]
    },
    {
      "id": "e7922d7d-e8de-40e9-9d17-6ec264e5fa9f",
      "validatorName": "propertyMaxCardinality",
      "objectType": "Human",
      "propertyName": "homeworld",
      "status": "SUCCESS",
      "queryable": true,
      "durationInMs": 6
    },
    {
      "id": "46140d74-a8f6-41eb-a8bf-cb2cf34836ab",
      "validatorName": "objectPropertyRangeMismatch",
      "objectType": "Human",
      "propertyName": "homeworld",
      "status": "WARN",
      "queryable": true,
      "durationInMs": 13,
      "result": [
        "The property 'Human.homeworld' is defined as 'range: Planet', but found 1 instance of values with types: 'iri' and 1 instance of 'xsd:string'"
      ]
    },
    {
      "id": "e2664435-43b2-421d-9c63-77f0c5dc398a",
      "validatorName": "scalarPropertyMismatch",
      "objectType": "Planet",
      "propertyName": "type",
      "status": "SUCCESS",
      "queryable": true,
      "durationInMs": 6
    },
    {
      "id": "cc862808-199a-405f-8247-04b3fff06358",
      "validatorName": "propertyMinCardinality",
      "objectType": "Planet",
      "propertyName": "name",
      "status": "SUCCESS",
      "queryable": true,
      "durationInMs": 5
    },
    {
      "id": "4a2a79f5-3755-4124-9aaa-3d58da7fc490",
      "validatorName": "propertyMaxCardinality",
      "objectType": "Planet",
      "propertyName": "rdfs:label",
      "status": "SUCCESS",
      "queryable": true,
      "durationInMs": 5
    },
    {
      "id": "b010696d-39e6-48d2-bef9-a52ca483b66c",
      "validatorName": "propertyMinCardinality",
      "objectType": "Planet",
      "propertyName": "rdfs:label",
      "status": "SUCCESS",
      "queryable": true,
      "durationInMs": 5
    },
    {
      "id": "53166e54-ba26-49ef-9ae5-da729d449652",
      "validatorName": "scalarPropertyMismatch",
      "objectType": "Planet",
      "propertyName": "rdfs:label",
      "status": "SUCCESS",
      "queryable": true,
      "durationInMs": 5
    }
  ]
}

And if set to all=failed:

curl "http://localhost:9995/soml/swapi-validations/validate?all=failed"

It will result in:

{
  "@context": {
    "@vocab": "http://ontotext.com/ontology/validation/",
    "@base": "http://data.ontotext.com/",
    "xsd": "http://www.w3.org/2001/XMLSchema#",
    "hydra": "http://www.w3.org/ns/hydra/core#"
  },
  "@type": "ValidationReport",
  "schemaId": "/soml/swapi-validations",
  "repository": null,
  "createdOn": "2021-09-08T20:51:46.060+03:00",
  "completedOn": "2021-09-08T20:51:46.372+03:00",
  "totalTasks": 12,
  "remainingTasks": 0,
  "infoTasks": 1,
  "warningTasks": 1,
  "failedTasks": 0,
  "done": true,
  "id": "3ed31fd6-a740-4bd3-9572-86ba2e415bf0",
  "tasks": [
    {
      "id": "d7523760-38aa-495b-abfc-63f95289c736",
      "validatorName": "maxCardinalityAdvice",
      "objectType": "Human",
      "propertyName": "rdfs:label",
      "status": "INFO",
      "queryable": false,
      "durationInMs": 5,
      "result": [
        "The property 'Human.rdfs:label' is defined as 'max: inf', but could not find an object with multiple values"
      ]
    },
    {
      "id": "46140d74-a8f6-41eb-a8bf-cb2cf34836ab",
      "validatorName": "objectPropertyRangeMismatch",
      "objectType": "Human",
      "propertyName": "homeworld",
      "status": "WARN",
      "queryable": true,
      "durationInMs": 13,
      "result": [
        "The property 'Human.homeworld' is defined as 'range: Planet', but found 1 instance of values with types: 'iri' and 1 instance of 'xsd:string'"
      ]
    }
  ]
}

Since Semantic Objects version 4.1, the validation response could also include the violations data. Add the argument dataLimit with value greater than 0 to the request to fetch the validation information:

Note

In case of security concerns, data loading could be disabled globally by setting the configuration soml.validation.disableViolatingDataReporting to true. For more information refer to the security section below.

curl -X POST "http://localhost:9995/soml/swapi-validations/validate?property=rdf*&property=habitable"
curl "http://localhost:9995/soml/swapi-validations/validate?all=failed&dataLimit=100"
{
    "@context": {
        "@vocab": "http://ontotext.com/ontology/validation/",
        "@base": "http://data.ontotext.com/",
        "xsd": "http://www.w3.org/2001/XMLSchema#",
        "hydra": "http://www.w3.org/ns/hydra/core#"
    },
    "@type": "ValidationReport",
    "schemaId": "/soml/swapi-validations",
    "createdOn": "2023-08-16T17:37:08.503+03:00",
    "completedOn": "2023-08-16T17:37:08.811+03:00",
    "totalTasks": 9,
    "remainingTasks": 0,
    "infoTasks": 0,
    "warningTasks": 2,
    "failedTasks": 0,
    "done": true,
    "id": "4ba74043-f4f0-4747-b998-801af055e115",
    "tasks": [
        {
            "id": "33a4313c-afdb-4c66-99fe-e10746f850fb",
            "validatorName": "propertyMaxCardinality",
            "objectType": "Human",
            "propertyName": "rdfs:label",
            "status": "WARN",
            "queryable": true,
            "durationInMs": 8,
            "result": [
                "The property 'Human.rdfs:label' is defined as 'max: 1', but found multiple values for one or more objects"
            ],
            "violations": [
                {
                    "id": "https://swapi.co/resource/human/69",
                    "values": [
                        {
                            "value": "Jango Fett"
                        },
                        {
                            "value": "\"Jango Fett\"@en"
                        },
                        {
                            "value": "\"Джанго Фет\"@bg"
                        },
                        {
                            "value": "\"ジャンゴ・フェット\"@jp"
                        }
                    ]
                }
            ]
        },
        {
            "id": "6274822c-69c9-442c-8b5a-668925923023",
            "validatorName": "propertyMinCardinality",
            "objectType": "Planet",
            "propertyName": "habitable",
            "status": "WARN",
            "queryable": true,
            "dataQueryable": true,
            "durationInMs": 4,
            "result": [
                "The property 'Planet.habitable' is defined as 'nonNullable: true', but some objects do not have any values for this property"
            ],
            "violations": [
                "https://swapi.co/resource/planet/25",
                "https://swapi.co/resource/planet/13",
                "https://swapi.co/resource/planet/15",
                "https://swapi.co/resource/planet/16",
                "https://swapi.co/resource/planet/17",
                "https://swapi.co/resource/planet/19",
                "https://swapi.co/resource/planet/20",
                "https://swapi.co/resource/planet/21",
                "https://swapi.co/resource/planet/26",
                "https://swapi.co/resource/planet/27",
                "https://swapi.co/resource/planet/3",
                "https://swapi.co/resource/planet/30",
                "https://swapi.co/resource/planet/32",
                "https://swapi.co/resource/planet/36",
                "https://swapi.co/resource/planet/4",
                "https://swapi.co/resource/planet/42",
                "https://swapi.co/resource/planet/46",
                "https://swapi.co/resource/planet/53",
                "https://swapi.co/resource/planet/61",
                "https://swapi.co/resource/planet/38",
                "https://swapi.co/resource/planet/55",
                "https://swapi.co/resource/planet/43",
                "https://swapi.co/resource/planet/50",
                "https://swapi.co/resource/planet/54",
                "https://swapi.co/resource/planet/35",
                "https://swapi.co/resource/planet/7",
                "https://swapi.co/resource/planet/11",
                "https://swapi.co/resource/planet/24",
                "https://swapi.co/resource/planet/47",
                "https://swapi.co/resource/planet/59",
                "https://swapi.co/resource/planet/49",
                "https://swapi.co/resource/planet/31",
                "https://swapi.co/resource/planet/57",
                "https://swapi.co/resource/planet/44",
                "https://swapi.co/resource/planet/18",
                "https://swapi.co/resource/planet/12",
                "https://swapi.co/resource/planet/48",
                "https://swapi.co/resource/planet/23",
                "https://swapi.co/resource/planet/56",
                "https://swapi.co/resource/planet/33",
                "https://swapi.co/resource/planet/58",
                "https://swapi.co/resource/planet/41",
                "https://swapi.co/resource/planet/34",
                "https://swapi.co/resource/planet/29",
                "https://swapi.co/resource/planet/60",
                "https://swapi.co/resource/planet/39",
                "https://swapi.co/resource/planet/40",
                "https://swapi.co/resource/planet/28",
                "https://swapi.co/resource/planet/8",
                "https://swapi.co/resource/planet/22",
                "https://swapi.co/resource/planet/45",
                "https://swapi.co/resource/planet/6",
                "https://swapi.co/resource/planet/10",
                "https://swapi.co/resource/planet/51",
                "https://swapi.co/resource/planet/37",
                "https://swapi.co/resource/planet/14",
                "https://swapi.co/resource/planet/1",
                "https://swapi.co/resource/planet/9",
                "https://swapi.co/resource/planet/5",
                "https://swapi.co/resource/planet/2",
                "https://swapi.co/resource/planet/17777"
            ]
        }
    ]
}

Stop Active Validation

POST /soml/{id}/validate?cancel=true

Example:

curl -X POST "http://localhost:9995/soml/swapi-validations/validate?cancel=true"

The request will stop the running validation for the given schema and will do nothing if the validation has been complete.

Get Validation Query

GET /soml/{id}/validate/{taskId}

Some of the validations allow visualizing the conflicting data in the GraphDB Workbench. The response of this request is a link that can be opened in a browser and will load the GraphDB Workbench with a query that can be executed to visualize the conflicting data.

The taskId argument is the id key for the particular task.

Example:

curl "http://localhost:9995/soml/swapi-validations/validate/46140d74-a8f6-41eb-a8bf-cb2cf34836ab"

Here is an example response:

http://localhost:9998/sparql?name=Human+homeworld+%287%29&infer=true&sameAs=true&query=base+%3Chttps%3A%2F%2Fswapi.co%2Fresource%2F%3E%0
Aprefix+rdf%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22-rdf-syntax-ns%23%3E%0Aprefix+voc%3A+%3Chttps%3A%2F%2Fswapi.co%2Fvocabulary%2F
%3E%0A%0ASELECT%0A++++%3Fsubject%0A++++%3Fvalue%0A++++%3Ftypename%0AWHERE+%7B%0A++++%3Fsubject+rdf%3Atype+%3Fsubject_rdf_type__0+.%0A+++
+FILTER+%28%3Fsubject_rdf_type__0+%3D+voc%3AHuman%29%0A++++%3Fsubject+voc%3Ahomeworld+%3Fvalue+.%0A++++%7B%0A++++++++FILTER+%28isIri%28%
3Fvalue%29%29%0A++++++++FILTER+%28%21EXISTS+%7B%0A++++++++++++%7B%0A++++++++++++++++%7B%0A++++++++++++++++++++%3Fvalue+rdf%3Atype+%3Fval
ue_rdf_type__0+.%0A++++++++++++++++++++FILTER+%28%3Fvalue_rdf_type__0+%3D+voc%3APlanet%29%0A++++++++++++++++++++BIND+%28%27Planet%27+as+
%3Fvalue_otherType__1%29+.%0A++++++++++++++++%7D+UNION+%7B%0A++++++++++++++++++++%3Fvalue+rdf%3Atype+%3Fvalue_rdf_type__0+.%0A++++++++++
++++++++++FILTER+%28%3Fvalue_rdf_type__0+%3D+voc%3AHuman%29%0A++++++++++++++++++++BIND+%28%27Human%27+as+%3Fvalue_otherType__1%29+.%0A++
++++++++++++++%7D%0A++++++++++++++++FILTER+%28BOUND%28%3Fvalue_otherType__1%29%29%0A++++++++++++%7D%0A++++++++%7D%29%0A++++++++BIND+%28%
27iri%27+as+%3Ftypename%29+.%0A++++%7D+UNION+%7B%0A++++++++FILTER+%28isLiteral%28%3Fvalue%29%29%0A++++++++BIND+%28datatype%28%3Fvalue%29
+as+%3Ftypename%29+.%0A++++%7D+UNION+%7B%0A++++++++FILTER+%28isBlank%28%3Fvalue%29%29%0A++++++++BIND+%28%27blank%27+as+%3Ftypename%29+.%
0A++++%7D+UNION+%7B%0A++++++++%3Fvalue+rdf%3Atype+%3Fvalue_rdf_type__0+.%0A++++++++FILTER+%28%3Fvalue_rdf_type__0+%3D+voc%3AHuman%29%0A+
+++++++BIND+%28%27Human%27+as+%3Fvalue_soType__1%29+.%0A++++++++BIND+%28%3Fvalue_soType__1+as+%3Ftypename%29+.%0A++++%7D%0A%7D%0ALIMIT+100%0A

After opening it in a browser, it should have the query pre-populated:

../_images/validation-query.png

Immediate Validation

GET /soml/{id}/validateNow?type=[SO type]* &property=[properties filter]* &validation=[validations]* &repository=[target repository]? &all=[true | failed]? &dataLimit=[int : 1000]? & dataOffset=[int : 0]? &timeout=[seconds]?

Semantic Objects 4.1 introduces the ability to perform a validation and for the response from the validation to be returned in the same response. When the REST endpoint is called, the request will be blocked until the validation is complete. In comparison with the non blocking validation above, the information for this validation is not preserved and will be forgotten when the response is serialized to the caller.

The endpoint supports:

  • Perform validation to one, multiple or all of the types in the given SOML schema. For example, type=Character will validate all Character sub types while type=Human&type=Droid will validate only the specified types.
  • Perform validation to one, multiple or all properties of the selected types. The property selection supports exact matching of the property names as defined in the SOML, or their representation in the GraphQL schema. There also is a support for wildcard matching using *. For example, rdf* will match the properties rdfs:label and rdf:type.
  • Perform only specific validations. The supported values are propertyMaxCardinality, propertyMinCardinality, objectPropertyRangeMismatch, scalarPropertyMismatch, maxCardinalityAdvice and typeValidation. For more information about each validation refer to their description here .
  • Specify the repository against which the validation should be performed.
  • Specify if the response should include all tasks or exclude the successful ones and return only the ones that have failed.
  • Load the conflicting data for the validation tasks that has failed. This is enabled by default and will return the first 1000 rows. This could be disabled by setting the dataLimit=0. Note that result may not include the same number IRIs as the limit value. If the database contains more violations, the rest of the pages could be queried by specifying an offset value in rows. For example, the next default page could be loaded with the argument dataOffset=1000.
  • Support time limit for the request. The default is 300 seconds (5 minutes) and this could be changed to 1 minute with timeout=60.

Note

In case of security concerns, data loading could be disabled globally by setting the configuration soml.validation.disableViolatingDataReporting to true. For more information refer to the security section below.

curl "http://localhost:9995/soml/swapi-validations/validateNow?all=failed&dataLimit=100&property=rdf*&property=habitable&validation=propertyMaxCardinality"
{
    "@context": {
        "@vocab": "http://ontotext.com/ontology/validation/",
        "@base": "http://data.ontotext.com/",
        "xsd": "http://www.w3.org/2001/XMLSchema#",
        "hydra": "http://www.w3.org/ns/hydra/core#"
    },
    "@type": "ValidationReport",
    "schemaId": "/soml/swapi-validations",
    "createdOn": "2023-08-17T14:39:11.286+03:00",
    "completedOn": "2023-08-17T14:39:11.313+03:00",
    "totalTasks": 3,
    "remainingTasks": 0,
    "infoTasks": 0,
    "warningTasks": 1,
    "failedTasks": 0,
    "done": true,
    "id": "ea0103a5-72d4-47b5-a3e2-887a1fe8f81b",
    "tasks": [
        {
            "id": "ac6eae0e-091e-4a5a-b3eb-c57d5054e63d",
            "validatorName": "propertyMaxCardinality",
            "objectType": "Human",
            "propertyName": "rdfs:label",
            "status": "WARN",
            "queryable": true,
            "durationInMs": 9,
            "result": [
                "The property 'Human.rdfs:label' is defined as 'max: 1', but found multiple values for one or more objects"
            ],
            "violations": [
                {
                    "id": "https://swapi.co/resource/human/69",
                    "values": [
                        {
                            "value": "Jango Fett"
                        },
                        {
                            "value": "\"Jango Fett\"@en"
                        },
                        {
                            "value": "\"Джанго Фет\"@bg"
                        },
                        {
                            "value": "\"ジャンゴ・フェット\"@jp"
                        }
                    ]
                }
            ]
        }
    ]
}

Validation API Security

If the Semantic Objects security is enabled, only authenticated users will be able to access the Validation API.

The only requirement is that the authenticated user must have read access to the schema being validated in the Schema Management API.

Semantic Objects 4.1 introduces the ability to retrieve the violated data for the failing validations in the response. It is important to be aware that the security rules within the RBAC section of the schema are not applied and the data will be returned unfiltered. In order to prevent a leak of unauthorized data by a production deployment, the validation data loading could be disabled by setting soml.validation.disableViolatingDataReporting to true. When disabled, the validation response will include a warning message that violations data loading is disabled:

{
    "@context": {
        "@vocab": "http://ontotext.com/ontology/validation/",
        "@base": "http://data.ontotext.com/",
        "xsd": "http://www.w3.org/2001/XMLSchema#",
        "hydra": "http://www.w3.org/ns/hydra/core#"
    },
    "@type": "ValidationReport",
    "schemaId": "/soml/swapi-validations",
    "createdOn": "2023-08-17T14:39:11.286+03:00",
    "completedOn": "2023-08-17T14:39:11.313+03:00",
    "totalTasks": 3,
    "remainingTasks": 0,
    "infoTasks": 0,
    "warningTasks": 1,
    "failedTasks": 0,
    "done": true,
    "id": "ea0103a5-72d4-47b5-a3e2-887a1fe8f81b",
    "tasks": [
        {
            "id": "ac6eae0e-091e-4a5a-b3eb-c57d5054e63d",
            "validatorName": "propertyMaxCardinality",
            "objectType": "Human",
            "propertyName": "rdfs:label",
            "status": "WARN",
            "queryable": true,
            "durationInMs": 9,
            "result": [
                "The property 'Human.rdfs:label' is defined as 'max: 1', but found multiple values for one or more objects",
                "Violations data reporting is disabled!"
            ],
        }
    ]
}