Backend¶

SOML endpoint¶

The /soml endpoint gives access to the Metadata Studio schema. It is described in detail in the Schema Management section of the Semantic Objects.

It is important that the schema that you send as a request body to the POST and PUT requests satisfies certain restrictions. Check out the Metadata Studio schema section for specifics of the schema.

See a configuration example of a schema request body customized for our Knowledge Net use case, which defines a specific simple document class and a Person inline annotation class that links to Wikidata people concepts.

GraphQL endpoint¶

The Metadata Studio Backend exposes a /graphql endpoint that handles the requests based on a GraphQL schema which is dynamically adapted to incoming mutations applied over the base SOML schema.

The /graphql endpoint satisfies the standard GraphQL specification.

See sample GraphQL queries for Document annotations and Inline annotations that fetch Metadata Studio objects.

Troubleshooting endpoints¶

The following endpoints, grouped into a Troubleshooting Controller, provide system and health information:

curl -X GET --header 'Accept: application/json' 'http://{host}:{port}/{context}/__health'

    {
    "status": "OK",
    "healthChecks": [
        {
            "status": "OK",
            "id": "1200",
            "name": "SPARQL checks",
            "type": "sparql",
            "impact": "SPARQL Endpoint operating normally, writable and populated with data.",
            "troubleshooting": "http://omds-service:8080/__trouble",
            "description": "SPARQL Endpoint checks.",
            "message": "SPARQL Endpoint operating normally, writable and populated with data."
        },
        {
            "status": "OK",
            "id": "1300",
            "name": "SOML checks",
            "type": "soml",
            "impact": "SOML bound, service operating normally.",
            "troubleshooting": "http://omds-service:8080/__trouble",
            "description": "SOML checks.",
            "message": "SOML bound, service operating normally."
        },
        {
            "status": "OK",
            "id": "1350",
            "name": "SOML RBAC checks",
            "type": "soml-rbac",
            "impact": "SOML RBAC schema is created, service operating normally.",
            "troubleshooting": "http://omds-service:8080/__trouble",
            "description": "SOML RBAC checks.",
            "message": "SOML RBAC schema is created, service operating normally."
        },
        {
            "status": "OK",
            "id": "1400",
            "name": "Query service",
            "type": "queryService",
            "impact": "Query service operating normally.",
            "troubleshooting": "http://omds-service:8080/__trouble",
            "description": "Query service checks.",
            "message": "Query service operating normally."
        },
        {
            "status": "OK",
            "id": "1500",
            "name": "Mutations service",
            "type": "mutationService",
            "impact": "Mutation service operating normally.",
            "troubleshooting": "http://omds-service:8080/__trouble",
            "description": "Mutation service checks.",
            "message": "Mutation service operating normally."
        }
    ]
}

curl -X GET --header 'Accept: application/json' 'http://{host}:{port}/{context}/__gtg'

{
  "gtg": "OK"
}

{
  "gtg": "UNAVAILABLE"
}

curl -X GET --header 'Accept: application/json' 'http://{host}:{port}/{context}/__about'

{
    "buildDate": "2022-08-03T10:13:25.446Z",
    "description": "The Ontotext Platform is a GraphQL interface for interacting with RDF data. It is used to expose your information in a more human-readable fashion.",
    "version": "3.0.0-SNAPSHOT",
    "documentation": "http://platform.ontotext.com/"
}

curl -X GET 'http://{host}:{port}/{context}/__trouble'

Registration query¶

The registration query is a query that instantiates the GraphDB Text Mining plugin. It specifies the URL to the text mining service, the headers that must be sent during annotation requests, and any specific transformations that should be applied over the annotation response.

For more information, see this example registration query.

Annotation query¶

Upon selection of a specific annotation service for a particular corpus, the Metadata Studio backend splits the documents from the corpus into batches of ten documents. It then sends the documents from each batch to the text mining API service to generate annotations for these documents.

The annotation query defines how the documents should be sent for annotation and how the response should be stored in GraphDB. It is entirely configurable by the user, which makes this process compatible with any third-party services accessible through HTTP, which produce annotations with text position offsets.

For more information, see this example annotation query.

Select¶

The select function determines what documents and annotations should be included in the report. It also includes any annotations metadata features that are required to compare the annotations from the base and target annotation set.

Equation¶

The equation function determines when two annotations are considered the same. An optional set of fields can be chosen for consideration during this comparison for each annotation type. The application supports two modes of comparison:

• Strict: requires that all the annotation type, features, and offsets strictly match.
• Weak: requires that the annotation type and features strictly match, but the offsets of the base annotation and the target annotation can overlap.

Aggregation¶

During aggregation, the following definitions are satisfied:

Let \(D\) be the set of all documents in a given corpus. Then for a given document \(t ∈ D\), we define the following:

• Correct/True Positive (\(TP_t\)): the number of matching annotations between the ground truth and the predictions for the document \(t\).
• Spurious/False Positive (\(FP_t\)): the number of annotations predicted by an annotator that do not match an annotation in the ground truth for the document \(t\).
• Missing/False Negative (\(FN_t\)): the number of annotations in the ground truth that do not match an annotation predicted by an annotator for the document \(t\).

The Aggregation function combines the ratios between the TP, FP, and FN as a single comparable number. The popular measures for the performance of an information extraction algorithm are:

\(Precision_t =\dfrac{TP_t}{TP_t + FP_t}\)

\(Recall_t= \dfrac{TP_t}{TP_t + FN_t}\)

\(F1.0_t = \dfrac{2 \times Precision_t \times Recall_t}{Precision_t + Recall_t}\)

Further, there are two different ways of aggregating these measures for the whole corpus:

• Micro (default): when all annotations from all documents are grouped together and later calculated.
• Macro: when the precision and recall of each document is calculated independently and then the final number is average.

\(Macro\_Precision = \dfrac{\sum_{t \in D} Precision_t}{|D|}\)

\(Macro\_Recall = \dfrac{\sum_{t \in D} Recall_t}{|D|}\)

\(Macro\_F1.0 =\dfrac{2 \times Macro\_Precision * Macro\_Recall}{Macro\_Precision + Macro\_Recall}\)

\(Micro\_Precision = \dfrac{\sum_{t \in D} TP_t}{\sum_{t \in D}(TP_t + FP_t)}\)

\(Micro\_Recall=\dfrac{\sum_{t \in D}TP_t}{\sum_{t \in D}(TP_t + FN_t )}\)

\(Micro\_F1.0 = \dfrac{2 \times Micro\_Precision \times Micro\_Recall}{Micro\_Precision + Micro\_Recall}\)