Skip to content
Snippets Groups Projects
Commit fd422f12 authored by ml bonhomme's avatar ml bonhomme :bee:
Browse files

Update documentation after removal of EntityLink and EntityRole models

parent a69aa0b7
No related branches found
No related tags found
1 merge request!111Update documentation after removal of EntityLink and EntityRole models
Pipeline #168856 passed
Showing
with 60 additions and 109 deletions
......@@ -156,7 +156,7 @@ lists and shorter sentences.
> A machine learning worker is a Docker image embedding any script, generally
> Python code, that gets automatically built upon pushing changes to a GitLab
> repository and can then be used to process elements and produce other
> elements, transcriptions, classifications, entities, entity links, etc.
> elements, transcriptions, classifications, entities, metadata, etc.
This sentence can be broken down in both multiple sentences and a list:
......@@ -169,7 +169,7 @@ This sentence can be broken down in both multiple sentences and a list:
> * Transcriptions
> * Classifications
> * Entities
> * Entity links
> * Metadata
Shorter sentences have a grammar that is easier to parse, and lists are more
visible and concise.
......
......@@ -33,7 +33,7 @@ This section will explain in details the different parts of content management i
- [Metadatas](@/project/metadata.md) allow you to describe more precisely the elements.
- [Classifications](@/project/classification/index.md) are used to apply classes on an element.
- [Transcriptions](@/project/transcription.md) represent the extracted text from your documents.
- [Entities](@/project/entity.md) are named and known entities linked to your elements.
- [Entities](@/project/entity/index.md) are named and known entities linked to your elements.
- [Exports](@/project/export/index.md) can be generated to retrieve all of the project's content at once.
To learn how to export, see [How to export a project](@/howto/export/index.md).
......
content/project/entities-panel.png

19.9 KiB

content/project/entity-approved.png

35.4 KiB

content/project/entity-details.png

43.3 KiB

content/project/entity-links.jpg

14.9 KiB

content/project/entity-models.jpg

21.4 KiB

content/project/entity-transcriptions.png

905 KiB

+++
title = "Entity"
weight = 50
+++
A **Named Entity** represents a real-world object inside Arkindex. It's linked to one or more `Element` through a `MetaData` or a `Transcription`.
## Rules
- An `Entity` must belong to a pre-defined set of type:
- `Person`
- `Location`
- `Organization`
- `Number`
- `Date`
- `Subject` (as in subject-matter: History, Architecture, ...)
- `Misc` (should be restricted as a fallback option)
- An `Entity` should be unique per project
- but we cannot enforce this at the database level, as we could have two different persons with the same name in a project.
- An `Entity` may be linked to another entity through an `EntityLink`:
- this link establishes a hierarchy between a parent entity and a child entity
- An `EntityLink` must have a role defined through`EntityRole`
- this is important to know which type of relation is defined between the two entities
- An `Entity` may be linked to an `Element` through a `MetaData`
- For example, this is used to share a common location for several books
- An `Entity` may be linked to an `Element` through a `Transcription`
- A single `Transcription` may have several `Entity` (for example 2 persons and a location)
- A `TranscriptionEntity` is used to specify precisely where in the transcription text is situated the entity
- A `TranscriptionEntity` can be moderated by a human annotator through the web interface
{{ figure(image="project/entity-models.jpg", height=300, caption="Relations between entities and other parts of a Project") }}
### Links between entities
A link between two entities can be created through an `EntityLink`, by specifying
- which `Entity` is the parent.
- which `Entity` is the child.
- which `EntityRole` links them together.
For example, to create "Mother - Son" relationship, you'll need:
- an `EntityRole` named "Mother - Son" (the name is just for display purposes)
- An `Entity` of type `Person`, named "Alice"
- An `Entity` of type `Person`, named "George"
Then you'll be able to link "Alice" and "George", using the "Mother - Son" relationship.
An `EntityLink` can be used to link entities of different types:
- an Organisation and a Person: this may be an "Employee" relationship
- a Date and a Location: this would become an Event
{{ figure(image="project/entity-links.jpg", height=250, caption="This system allows us to modelise any Entity structure as a graph") }}
## Web interface
Arkindex's web interface enables you to:
- View each entity and their usage on an element.
- View relations between entities.
Contributors on a project are allowed to:
- Validate or Reject entities usage on a transcription.
{{ figure(image="project/entity-transcriptions.png", height=450, caption="Entities appear clearly on a transcription, with their type and approval") }}
{{ figure(image="project/entity-approved.png", height=300, caption="You can validate entities positions and pertinence on a Transcription") }}
Below on the same panel, you can found the list of entities attached to this element and their relations.
{{ figures(images=["project/entities-panel.png", "project/roles-panel.png"], height=300, caption="Entities list and roles panel on the details panel") }}
By clicking on an entity you can access its dedicated page, which contains its details, relations and links to elements.
{{ figure(image="project/entity-details.png", height=400, caption="Entity details page") }}
## API Endpoints
These endpoints are the most useful to handle entities:
- [CreateEntity](https://demo.arkindex.org/api-docs/#operation/CreateEntity)
- [ListEntityElements](https://demo.arkindex.org/api-docs/#operation/ListEntityElements)
- [ListTranscriptionEntities](https://demo.arkindex.org/api-docs/#operation/ListTranscriptionEntities)
- [CreateEntityRole](https://demo.arkindex.org/api-docs/#operation/CreateEntityRole)
- [CreateEntityLink](https://demo.arkindex.org/api-docs/#operation/CreateEntityLink)
content/project/entity/entity_details.png

783 KiB

<svg aria-roledescription="flowchart-v2" role="graphics-document document" viewBox="-8 -8 424.83331298828125 322" style="max-width: 100%;" xmlns="http://www.w3.org/2000/svg" width="100%" id="graph-div" height="100%" xmlns:xlink="http://www.w3.org/1999/xlink"><style>@import url("https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.5.1/css/all.min.css");'</style><style>#graph-div{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}#graph-div .error-icon{fill:#552222;}#graph-div .error-text{fill:#552222;stroke:#552222;}#graph-div .edge-thickness-normal{stroke-width:2px;}#graph-div .edge-thickness-thick{stroke-width:3.5px;}#graph-div .edge-pattern-solid{stroke-dasharray:0;}#graph-div .edge-pattern-dashed{stroke-dasharray:3;}#graph-div .edge-pattern-dotted{stroke-dasharray:2;}#graph-div .marker{fill:#333333;stroke:#333333;}#graph-div .marker.cross{stroke:#333333;}#graph-div svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#graph-div .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#graph-div .cluster-label text{fill:#333;}#graph-div .cluster-label span,#graph-div p{color:#333;}#graph-div .label text,#graph-div span,#graph-div p{fill:#333;color:#333;}#graph-div .node rect,#graph-div .node circle,#graph-div .node ellipse,#graph-div .node polygon,#graph-div .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#graph-div .flowchart-label text{text-anchor:middle;}#graph-div .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#graph-div .node .label{text-align:center;}#graph-div .node.clickable{cursor:pointer;}#graph-div .arrowheadPath{fill:#333333;}#graph-div .edgePath .path{stroke:#333333;stroke-width:2.0px;}#graph-div .flowchart-link{stroke:#333333;fill:none;}#graph-div .edgeLabel{background-color:#e8e8e8;text-align:center;}#graph-div .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#graph-div .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#graph-div .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#graph-div .cluster text{fill:#333;}#graph-div .cluster span,#graph-div p{color:#333;}#graph-div div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#graph-div .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#graph-div :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}</style><g><marker orient="auto" markerHeight="12" markerWidth="12" markerUnits="userSpaceOnUse" refY="5" refX="6" viewBox="0 0 10 10" class="marker flowchart" id="graph-div_flowchart-pointEnd"><path style="stroke-width: 1px; stroke-dasharray: 1px, 0px;" class="arrowMarkerPath" d="M 0 0 L 10 5 L 0 10 z"></path></marker><marker orient="auto" markerHeight="12" markerWidth="12" markerUnits="userSpaceOnUse" refY="5" refX="4.5" viewBox="0 0 10 10" class="marker flowchart" id="graph-div_flowchart-pointStart"><path style="stroke-width: 1px; stroke-dasharray: 1px, 0px;" class="arrowMarkerPath" d="M 0 5 L 10 10 L 10 0 z"></path></marker><marker orient="auto" markerHeight="11" markerWidth="11" markerUnits="userSpaceOnUse" refY="5" refX="11" viewBox="0 0 10 10" class="marker flowchart" id="graph-div_flowchart-circleEnd"><circle style="stroke-width: 1px; stroke-dasharray: 1px, 0px;" class="arrowMarkerPath" r="5" cy="5" cx="5"></circle></marker><marker orient="auto" markerHeight="11" markerWidth="11" markerUnits="userSpaceOnUse" refY="5" refX="-1" viewBox="0 0 10 10" class="marker flowchart" id="graph-div_flowchart-circleStart"><circle style="stroke-width: 1px; stroke-dasharray: 1px, 0px;" class="arrowMarkerPath" r="5" cy="5" cx="5"></circle></marker><marker orient="auto" markerHeight="11" markerWidth="11" markerUnits="userSpaceOnUse" refY="5.2" refX="12" viewBox="0 0 11 11" class="marker cross flowchart" id="graph-div_flowchart-crossEnd"><path style="stroke-width: 2px; stroke-dasharray: 1px, 0px;" class="arrowMarkerPath" d="M 1,1 l 9,9 M 10,1 l -9,9"></path></marker><marker orient="auto" markerHeight="11" markerWidth="11" markerUnits="userSpaceOnUse" refY="5.2" refX="-1" viewBox="0 0 11 11" class="marker cross flowchart" id="graph-div_flowchart-crossStart"><path style="stroke-width: 2px; stroke-dasharray: 1px, 0px;" class="arrowMarkerPath" d="M 1,1 l 9,9 M 10,1 l -9,9"></path></marker><g class="root"><g class="clusters"></g><g class="edgePaths"><path marker-end="url(#graph-div_flowchart-pointEnd)" style="fill:none;" class="edge-thickness-normal edge-pattern-solid flowchart-link LS-Entity LE-EntityType" id="L-Entity-EntityType-0" d="M35.178,133.5L42.651,114.5C50.124,95.5,65.07,57.5,80.595,38.5C96.119,19.5,112.222,19.5,120.274,19.5L128.325,19.5"></path><path marker-end="url(#graph-div_flowchart-pointEnd)" style="fill:none;" class="edge-thickness-normal edge-pattern-solid flowchart-link LS-Entity LE-Project" id="L-Entity-Project-0" d="M50.518,133.5L55.434,129.333C60.351,125.167,70.184,116.833,85.227,112.667C100.269,108.5,120.522,108.5,130.649,108.5L140.775,108.5"></path><path marker-end="url(#graph-div_flowchart-pointEnd)" style="fill:none;" class="edge-thickness-normal edge-pattern-solid flowchart-link LS-Entity LE-TranscriptionEntity" id="L-Entity-TranscriptionEntity-0" d="M50.518,172.5L55.434,176.667C60.351,180.833,70.184,189.167,78.383,193.333C86.583,197.5,93.15,197.5,96.433,197.5L99.717,197.5"></path><path marker-end="url(#graph-div_flowchart-pointEnd)" style="fill:none;" class="edge-thickness-normal edge-pattern-solid flowchart-link LS-TranscriptionEntity LE-Transcription" id="L-TranscriptionEntity-Transcription-0" d="M251.933,197.5L256.1,197.5C260.267,197.5,268.6,197.5,276.05,197.5C283.5,197.5,290.067,197.5,293.35,197.5L296.633,197.5"></path><path marker-end="url(#graph-div_flowchart-pointEnd)" style="fill:none;" class="edge-thickness-normal edge-pattern-solid flowchart-link LS-Entity LE-MetaData" id="L-Entity-MetaData-0" d="M35.178,172.5L42.651,191.5C50.124,210.5,65.07,248.5,81.038,267.5C97.006,286.5,113.994,286.5,122.489,286.5L130.983,286.5"></path></g><g class="edgeLabels"><g class="edgeLabel"><g transform="translate(0, 0)" class="label"><foreignObject height="0" width="0"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel"><g transform="translate(0, 0)" class="label"><foreignObject height="0" width="0"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel"><g transform="translate(0, 0)" class="label"><foreignObject height="0" width="0"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel"><g transform="translate(0, 0)" class="label"><foreignObject height="0" width="0"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel"><g transform="translate(0, 0)" class="label"><foreignObject height="0" width="0"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="edgeLabel"></span></div></foreignObject></g></g></g><g class="nodes"><g transform="translate(27.508331298828125, 153)" data-id="Entity" data-node="true" id="flowchart-Entity-220" class="node default default flowchart-label"><rect height="39" width="55.01666259765625" y="-19.5" x="-27.508331298828125" ry="0" rx="0" style="" class="basic label-container"></rect><g transform="translate(-20.008331298828125, -12)" style="" class="label"><rect></rect><foreignObject height="24" width="40.01666259765625"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="nodeLabel">Entity</span></div></foreignObject></g></g><g transform="translate(178.4749984741211, 19.5)" data-id="EntityType" data-node="true" id="flowchart-EntityType-221" class="node default default flowchart-label"><rect height="39" width="89.69999694824219" y="-19.5" x="-44.849998474121094" ry="0" rx="0" style="" class="basic label-container"></rect><g transform="translate(-37.349998474121094, -12)" style="" class="label"><rect></rect><foreignObject height="24" width="74.69999694824219"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="nodeLabel">EntityType</span></div></foreignObject></g></g><g transform="translate(178.4749984741211, 108.5)" data-id="Project" data-node="true" id="flowchart-Project-223" class="node default default flowchart-label"><rect height="39" width="64.80000305175781" y="-19.5" x="-32.400001525878906" ry="0" rx="0" style="" class="basic label-container"></rect><g transform="translate(-24.900001525878906, -12)" style="" class="label"><rect></rect><foreignObject height="24" width="49.80000305175781"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="nodeLabel">Project</span></div></foreignObject></g></g><g transform="translate(178.4749984741211, 197.5)" data-id="TranscriptionEntity" data-node="true" id="flowchart-TranscriptionEntity-225" class="node default default flowchart-label"><rect height="39" width="146.9166717529297" y="-19.5" x="-73.45833587646484" ry="0" rx="0" style="" class="basic label-container"></rect><g transform="translate(-65.95833587646484, -12)" style="" class="label"><rect></rect><foreignObject height="24" width="131.9166717529297"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="nodeLabel">TranscriptionEntity</span></div></foreignObject></g></g><g transform="translate(355.3833312988281, 197.5)" data-id="Transcription" data-node="true" id="flowchart-Transcription-227" class="node default default flowchart-label"><rect height="39" width="106.89999389648438" y="-19.5" x="-53.44999694824219" ry="0" rx="0" style="" class="basic label-container"></rect><g transform="translate(-45.94999694824219, -12)" style="" class="label"><rect></rect><foreignObject height="24" width="91.89999389648438"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="nodeLabel">Transcription</span></div></foreignObject></g></g><g transform="translate(178.4749984741211, 286.5)" data-id="MetaData" data-node="true" id="flowchart-MetaData-229" class="node default default flowchart-label"><rect height="39" width="84.38333129882812" y="-19.5" x="-42.19166564941406" ry="0" rx="0" style="" class="basic label-container"></rect><g transform="translate(-34.69166564941406, -12)" style="" class="label"><rect></rect><foreignObject height="24" width="69.38333129882812"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; white-space: nowrap;"><span class="nodeLabel">MetaData</span></div></foreignObject></g></g></g></g></g></svg>
\ No newline at end of file
%% Mermaid diagram for the database structure
graph LR
Entity --> EntityType
Entity --> Project
Entity --> TranscriptionEntity
TranscriptionEntity --> Transcription
Entity --> MetaData
\ No newline at end of file
content/project/entity/entity_transcription.png

1.85 MiB

content/project/entity/entity_types.png

109 KiB

+++
title = "Entity"
weight = 50
+++
A **Named Entity** represents a real-world object inside Arkindex. It's linked to one or more `Element` through a `MetaData` or a `Transcription`.
## Rules
- An `Entity` must be of a type belonging to a set of `EntityType` which are defined at project level by project administrators.
{{ figure(image="content/project/entity/entity_types.png", height=400, caption="Entity type management panel in a Project") }}
- An `Entity` should be unique per project
- but we cannot enforce this at the database level, as we could have two different persons with the same name in a project.
- An `Entity` may be linked to an `Element` through a `MetaData`
- For example, this is used to share a common location for several books
- An `Entity` may be linked to an `Element` through a `Transcription`
- A single `Transcription` may have several `Entity` (for example 2 persons and a location)
- A `TranscriptionEntity` is used to specify precisely where in the transcription text is situated the entity
<figure>
![Relationships between an entity and other objects in a Project](./entity_models.svg)
<figcaption>Relationships between an entity and other objects in a Project</figcaption>
</figure>
## Web interface
Using Arkindex's web interface, you can view each entity and their usage on elements.
Contributors on a project are allowed to validate or reject entities usage on a transcription using the [`(Partial)UpdateEntity` API endpoint](https://demo.arkindex.org/api-docs/#tag/entities/operation/PartialUpdateEntity).
{{ figure(image="content/project/entity/entity_transcription.png", height=450, caption="Entities appear clearly on a transcription, with their type") }}
By clicking on an entity in the transcription you can access its dedicated page, which contains its details and links to elements.
{{ figure(image="content/project/entity/entity_details.png", height=450, caption="Entity details page") }}
## API Endpoints
These endpoints are the most useful to handle entities:
- [CreateEntity](https://demo.arkindex.org/api-docs/#operation/CreateEntity)
- [ListEntityElements](https://demo.arkindex.org/api-docs/#operation/ListEntityElements)
- [ListTranscriptionEntities](https://demo.arkindex.org/api-docs/#operation/ListTranscriptionEntities)
......@@ -22,7 +22,7 @@ A [Python library](https://pypi.org/project/arkindex-export/) is available to he
Name | Type | Description
---- | ---- | -----------
version | `INTEGER` | Version number of the database structure. Currently set to `8`.
version | `INTEGER` | Version number of the database structure. Currently set to `9`.
### classification
......@@ -103,25 +103,6 @@ metas | `TEXT` | Arbitrary metadata about the entity, as a JSON object with stri
worker_version_id | `VARCHAR(37)` | UUID of a worker version that created this entity. Nullable.
worker_run_id | `VARCHAR(37)` | UUID of a worker run that created this entity. Nullable.
### entity_link
Name | Type | Description
---- | ---- | -----------
id | `VARCHAR(37)` | UUID of the entity link.
parent_id | `VARCHAR(37)` | UUID of the parent entity.
child_id | `VARCHAR(37)` | UUID of the child entity.
role_id | `VARCHAR(37)` | UUID of the entity role used to link both entities.
### entity_role
Name | Type | Description
---- | ---- | -----------
id | `VARCHAR(37)` | UUID of the entity role.
parent_name | `VARCHAR(250)` | Name of the parent entity in a link, such as "employer".
child_name | `VARCHAR(250)` | Name of the child entity in a link, such as "employee".
parent_type_id | `VARCHAR(37)` | UUID of the entity type of the parent entities that can have this role in a link.
child_type_id | `VARCHAR(37)` | UUID of the entity type of the child entities that can have this role in a link.
### image
Name | Type | Description
......
This diff is collapsed.
......@@ -13,9 +13,6 @@ graph LR
entity --> worker_version
entity --> worker_run
entity --> entity_type
entity_role --> entity_type
entity_link --> entity
entity_link --> entity_role
transcription_entity --> transcription
transcription_entity --> entity
transcription_entity --> worker_version
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment