Issues with importing textgrind

[MartinKl]

arg, textgrid, sorry. I'm not sure to understand what the 'morph = ["morph-cat", "morph-flex", ...]' formula stand for (what is "annotation" in the importer doc?). The importer complains (but doesn't crash) when either the left side or the right side of the assignation is not toolbox tier(s). I would be reinsured by being able to export back the textgrid from the salt graph with the textgrid exporter. Also it looks like the timestamps of the timeline are not imported. Sorry for this messy report. With better understand about the parameters of the importer I will try to narrow down the problem(s) -- if there are. Thank you very much for any help.

Originally posted by @sylvainloiseau in #447

[MartinKl]

So you are referring to the tier_groups configuration. Let's consider this example:

[import.config.tier_groups]
tok = ["pos", "lemma"]

This means that in the textgrid files the annotation tiers pos and lemma refer to items given in tier tok. This is required to create a graph in which items and annotations have properly aligned boundaries as this is not naturally provided from textgrid files (it's fairly easy to mess things up by microseconds).

Does this explain it? Do you have a minimal example file I could help you with?

Just a side note: Annatto uses graphANNIS internally, a graph database (and model) that replaces Salt. There are basically just nodes and edges. You can find more details (if you like) in the documentation.

[sylvainloiseau]

Thank you very much for your answers.

Yes I'm referring to the tier_groups configuration. After some tests, if I understand correctly, the declaration

tok = ["pos", "lemma"]

Produce nodes of type sDocumentStructure:SToken for each annotation in the tier tok and nodes of type sDocumentStructure:SSpan for each annotations contained in the tiers pos and lemma. Maybe using those terms could be useful in the documentation.

Furthermore, if I create other entries, for instance a morph entry:

tok = ["pos", "lemma"], morph=[morph_pos]

It creates two independant timelines, with their set of sDocumentStructure:SAudioRelation, for linking the two set of tokens (from tok and from morph) to the timeline ; and two sTextualDS with their set of sDocumentStructure:STextualRelation ?

If I can ask some more questions:

• How some timeline difference are dealt with, when for instance annotation in tok, and annotation in pos do not share exactly the same timestamp?

• I understand that the Salt java library is discontinued, and that graphannis now use a graph database rather than a postgresqp data base for the storage of the graph, but isn't still "salt" the name of the data model used by graphannis (an implementation)? For instance there is a saltxml exporter : isn't the salt data model with xml representation? Some good documentation about the salt data model were available with the salt library that seems to be only in the legacy section now (such as https://github.com/korpling/salt/raw/master/gh-site/doc/salt_modelGuide.pdf).

Thank you very much for your help.

[sylvainloiseau]

Correction:

if I create other entries, for instance a morph entry:

tok = ["pos", "lemma"], morph=[morph_pos]

It creates two independant timelines, with their set of sDocumentStructure:SAudioRelation, for linking the two set of tokens (from tok and from morph) to the timeline ; and two sTextualDS with their set of sDocumentStructure:STextualRelation ?

No, it creates one STimeline (with the nodes from tok and morph anchored to it) and two sDocumentStructure:STextualDS (one for tok, the other for morph).

[MartinKl]

Salt is indeed discontinued (the existence of the importer and exporter for saltxml is for supporting some use cases where the Salt representation was necessary, but it's basically like any other format now: just a format) and thus no STimeline, STextualRelation, STextualDS or any other SaltType is created. The only place were Salt is still used is in ANNIS' Visualizers. The visualizers get their input as a SaltGraph which is derived from a graphANNIS graph. The latter is a different and more generic graph model. Nodes have a name and a type (usually node or corpus) and can be annotated using annotation keys (a pair of namespace and name) and a value. Edges, on the other hand, have a component type, a layer (like an edge namespace), and a name and are always directed from source to target node. Edges can hold annotations in the same fashion as nodes can.

There are some ideas/concepts that originate in Salt and are sometimes mimicked in graphANNIS graphs.

As for STokens, there is no such thing in graphANNIS. Nodes can be ordered via edges of component type Ordering. They can have no outgoing Coverage edges, i. e., they are terminal edges in any component of type Coverage that connects nodes that can be considered spans (but again, there is no explicit type span) that are non-terminal nodes in the same component. And nodes can carry an annotation value for annotation key {ns: "annis", name: "tok"}. These three ingredients are not all necessary, but in different situations sufficient conditions for a node to be considered a "token".

So from that a timeline in graphANNIS is usually a set of nodes of type node (in contrast to corpus, which is used for documents and subcorpora), that are ordered by edges in the component { ctype = "Ordering", layer = "annis", name = "" } and each node carries an annotation {ns: "annis", name: "tok"} with the value and if required another annotation {ns: "annis", name: "time"} with a time value (only required for selecting the relevant sections from audio or video files, the ordering itself comes from the ordering component). These empty tokens are then anchor points (or target nodes) for edges of Coverage components from spanning nodes across certain intervals. These spanning nodes can then themselves have their own ordering edges in a component of type Ordering (usually { ctype = "Ordering", layer = "default_ns", name = "{ordering_name}"}. So if you configure the tier groups as you said (and repeated here, I replaced tok for text to avoid confusion):

[[import]]
format = "textgrid"
path = "..."

[import.config.tier_groups]
text = ["pos", "lemma"]
morph = ["morph_pos"]

This will (roughly) have the following consequences:

• Each textgrid interval start and end time of the tiers text and morph form a set of available time values. This set is then sorted in ascending order and for each time value t (with its following time value t') in this set a node of type node is created with annotations {ns: "annis", name: "tok", value: " "} (short here for the sake of brevity) and {ns: "annis", name: "time", value: "{t}-{t'}"}. And the nodes are ordered nodes in { ctype = "Ordering", layer = "annis", name = "" }. Let's refer to this as the timeline in the following.
• The annotations on tiers text and morph are then spans over the timeline's nodes according to the times set in their intervals using edges in component { ctype = "Coverage", layer = "annis", name = "" } targeting the respective timeline node for start and end time. These spans are ordered using edges in component { ctype = "Ordering", layer = "default_ns", name = "text" } and { ctype = "Ordering", layer = "default_ns", name = "morph" } and have annotations {ns: "default_ns", name: "text", value: "{interval_value}"} or {ns: "default_ns", name: "morph", value: "{interval_value}"}, respectively.
• interval tiers pos and lemma also target the timeline in the same way, but the timeline items are derived from the times in the text tier, not from their own time values. The same holds for morph_pos w. r. t. morph. This way you get perfect overlap of item boundaries and annotation boundaries.

If you do not want a tier to lose it's own timestamps, you can easily make it a key in the tier group configuration and "hide" the dependencies between the tiers. You can basically do that with any tier, for the above configuration you could do something like this:

[[import]]
format = "textgrid"
path = "..."

[import.config.tier_groups]
text = []
pos = []
lemma = []
morph = []
morph_pos = []

or, alternatively, not provide a tier grouping, so all timestamps are valid and all tiers are mapped into the graph independently of each other.

On the documentation in general: These things should be stated clearly there, I agree, but as for now the graphANNIS documentation has to serve for this purpose. We are planning to extend the documentation once certain standards are set and all importers create their graphs in the same fashion (just as an example, some importers create orderings like the above, others would use { ctype = "Ordering", layer = "annis", name = "text" } (a different layer name). This is still in conceptual development, thus also the documentation is pending. Providing sample graphs for each importer is also a todo.

Please let me know if you have any further questions.

[sylvainloiseau]

Thank you so much. The explanation you gave here could be included in the documentation of the textgrid importer. When you say:

interval tiers pos and lemma also target the timeline in the same way, but the timeline items are derived from the times in the text tier, not from their own time values.

do you mean that the intervals in, say, pos, have edges towards "timeline node" or have edges towards the annotations in text? In the former case, if annotations in pos and text do not have exactly the same timestamps (since it's not enforced in praat, or in the textgrid format), how is computed the association of pos annotations with text ones?

Thank you very much for the explanations about the new graphannis data model. I was indeed using the saltxml exporter to inspect the result of the conversion. Is there an exporter that produce a sort of official/straighforward serialisation of the graphannis data model?

[MartinKl]

For expecting the graph I generally recommend using the graphml as export format (for keeping results more comparable between graph versions, you should set stable_order = true). This gives you direct feedback on what an importer did with your import data. For visualizations you can also turn towards the visualize graph op, btw.

Regarding your question which nodes are targeted, here comes the tricky part: The standard way of doing it is to always target timeline nodes, but there are some exceptions or let's say there is generally the option to actually target the text nodes. But this is usually not required, as text and, for example, pos nodes only need to target the same Coverage terminals (i. e., timeline nodes) to codify association, i. e. text _=_ pos in ANNIS logic. The given query has the same result for pos nodes targeting text nodes or for targeting the same timeline nodes as the respective text node, as the coverage relation is transitive at evaluation time (which does not hold for other edge components). By targeting the timeline nodes instead of intermediate nodes (such as text), the maximum path length in the Coverage component is shorter. But as I said, both is possible and Salt xml imports might actually use the other approach to ensure a stable result when importing and exporting again, as directly targeting the text nodes has stronger semantics, such as direct association in the sense of an item-annotation-relationship that might be required in certain cases. This could also be achieved by choosing the annotation namespace of the dependent annotations to be the annotation name of the independent annotation/segmentation. In our example that would mean to give pos annotations the namespace text. But this option is not always available and also has the downside of occupying the namespace which might be required for something else (plus semantically charges it).

About the timestamps, let's assume the following minimal example only containing a text and a sentence tier:

times	0       1	1    2	2    3	3          4	4   5	5   6	6     7	7   8
text	this	is	an	example			isn’t	it
sentence	S1						S2

Now imagine you set up the import tier groups as follows:

text = ["sentence"]

As only text is a key in the map, only its interval boundary time values are considered for the timeline to be created, so time values [1, 2, 3, 5, 6, 7, 8] (but not 4!). For each time value in that list a timeline node is created and they are ordered and annotated as mentioned above.

Now, when importing the sentence spans, the question is, where the end of span S1 value comes from, as 4 is not a valid time. This is derived from looking up the non-empty intervals of its associated tier text, which provides [3, 5] as a filled interval, from which 5 is picked, as it provides the maximal span.