Technology and research · November 12, 2026 · 3 min read

Knowledge graph LLM extraction vs span models, measured

We measured GLiNER against a language model on 1,524 documents: on company text the language model found relations about twice as reliably, for $0.77 for all documents.

Niclas GregorCo-founder of Scibly

The key points at a glance

A knowledge graph from company text can be read out in two ways: by a small span model such as GLiNER or by a language model (LLM).
On 1,524 documents the language model found relations in company text about twice as reliably as the best span model (F1 0.60 against 0.32), for $0.77 in all.
It gives each relation the sentence that states it, since when it holds and whether it has ended; a span model gives names and pairs only.
Limits: the company texts are written by a language model, and the model that reads today is not the one we measured.

How do you build a knowledge graph from company text?

A knowledge graph keeps track of who on the team works on what, which systems use what and what was decided, so people and agents find it even when it is spread over many emails and meetings. To build one, a model reads each text and writes down which people, projects, systems and companies it names and how they relate. There are two ways to do it: a small span model that marks names in the text and pairs them, or a language model (LLM) that returns names and relations as a structured answer. We measured both on 1,524 documents. On company text the language model found relations about twice as reliably as the best span model (F1 0.60 against 0.32), for $0.77 for all documents.

What is a span model, and what does the language model do?

A span model such as GLiNER is given a list of types, such as Person or Organization, and marks the places in the text that fit them. GLiNER-Relex also finds relations between the marked names. A model like this is small enough for your own server and costs nothing per call.

The language model reads each text in a single call and fills in a fixed form: every name with its type and every relation with the sentence that states it. Here is the same mail as a graph, read once by each:

Jonas WeberCheckout handover

Since October 1 the checkout has run on Adyen instead of Stripe. Mia (payments team) has taken the project over from me.

Span model1 of 3 relations right

usesuses (out of date)member of (wrong)CheckoutAdyenStripeMiaPayments team

Language model5 of 5 right, each with its sentence as proof

uses since Oct 1used until Oct 1takes overhanded over (hinted)member ofCheckoutAdyenStripeMiaJonas (“me”)Payments team

The span model misses that Stripe no longer holds, puts Mia with Adyen and leaves the team on its own. The language model also sees who “me” is, that Stripe no longer holds and which team Mia is in. Point at a node to see its place in the mail; dashed is what is wrong or no longer holds. Example with invented data.

A knowledge graph needs more than pairs: since when does something hold, does it still hold, and in which sentence does it stand? The language model gives that for each relation; a span model gives only the names and pairs.

How did we measure?

Data. 1,024 documents from CorporateBench, made-up mail and calendar items of three companies, each relation marked as stated or only implied. Plus 500 Wikipedia texts from Re-DocRED.
Schema. Every model was given the same types and relations.
Metric. F1, a score between 0 and 1. It is only high when a model finds much of what the text says and is rarely wrong.
Models. Three GLiNER models against deepseek-v4.1-flash with our prompt, measured on October 6, 2026.

What did we find?

Language model
Span model (GLiNER)

RelationsCompany text

0.320.60

NamesCompany text

0.650.80

RelationsWikipedia

0.21

NamesWikipedia

0.69

0.000.250.500.751.00

Language model, all 1,524 documents: $0.77
Span models: no bill per call, only compute time on your own server: $0

Each dot a model, further right is better. On company text the language model scores about twice as high on relations as GLiNER-Relex-large; on Wikipedia the two are almost level. Point at a dot for its model and scores. Our measurement: 1,024 documents from CorporateBench, 500 from Re-DocRED, F1 at the threshold where each model does best.

What is implied. The language model found relations a text only implies almost as well as those it states.
Guessing does not help. Lower the threshold at which GLiNER-Relex reports a relation and it finds nearly as many relations. But then four of its five guesses are wrong.
Wikipedia is different. There GLiNER-Relex comes close to the language model. Its authors measured it on text like this.

Nor does a span model help as a pre-filter that hands the language model the names: the language model finds the names better itself. Scibly uses none, not even as a first step.

Where does the measurement fall short?

CorporateBench is written by a language model. That likely flatters a language model as reader.
Re-DocRED fits only roughly. We mapped its relations roughly onto ours. Its scores rank the models; they do not measure them exactly.
The model has changed. Another language model reads the texts by now. We have not measured it against the span models again.

If your texts may only be read on your own hardware, you could train a small model such as GLiNER-Relex on the language model's answers (distillation). Scibly itself also runs on your own infrastructure on request; more under Security.