From Recognition to Understanding - NE for Cultural Heritage.pdf

Presentation at the 2025 Lorentz Workshop: Enriching Digital Heritage with LLMs and Linked Open Data on work carried out in the EU ATRIUM project on NER from archaeological reports

1. From Recognition to Understanding: Framing
Named Entity Recognition for Enriching Cultural
Heritage with LLMs and Linked Open Data
Dr Diana Maynard
University of Sheffield, UK

2. Introduction: NER in Cultural Heritage
●Purpose: Explore how NER, LLMs, and LOD can enrich CH
metadata.
●NER is the foundation for this pipeline: Recognition →
Disambiguation → Relations
●Your expertise: many approaches!

3. Example: A museum catalogue entry
●Not tagged: Mary Stuart, François Clouet, Hamilton
family, Lanarkshireare all entities that appear only as
free text.
●Multilingual: "Mary Stuart" might also appear as Maria
Stuart, Marie Stuart, or Maria Stuarda
●Historically complex:
○Mary Stuartcould refer to multiple historical
identities (Mary Queen of Scots, or other
contemporaneous Mary Stuarts in noble families).
○François Clouetcould be an attribution rather than a
confirmed creator, raising questions of
certainty/provenance.
"Oil on canvas
portrait of Mary
Stuart, Queen of
Scots, painted in the
style of François
Clouet.Acquired
from the Hamilton
familycollection in
Lanarkshire, circa
1850."

4. Why enrich cultural heritage data?
Named Entity Recognition —especially when combined
with LOD and human validation —is foundational for the
digital future of CH collections.

5. Improved Discoverability
●Enrichment: Adding NEs (e.g. artist names, place names, historical
periods) improves keyword search, faceted browsing, and multilingual
access.
●Example: A record saying "painted in Florence" can be linked to the
entity Florence (Italy), allowing users to retrieve all works from that
location even if the original metadata phrasing varies ("Firenze",
"Florenz").

6. Contextual understanding
●Enrichment:Adding relationships between entities (e.g. artist →
artwork → patron) helps users understand historical or artistic
context.
●Example: Linking "The Arnolfini Portrait" not just to Jan van Eyck
(creator), but also to Giovanni di Nicolao Arnolfini(patron)
deepens interpretive value.

7. Authority Control and Data Consistency
●Validation: Matching names to controlled vocabularies (e.g. Getty
ULAN, VIAF) reduces duplication and spelling inconsistency.
●Example: "Rembrandt", "Rembrant van Rijn", "Rijn, R." → all
resolved to a single authoritative ID ensures uniform retrieval and
avoids fragmentation.

8. Global access and reuse
●Enrichment: Linking tomultilingualLOD sources (e.g. Wikidata) enables
users to find records using their native language or regional terms.
○Example: A Japanese user searching for "フェルメール " (Vermeer) can
be served relevant records if entities are linked via LOD with language
labels.
●InteroperabilityStructured, validated metadata allows aggregation across
institutions (e.g. Europeana, DPLA) and integration into platforms like
Wikidata or IIIF.
○Example: A dataset enriched with standard entity URIs can be re-used
by external digital exhibitions or APIs.

9. Enabling Advanced Research through Enrichment
●Enables network analysisand temporal mapping
●Apply NER to historical texts and link extracted place names and dates to
LOD sources (Pleiades, GeoNames, Wikidata…)
●When repeated across many artists, this enriched data allows researchers to
visualise broader mobility patterns, such as:
○Clustering of N. European artists in Italy (Rome, Venice, Florence).
○Shifts over time, e.g., the rise of Antwerp as a hub in the 16th century.
○How political or religious changes (like the Reformation) influenced
artistic relocation.
Example:tracing the migration of Flemish painters to Italy during the Renaissance

10. Ethical Stewardship and Representation
●Validation:Reviewing and updating colonial or biased metadata
ensures more respectful, inclusive representation.
●Example:Changing "primitive tribe" to a recognised community name
linked to a contemporary identifier supports reparative description
practices.

11. Traditional NER Pipelines
●Rule-based (gazetteers, regex) or ML-based (CRFs, BiLSTMs, etc.).
●Usually domain-trained and deterministic.
●Strengths: transparent, stable, leverage existing vocabularies
●Limitations: rigid. language-specific, limited generalisation
●Example: GATE’s ANNIE tool

12. Enter LLMs: a new era for NER?
LLMs bring several innovations to NER:
●Prompt-based learning: few-shot/zero-shot capabilities can
extract entities without explicit training data. Also “one model, any
schema” (can be defined at inference time).
●Multilingual capabilities: one model, many languages
●Contextual embeddingsprovide richer context modelling. Entities
interpreted in light of full sentence/paragraph.
●Transfer learning: Models pre-trained on massive corpora adapt
quickly to new domains.

13. Examples
●Input:"Rembrandt’s Night Watchhangs in the Rijksmuseum."
●Prompt:"Extract named entities from this text and identify their
types."
●Output:
○Rembrandt [Person]
○Night Watch [Work of Art]
○Rijksmuseum [Organisation/Location]

14. Challenges of LLMs
●Hallucinations: Model may infer entities that don’t exist in
text.
●Inconsistency: Different outputs across runs.
●Validation: Harder to verify correctness without gold
standard.
●Context-aware models (e.g. BERT) better handle ambiguity through
embedding similarity, but may require prompt design or fine-tuning.

15. Prompt Engineering and Fine-Tuning
●Prompting: Adjust the question to control model output.
○Prompt 1: "List all people mentioned."
○Prompt 2: "Extract all named entities with type: person, place,
organisation."
●Few-shot examples: Embedding 2–3 example texts with annotated entities
improves accuracy.
●Fine-tuningimproves accuracy and domain alignment
○Training on small, curated CH datasets (e.g. British Library descriptions)
●Tips:
○Use task-specific instruction templates or custom tuning for better control.

16. Challenges for NER in CH data
●Ambiguity: "Victoria" —person, monarch, location, ship?
●Granularity: "Florence" → city vs district vs gallery name
●Bias: Underrepresentation of non-Western names/entities.
●Legacy language: Outdated, colonial terms.
●Multilingualism: Multiple scripts, historical spellings.
●Evaluation:LLMs are exciting but unpredictable: we need
good frameworks for this

17. Bias and Historical Language
●Bias in models:
○Western-centric datasets dominate training data
○LLMs may underperform on underrepresented regions/periods/languages
●Examples:
○"Nzinga" often misclassified or unlinked
○"Bombay" vs "Mumbai" –colonial vs post-colonial naming
●Solutions:
○Custom entity lexicons
○Augment LLMs with regional/historical corpora
○Human-in-the-loop review

18. GeoLocations in ATRIUM: Task and Challenges
Task:extracting new knowledge from archaeology’s “grey literature” using NER to
uncover places, objects, and people in unpublished reports and notes, making
them findable and linkable within cultural heritage infrastructures.
Challenges for geolocation:
●Document types: mixture of new and old/ancient texts with very different
language and style
●Coverage of ancient places in common LOD sets is minimal
●Recognition of ancient (or just unusual) place names by common NE tools is
often quite poor
●Useful data sources (Pleiades, World Historical Gazetteer etc.) are often not
used by existing NER tools

19. GeoLocations in ATRIUM: Approach
Approach: Turn relevant datasets into gazetteers and then disambiguate matches using
a geometrical approach.
Example with Pleiades:
●Create a gazetteer using every name in every entry
○For each name, record lat/lon, location type, and time period
●Find groups of overlapping matches in a document
○The same name may refer to different places, or similar names may overlap etc.
●Disambiguate by picking one Pleiades entry per group, such that we minimise the
area covered by the selected places
○assume a document is likely to talk about a set of places which are physically
close to each other (one area per document, similar to one sense per
discourse)

21. Disambiguation
The army marched from
Alexandria to Memphis,
moving south along the Nile to
Thebes.

22. Other NER tools fail…
ANNIE
YODIE

23. Factoring in the time period
Herodotus describes the battles near Marathonand Thebes, while later Roman
accounts mention Londinium as a rising city of the empire.
Marathon→ site in Greece (490 BCE battle) or a modern town elsewhere
(Florida, Texas, Ontario)
Thebes -Greece or Egypt
Londinium→ clearly Roman Britain, but only makes sense in the Roman period,
not earlier.
A geometry-onlyapproach might cluster Thebes (Egypt) + Marathon (Greece) +
Londinium (Britain) incorrectly.

24. Questions to consider today
●Can LLMs be trusted to improve Cultural Heritage NER?
●What prompting techniques yield better results?
●How do we combine outputs with authority files?
●What validation workflows are needed?
●How do we ensure FAIRness and ethical practices?

25. Conclusions and discussion points
●NER is not just technical —it's cultural.
●Good NER respects context, representation, and provenance.
○What is a “good enough” entity match?
○Where is human-in-the-loop most essential?
○What unseen assumptions are we embedding?