Sharing with Europeana: Depositing and publishing 3D datasets for preservation and future access

Presentation given by Kate Fernie (CARARE) as part of a 4CH webinar for the TwinIt campaign organised by the European Commission and Europeana Foundation.

TwinIt! marks a massive expansion of 3D digitisation in the cultural heritage sector. The webinar series organised by the 4CH project cover the digitisation workflow from the initial project planning to archiving and dissemination. In this presentation, Kate Fernie discusses the data that is created in 3D projects and strategies for depositing the data and publishing the resulting 3D models.
Data is created through the 3D workflow with various pieces of equipment and software used at different stages. A 3D dataset will contain various data types from point clouds and meshes, ground control and TPS points used for registering/locating other data data against the real world object, various types of imagery and 3D models with textures. Interoperability between these file types and their metadata are important for future re-use, and for depositing the data with an archive. In the presentation Kate Fernie discusses planning for archiving and some possible suitable digital archives. She notes that once the archive is deposited, organisations can publish their 3D data in various formats for different audiences (from printing to online viewing or offline platforms). Archiving and preserving the 3D datasets secures the results for future projects, supports conservation, tourism, education and enjoyment of the cultural heritage.

1. Kate Fernie, CARARE Sharing with Europeana: Depositing and publishing 3D datasets for preservation and future access

2. Introduction Twin iT! marks a massive expansion of 3D in the cultural heritage sector The webinars in this series by 4CH have covered the digitisation workflow from the initial plans to modelling techniques Today I will talk about the data that you create and strategies for depositing and publishing your results Oxbow Books, 15 Dec. 2003 ISBN: ‎ 1842170406

3. From fieldwork to depositing your 3D dataset and providing access ‹#›

4. 3D Workflow CAPTURE MODELLING TEXTURING OUTPUTs Point Clouds High Poly Model Low Poly Model Textures Photos

5. Workflow and data 3D workflow uses various equipment and software Survey equipment which produces raw data in various file types Software to process the data to produce the desired results which is capable of accepting data in certain file types exporting data in certain file types

6. From fieldwork to depositing your 3D dataset and providing access Interoperability matters during the project and for future access https://open-science-training-handbook.github.io/Open-Science-Training-Handbook_EN/

7. What’s in a 3D d ata set? ‹#›

8. Laser scanning Thousands of points representing X, Y, Z coordinates, colour and intensity Multiple scans Imported into software for registration and processing Photogrammetry Thousands of pictures taken at different angles around a CH object Imported into software which uses a Structure from Motion (SfM) algorithm to produce a point cloud or a mesh Reference measures GNSS or GPS ground control points TPS points on building features and elements Used for registering laser scan data and photogrammetry Data capture

9. Data - initial processing Data produced in 3D projects is rarely used in raw, unedited form In most projects there will be multiple datasets (e.g. overview and detail scans, photogrammetry, drone imagery , etc.) which are each registered to a coordinate system to create a unified point cloud Point clouds are used to generate various products (such as polygonal meshes , CAD drawings and elevation models)

10. F ile types/formats A lot of different hardware and software is involved. There are many file formats but common file types have potential for deposit and archiving. Photogrammetry Import - images (proprietary raw or standard JPG/TIFF) Export - point cloud/, meshes and textures Point clouds Original raw scan data (proprietary format) + XYZ (ASCII text) Registered point cloud: XYZ (ASCII text) E57 (evolving as a standard) LAS proprietary format … Meshes https://archaeologydataservice.ac.uk/help-guidance/guides-to-good-practice/data-collection-and-fieldwork/laser-scanning-for-archaeology/archiving-laser-scan-data/file-formats-for-archiving-datasets/

11. From data to 3D model In reality capture the point cloud is used to generate a mesh which defines the shape of an object The data undergoes processing/editing Textures are applied and The mesh is optimized A high poly count mesh is the first output from this you can generate lower polygon count meshes models in various formats (according to the planned uses) 137 million polygons Discovery Programme

12. File types/formats Meshes Open formats: OBJ + MTL Collada (.dae) X3D Other formats: PLY glTF STL (for printing) Textures Can be saved as: JPEG PNG TIFF TARGA (TGA)

13. Need for archiving Lower priority Derivatives - 3D models for web; for 3D printing etc. High priority Original raw scan data (proprietary & ASCII) Registered point cloud (ASCII + Open format) Original images (photogrammetry) High Poly mesh + textures Low poly mesh

14. Data files plus metadata https://open-science-training-handbook.github.io/Open-Science-Training-Handbook_EN/ What’s in a 3D dataset?

15. Metadata When depositing 3D datasets archives need good documentation to enable preservation and access, this includes: Details about the project and how the content was produced The technical characteristics – tools and software used, data processing, control, geometry, textures, etc. Info to enable discovery and access – the CH subject, period, copyright, access conditions, media, links etc.

16. Depositing for future access Sean Doran, Discovery Programme

17. Plan for archiving from the start of the project Image: European Data Journalism Network Find a suitable digital archive or repository and check their requirements and what level of access they can offer. University repositories (larger) cultural institutional repositories National repositories such as DANS Data Vault , SND and DRI International repositories such as https://zenodo.org/ Specialist data archives such as ADS

18. Preparing for deposit Don’t leave preparation of the dataset for archiving to the end of the project Specify the data for archiving Specify formats which have most potential for archiving open, industry standard Specify the documentation and metadata that’s needed Agree the copyright status and access permissions

19. Deposit the dataset at the end of the project https://doi.org 10.5281/zenodo.77350786 Now you can plan to publish other outputs!

20. The 3D workflow can produce many outputs – from high poly models for HBIM, low poly models for publishing online to 3D models for printing and more. These can be uploaded with texture files for publication on hosting platforms: Commercial National Local Publishing

21. Viewers Ideally you would select a hosting platform that can render your 3D model in a viewers. There are 3D viewers available as open source software which can be installed on your local server/repository. For example, 3D Hop and the Smithsonian’s 3D viewer. The alternative is to upload the content to a hosting platform (commercial or non-commercial) and take advantage of their viewer

22. Share3D I’ve spoken previously about Share3D, which aims to help CH Sketchfab users to share their 3D models with Europeana. Sign up for an account here: https://dashboard.share3d.eu/login.php

23. More publishing options Smithsonian Voyager https://smithsonian.github.io/dpo-voyager/

24. EC directive focusses on cultural heritage at risk and the most visited sites 3D digitisation e nables monitoring of the heritage (changes over time) and increases access Securing your results for the future Increases the possibility for reuse Leads to future research Supports tourism, education, enjoyment Why storing and preserving your results matters Monitoring

25. Acknowledgements and references Anthony Corns, Technology Manager, Discovery Programme Marco Medici, INCEPTION 4CH Deliverable D3.1 Design of the CH Cloud and 4CH platform 4CH Deliverable D4.1 Report on standards, procedures and protocols ADS Guides to Good Practice ‹#›

26. Kate Fernie [email protected] Thanks for your attention! https://www.carare.eu Training hub: https://pro.carare.eu/en/training-hub/ CARARE is a non-profit membership association which offers advice, guidance and training in the creation, publication and use of digital data. A Europeana aggregator CARARE offers technical services to help institutions share their archaeological and architectural heritage content.