Embracing 3D and immersive technologies in digital libraries: interview with Gabriele Guidi

Citation

Guidi, G. and Tammaro, A.M. (2024), "Embracing 3D and immersive technologies in digital libraries: interview with Gabriele Guidi", Digital Library Perspectives, Vol. 40 No. 1, pp. 148-153. https://doi.org/10.1108/DLP-02-2024-145

Gabriele Guidi is Professor of Informatics at Indiana University (IU), Bloomington. At the end of the 1990s, he began studying three-dimensional (3D) optical imaging with active and passive approaches, developing a broad research program dedicated to 3D digitization for cultural heritage. He is also the director of the Virtual World Heritage Lab, applying these new tools not only as interactive illustrations but as heuristic tools of discovery.

Gabriele Guidi is the Editor in Chief of Studies in Digital Heritage, a peer-reviewed and open access journal. This journal covers topics relating to the use of technology in the study of cultural heritage, with special emphasis on 3D technologies, including 3D data capture, processing of 3D models, theory and practice of 3D restoration of cultural heritage objects, use of 3D models in research and instruction, metadata and paradata standards and best practices for 3D models and the use of 3D models on VR and AR devices as well as on Web pages. Authors are encouraged to embed interactive 3D models into their articles instead of traditional two-dimensional (2D) illustrations.

3D and immersive technologies have become a new and powerful form of information visualization and reuse for digital libraries, and we asked Gabriele Guidi about ongoing 3D projects and research prospects that can transform digital libraries.

Q1. What are the current state of art applications of 3D technology to digital libraries?

The first application that comes to mind concerns the creation of “digital repositories” for objects or structures of cultural interest in the form of 3D digital models. The object’s metadata is stored in a database, similar to how the metadata of a publication is accumulated in a bibliographic database. Through these metadata, it is possible to conduct searches. For example, for museum collections, it is possible to search by era, material, place of discovery or other similar parameters contained in the associated metadata structure. For each result, it is possible to view a navigable 3D model of the object.

EUROPEANA began including 3D digital objects in its collections as early as 2012 with the project “Connecting ARchaeology and ARchitecture in Europeana” (CARARE) [1], uploading several dozen examples from the digitization of the Pompeii Forum conducted in 2007–2008 by the laboratory at Politecnico di Milano, which I then coordinated. Subsequently, 3DICONS [2], another project where I participated as the head of the WorkPackage “3D digitization,” further expanded the CARARE experiment. It defined a refinement of the metadata structure associated with 3D models of cultural heritage and their corresponding digital representations, providing EUROPEANA with more than 3,000 3D models of monumental buildings and museum artifacts. Now, it is pretty common to find 3D material on that platform; but you can also see the flourishing of analogous applications at the Smithsonian Museum (Smithsonian 3D [3]), at the Library of Congress LOC3D [4], and many others worldwide.

Another application of 3D technology arises when one aims to represent the library in its physical form. This becomes particularly significant in cases where the building housing the library holds historical importance. In such instances, the capability to navigate through a virtual reproduction of the library carries intrinsic value as it enables simultaneous exploration of a cultural asset and, eventually, access to a book. During my tenure as a professor at the Polytechnic University of Milan, I supervised a thesis on this precise topic, with co-supervision by Fabio Venuda. It was titled “Bibliogate. Progetto di un'applicazione di realtà virtuale per visitare ed esplorare biblioteche storiche” [5].

Finally, In the context of our 3D digitization courses at IU, I have had the privilege of guiding students from the field of library science who expressed keen interest in using these techniques for crafting 3D models of books, treating them not merely as vessels for written content but as tangible, physical objects. One notable student, Alexandra Wingate, a PhD candidate in library science at IU, investigated this subject for an assignment in my course. Her insightful paper, titled “Photogrammetric Modeling of Rare Books for Book History and Special Collections,” explores the applicability of photogrammetry for book historians, rare books librarians and other professionals in special collections. Wingate created four models representing two ancient books housed at the Lilly Library, a treasure on our campus. One of these, the 1570 edition of “The Elements of Geometrie”' by Euclid, not only contains formulas but also features pop-up content illustrating certain shapes that are impossible to appreciate in a 2D format. This particular example demonstrated that 3D models generated through photogrammetry hold significant utility for various purposes within the context of rare books.

Q2. How can the use of 3D technology broaden discovery and access to the digital library’s collections?

The utilization of 3D technology presents a transformative impact on the discovery and accessibility of digital library collections. One compelling illustration is the implementation of virtual models representing the architectural structures of libraries. This innovative approach allows users to go beyond traditional navigation through a set of records and immerse themselves in the virtual representation of the library’s physical space. This immersive experience becomes particularly valuable when identifying specific volumes, especially those associated with particular sections in historical libraries.

Moreover, the incorporation of 3D models within the content of a digital library introduces supplementary metadata sets, providing a wealth of information beyond what is typically found in descriptive metadata associated with printed texts. These additional sets detail the physical characteristics of books, such as their form or unique features. In essence, this goes beyond the conventional metadata, allowing users to include the intricate details of the physical objects as additional search parameters. The real advantage lies in the ability to cross-reference elements of the physical information container with descriptive elements of the content. This comprehensive approach enhances the overall discovery experience, making it more interactive, informative and conducive to a deeper understanding of the digital library’s collections.

Q3. With 3D technologies, how could the digital library go beyond the simple transmission of information and become more participatory and inclusive?

This will happen thanks to the different features offered by 3D content.

First, digital libraries can leverage 3D technologies to create immersive learning environments. Users can actively engage with 3D models of historical artifacts, scientific specimens or cultural objects, fostering a participatory learning experience. An example of this is Flyover Zone [6], a company here in Indiana, where Bernie Frischer, a renowned archaeologist and my close friend, has created an exceptional 3D model representing the entire ancient Rome at its zenith. Surprisingly, a significant number of licenses for this interactive experience are purchased by university and high school libraries. These institutions use the material to let the instructors teach Roman history in an alternative and engaging manner. Therefore, aside from reading about ancient Rome, you can stroll through the streets of the city, observing buildings that no longer exist from perspectives that are no longer accessible, seamlessly integrating written descriptions with a visual experience.

Second, the integration of 3D technologies can significantly improve accessibility for users with diverse learning styles and abilities. For instance, the affordability of 3D printing enables libraries to create tangible replicas of digital 3D models, facilitating the incorporation of Braille annotations on the surface of these printed artifacts. This approach not only provides tactile experiences for visually impaired users but also introduces interactive features tailored to various preferences, thereby fostering a more inclusive digital library environment.

Third, digital libraries can use 3D models as a basis for community engagement. Collaborative projects, workshops or events centered around exploring and annotating 3D content can foster a sense of community participation and inclusivity. In a recent project in Milan, our laboratory reconstructed the ancient Roman Circus that once stood when Milan served as the capital of the Western Roman Empire. This undertaking involved extensive collaboration with the current owners of buildings in the city center, which now stand above the ancient circus and preserve sections of its walls in their basements. During the concluding presentation of the results at the Archaeological Museum of Milan, it was remarkable to witness the significant attendance of these property owners, all of whom took pride in being part of this historical community.

Fourth, digital libraries can use 3D technologies to tell interactive and immersive stories. Users can navigate through historical events or cultural narratives using 3D reconstructions, offering a participatory experience that goes beyond traditional storytelling.

Finally, 3D models can serve as platforms for crowdsourced annotations and insights. Users may contribute their knowledge, perspectives or historical context, enriching the content with diverse viewpoints and fostering a sense of community participation.

In other words, by embracing 3D technologies, digital libraries have the potential to transform into dynamic, participatory spaces that cater to diverse user interests, preferences and contributions, thereby promoting inclusivity in the exploration and understanding of digital collections.

Q4. What are the issues for the preservation and reuse of 3D objects? What good digital library practices do you know?

Preserving and repurposing 3D objects within digital libraries poses various challenges. When orchestrating a systematic 3D digitization project, such as those undertaken at the Archaeological Museum in Milan for the 3DICONS project (2012–2015), the IU-Uffizi [7] project (2016–2021), where our Virtual World Heritage Lab digitized the entire collection of Roman sculptures at the Uffizi gallery in Florence (Italy), or the ongoing project at the National Archaeological Museum in Naples MANN [8] (2022–2027), where we are digitizing 400 archaeological objects “hidden” in the museum storage for allowing public outreach, the primary challenge arises in determining the type of data to produce and organizing its storage. This dilemma involves both the presentation content and the high-resolution raw version of the 3D models, which are often stored differently.

In our initial project 3DICONS, we opted for the OBJ 3D (a format developed in the 1980s that remains well-supported) to store high-resolution models. For the presentation, we used 3DPDF, leveraging Adobe Acrobat Reader for displaying 3D content, a function still little known by many users. Although effective at the time, 3DPDF had drawbacks, requiring users to download the model locally before navigating through a 3D viewer installed on the client machine. Subsequently, in the IU-Uffizi project, we maintained OBJ for high-res models but shifted to Sketchfab, an online service analogous to YouTube for 3D models, providing both storage and a built-in 3D viewer. In our latest project at MANN, while retaining OBJ for backward compatibility, we introduced GLB, a relatively new 3D standard collaboratively developed by major industry players like Apple, Google, Intel, NVIDIA, Samsung and Sony. This move aims to ensure the longevity of 3D content, considering the collaborative effort invested in its development. Although the presentation format has yet to be finalized, we are contemplating a shift from Sketchfab, which is owned by a private company and, for this reason, is prone to market logic, to a more flexible open-source solution like 3DHOPS, that would also give us complete control on the 3D repository.

In essence, selecting appropriate and widely supported file formats is crucial for the long-term preservation of 3D content. Ensuring compatibility with future technologies and software poses a challenge, given the potential obsolescence of certain formats over time. For this reason, regularly updating data formats is also a critical operation to ensure the relevance of 3D content. This practice should be integrated into project management considerations, including budgetary planning, to maintain the currency and accessibility of 3D objects within digital libraries.

A parallel discussion can be extended to metadata; establishing consistent and comprehensive metadata standards for 3D objects is paramount. Metadata should encompass information about the creation process, context and intended use to facilitate future comprehension and reuse. However, currently, there are a few schemas, but in my view, none is sufficiently flexible to enable swift integration into a database and the potential subsequent in-depth enhancement of the descriptive section, an operation often needed for museum objects with an articulated story.

Q5. What is 3D in the framework of immersive technologies and virtual heritage? What could be the impact of artificial intelligence?

In the immersive technologies and virtual heritage framework, 3D refers to the creation and representation of 3D digital models replicating real-world historical artifacts, buildings or environments. This simulation can encompass either a reality-based 3D model, essentially a 3D snapshot of an artifact or building in its current state, or a restored version incorporating synthetically generated components based on iconographic rules or architectural grammars. In both instances, the ultimate outcome involves computer-generated graphics and interactive technologies to provide users with a more immersive and realistic experience compared with traditional 2D representations.

The impact of artificial intelligence (AI) in this context can be profound because AI technologies can enhance the creation and interaction with 3D models in several ways.

The well-known ability of AI to recognize similarities or patterns in 2D has been reapplied to the 3D domain. So, in theory, AI algorithms could aid in the automated reconstruction of historical sites or artifacts using fragmented or incomplete data, potentially streamlining the 3D modeling process and conserving time and resources. Despite several studies that have been conducted in this field over the past 10 years, the journey toward tangible and applicable results seems still extensive.

A promising application of AI is the creation of intelligent agents navigating into virtual heritage environments. These agents are designed to respond to user queries, allowing scholars to investigate the dynamics between a reconstructed heritage site and simulated humans, including entire crowds that may have occupied a specific building in the past. This same approach can also enhance the experience for casual visitors who wish to learn, for example, about the complexities involved in clearing the Colosseum of spectators after a gladiator game.

Finally, with AI-based natural language processing, user interactions in virtual heritage settings can be enhanced. Users might communicate with the system using natural language, asking questions about historical contexts or artifacts, and AI, properly trained by experts in the field, can provide relevant information in response.

Q6. What further research needs do you see in the application of 3D technology to the digital library?

In addition to the aforementioned AI research, which is likely to be a predominant focus in the coming year, there is a crucial need for further research in standardizing metadata for heritage objects in 3D digital libraries. Developing comprehensive and standardized metadata formats is essential to enhance interoperability and facilitate effective organization, retrieval and sharing of 3D content. Although initial efforts in projects like EUROPEANA have established a widely recognized metadata model, it is often considered overly simplified. Although the latest incarnation of the CARARE standard should go in this direction, it appears to fall short of achieving this goal. Research in this area should also explore methods to improve interoperability among various 3D digital libraries and platforms, necessitating the development of standards and protocols that enable seamless integration, sharing and collaboration across diverse repositories of 3D content.

The process of enriching 3D models with information also involves semantic subdivisions. The significance of this step lies in the fact that reality-based 3D models typically constitute a single digital object, regardless of the semantic and functional differences between represented content, such as a capital, column or temple roof. The integration of AI in this realm holds the potential to streamline an otherwise labor-intensive manual process. Research endeavors could concentrate on integrating semantic information into metadata, offering a deeper understanding of the cultural and historical context of 3D objects within digital libraries.

Further research is imperative to enhance the user experience and visualization aspects of 3D digital libraries. This entails the development of visually compelling 3D visualization tools that are not only powerful and quantitative but also user-friendly, fostering engagement and understanding, integrated with sophisticated metadata queries capable of investigating a 3D repository with an advanced data-driven approach. Exploring the creation of collaborative environments for 3D digital libraries is also crucial. This involves the development of platforms that facilitate collaboration among scholars, researchers and the public in contributing to the enrichment of 3D heritage content.

Finally, research efforts and policies should prioritize strategies for the long-term preservation and accessibility of 3D digital content. This includes an exploration of best practices for archiving 3D models, ensuring their sustained accessibility and usability over extended periods.

Notes

1. www.carare.eu/

2. http://3dicons-project.eu/

3. https://3d.si.edu/

4. https://labs.loc.gov/work/experiments/loc-3d/

5. www.politesi.polimi.it/handle/10589/140890

6. www.flyoverzone.com/author/flyover-zone/

7. www.digitalsculpture.org/florence/

8. https://news.luddy.indiana.edu/story.html?story=Virtual-World-Heritage-Lab-joins-with-Italian-museum-to-digitize-ancient-artifacts