Organizing subject access to cultural heritage in Swedish online museums

2.2.1 Heterogeneity of museum object types and broad cultural heritage

The International Council of Museums (ICOM Statutes, 2007, p. 2) defines a museum as “… a non-profit, permanent institution in the service of society and its development, open to the public, which acquires, conserves, researches, communicates and exhibits the tangible and intangible heritage of humanity and its environment for the purposes of education, study and enjoyment”. Compared to archives and libraries, museums collect a vast number of heterogeneous object types. Also, when we speak about museum objects, we predominantly mean unique objects: well over 99% of all museum objects are unique objects/works, entities that do not come as multiple copies or examples of manifestation of some work. With respect to their subject, most museum objects do not even have narrative content, but are instead designated by their material, form and function. Therefore, in subject analysis and indexing, it is important to identify and represent not only motives depicted in an object (ofness) or what an object/work is about (aboutness) but also what an object is per se (isness) and what its function is.

2.2.2 Isness, aboutness and ofness

To further elaborate the concepts of isness, ofness and aboutness, let us take Hagia Sophia (Istanbul) as an example in two different cataloguing procedures:

1. When cataloguing a photograph (the Object/work type is photograph) depicting Hagia Sophia from 2020, the title should be “Hagia Sophia” representing work (ofness), and one of the terms of the subject index should be “mosque” (aboutness).

2. When cataloguing the Hagia Sophia itself as a built work, according to the book Cataloguing Cultural Objects: a Guide to Describing Cultural Works and Their Images (CCO), three values should be recorded in the Object/work type element (Baca et al., 2006, p. 57): “cathedral”, “mosque” and “museum”; the general subject term should be “architecture”; and the Sspecific subject terms should be “cathedral”, “mosque” and “museum” (isness), that is, the same as the Oobject/work type.

The function of the Hagia Sophia building has changed over time: It was built in 537 as Byzantine Christian cathedral; in the 13th century it became the city's Roman Catholic cathedral and in 1453, after the fall of Constantinople to the Ottoman Empire, it was converted into a mosque. In 1935, the Republic of Turkey established it as a museum; in 2020, it was reopened as a mosque. Consequently, when cataloguing a building per se (example 2) we must index all the functions that the building had, and when cataloguing a photograph showing Hagia Sophia (example 1), then it is crucial to identify the time when the photograph was taken and index only the function the building had at that very moment.

The short definition of isness from the Categories for the Description of Works of Art (CDWA) guidelines is based precisely on the overlap of object/work type and subject index elements: “If the work is the subject term, this is called isness” (Baca and Harpring, 2016). Isness denotes what a work is or which class of objects it belongs to; therefore, these kinds of subjects are based on the form or type of the object or its genre.

Values for isness are ideally taken from controlled vocabularies such as the Art and Architecture Thesaurus' (AAT) Objects Facet (i.e. visual works by form: medals, tapestries, diptychs). In addition to the object/work type, the function of the object and changes in its function over time are essential for documenting utilitarian objects and various forms of industrial and technical heritage. In this respect, the CDWA clearly recommends the following: “Works that have a primarily functional purpose, such as architecture and utilitarian objects, should also be analyzed for subject, including the work's function and/or form” (Baca and Harpring, 2016). AAT's Objects Facet commonly lists concepts/object names (e.g. parade armours, wedding dresses) in various classes that provide guide terms related to function (e.g. <armors by function>, <dresses by function>).

In practice, however, when creating metadata using established standards, the isness subject type is problematic because it overlaps with the Object/work type metadata element. The term “pocket watch” must be recorded in the Object/work type element in CDWA (or in the semantically equivalent Object name element in SPECTRUM). Moreover, isness overlaps with elements in the group Object history and association information in the SPECTRUM standard: in this group, in addition to the element in which the function is recorded, there are elements that determine the spatial and temporal context of functions. However, should we record this term as a subject as well? Is that unnecessarily redundant, or would this redundancy help users search and retrieve subjects? This problem requires further research and feedback from user studies.

While on the one hand isness is related to a work's function or form, and is therefore used mostly for utilitarian objects and architecture, ofness and aboutness, on the other hand, are most often encountered in the field of fine arts. Ofness is about what a work depicts, what a non-expert viewer could see and recognize in the visual content of the object (e.g. plants, animals, objects), and aboutness is related to narrative, thematic, iconographical or symbolic meaning. Ofness and aboutness reflect the dual nature of the visual symbol, which further contributes to the complexity of image analysis (Jack, 2001): an image represents both a physical object (e.g. owl) and an idea (e.g. wisdom). Both of these aspects of the object are important for subject access. For example, one user may want to find works that simply represent owls (ofness). As owls commonly represent wisdom in Western civilization; another user might be interested in wisdom as a subject (aboutness). At the aboutness level of analysis, metadata creators should be especially careful of culturally conditioned symbolisms, meanings and interpretations, as in this example: in most Native American legends, owls are a symbol of death.

Furthermore, Erwin Panofsky distinguishes three levels of the analysis of subject matter or meaning that are applicable to artistic and utilitarian objects that have a certain visual content (e.g. pictures, drawings, tools with depicted content) or to objects whose material is designed to represent a certain form and content (e.g. sculptures, totem poles, origami, lace):

1. At the first level, by which we describe the primary or natural subject matter, we determine the factual and expressional visual content. All these objects and features form a list of artistic motifs, and their enumeration about a particular work of art constitutes a pre-iconographic description (i.e. ofness).

2. At the secondary or conventional subject matter level, we connect the presented artistic motifs with certain concepts, themes or ideas, or stories and allegories, and thus we discover the secondary or conventional meaning (i.e. aboutness).

3. Intrinsic meaning or content is achieved by interpreting motifs, themes and allegories from previous levels of analysis and constituting the world of symbolic values. It is the subject of iconography in a deeper sense, which implies a method of interpretation that is more of a synthesis than an analysis (Panofsky, 1993, pp. 53–55).

To provide a distinction between the aforementioned levels of subject analysis, indexing and interpretation, CDWA metadata categories include the General Subject Type and Specific Subject Type elements with a controlled set of values limited to description, identification, interpretation, isness, aboutness, ofness.

2.2.3 The problem of translatability between different media on subject indexing

Svenonius (1994) pointed out the key challenges of how to represent in subject index terms a topic from an object represented in media that does not use any words, but is instead represented in visual (art) or aural (music) terms. In comparison, when conducting subject analysis and indexing of textual documents, we always use the medium of language: while examining documents, we read natural language; while determining their subjects, we use language concepts; and while selecting index terms, we use various artificial languages. But museum and heritage objects (with relatively few exceptions) typically do not have content encoded in text that we could read. Most of them do not have narrative content; and in those which have, this content is encoded primarily in a visual system that needs to be determined, expressed into natural language concepts and finally translated into metadata that will represent the work in an information system.

2.2.4 The importance of visual representations

For all of the reasons mentioned above, the existence of high-quality visual representations is crucial for museum and heritage resources. As access to image-based resources is fundamental to research, scholarship and the communication of cultural knowledge, it is no longer enough to provide only simple (thumbnail or full screen) images, but it is necessary to enable functionalities such as deep zooming or annotation capabilities that enable transfer of knowledge about the various characteristics of heritage objects mentioned so far.

In that context, the International Image Interoperability Framework (IIIF – International Image Interoperability Framework, 2021) represents an important standardization initiative which defines, develops, cultivates and documents methods for access and use of high quality image resources, not only deep zoom and annotation capabilities but also features for image comparison and manipulation (rotating, setting brightness, contrast, etc.) as well as a set of common application programming interfaces (APIs) that support interoperability between image repositories (IIIF Image API and IIIF Presentation API). An example of such interoperability is enabling the display of an image from one repository in another information context without the need to create a copy of the image.

2.2.5 Content-based image retrieval and computer vision in subject indexing

While in the context of text-based documents we talk about full-text search, in the context of visual resources we talk about content-based image retrieval (CBIR), a methodology for retrieving images from databases of images based on visual content. In CBIR technology, content most often refers to various low-level visual features such as textures, colours and shapes that can be detected within the image and are then used as index terms. At the beginning of image retrieval, a user expresses his or her imaginary intention into some concrete visual query which can be query by example image, query by sketch map, query by colour map, query by context map, etc. (Zhou et al., 2017, p. 2). An example from Europeana, illustrated in Figure 1, shows the combination of textual search criteria (“painting”), the providing country (Sweden), the licence type (free reuse) and a query by colour (red and green).

Zhou et al. (2017) emphasize the role of deep learning (DL), belonging to a broader family of machine learning (ML) methods, as a part of the field of artificial intelligence (AI) in CBIR, which enables learning of high-level abstractions close to human cognition processing. This could help museums not only to recognize real-world objects (ofness) automatically and identify particular people with facial recognition systems but also to obtain high-level understanding of images (e.g. ambience, moods, expressions).

2.3.1 Conceptual models

In the museum’s community, the standard corresponding to the Library Reference Model (LRM) developed by IFLA (International Federation of Library Associations and Institutions) is the ISO standard CIDOC-CRM developed by the International Council of Museums (ICOM)'s International Committee for Documentation (CIDOC Conceptual Reference Model, ISO, 2006). This is described further in the section “User requirements” below. However, the two are rather different in that CIDOC-CRM focuses on events and processes while IFLA LRM primarily models the outcomes of the processes. In order to align the two models, the International Working Group on Functional Requirements for Bibliographic Records (FRBR)/CIDOC-CRM Harmonisation was established. It published FRBRoo (FRBR Object Oriented, ICOM and CIDOC, 2006) to serve as an ontology to facilitate the integration and interchange of bibliographic and museum information.

While CIDOC-CRM supports aboutness (ICOM and CIDOC, 2002) at a very general level, and attempts exist to evolve this further (see, for example, Carboni and Luca, 2017), we would like to point to how subjects are modelled in the IFLA LRM model, hoping that the FRBR/CIDOC CRM Harmonisation Group will take the FRBRoo development further towards modelling aboutness more specifically.

2.3.2 Subjects in data structure standards

Data standards for museum resources provide a good structure for recording subject-related information, with small differences that we hope will be harmonized in the future. The key reference point for the documentation and management of all types of museum collections is SPECTRUM, the United Kingdom's collection management standard that is increasingly adopted in other countries. SPECTRUM standardizes collection management procedures and data structure. Regarding subject indexing, one of the prescribed primary procedures of SPECTRUM Version 5.0, Cataloguing, defines the Object description information group that contains the Content and subject information subgroup. The CDWA comprises a set of guidelines for best practice in cataloguing and describing works of art, architecture, other material culture, groups and collections of works, and related images. CDWA is arranged in a conceptual framework that may be used for designing data models and databases, as well as for accessing information. CDWA includes around 540 categories and subcategories of information (Baca and Harpring, 2016). CDWA guidelines define a group of metadata categories called Subject matter which contains 17 subcategories related to the subject.

In order to establish semantic interoperability between different metadata schemes, a number of mappings have already been implemented, such as the Metadata Standards Crosswalk included in the CDWA standard. In establishing interoperability in the museum community, the LIDO (Lightweight Information Describing Objects) XML metadata exchange schema (Coburn et al., 2010) plays a particularly important role. LIDO takes into account both SPECTRUM and CDWA data structures, and its development (currently in beta version 1.1) has been prompted by the need for better connectivity with LOD vocabularies and more functional data display in public catalogue interfaces.

2.3.3 Controlled vocabularies related to the subject

Here we would like to focus on a few key international classification systems and thesauri that serve as data value standards and provide vocabulary control for subject terms in museums: Iconclass, the Getty Vocabularies, and the Social History and Industrial Classification.

Iconclass is a classification system designed for art and iconography. It is the most widely accepted scientific tool for the description of subjects represented in images (Iconclass, 2021). The Iconclass classification is accessible through the web-based Iconclass Browser and is also available as linked open data (LOD) that enable interoperability in terms of linking, exchange and enrichment of metadata.

The Getty Vocabularies include the following series of highly influential multilingual thesauri and databases that are constantly evolving and being updated through a number of international initiatives and projects. These can also be used to control vocabulary in subject indexing. The AAT contains concepts and terms on objects, materials, techniques, styles, periods and other concepts related to art, architecture and the broader field of cultural heritage. The Cultural Objects Name Authority (CONA) is a relatively new terminology resource that compiles a list of important works of art and buildings. Examples of names in CONA are Chayasomesvara Temple, Mona Lisa, Livre de la Chasse, Hagia Sofia (Harpring, 2018, p. 3). CONA could be used in cases when a work depicts another work. The Getty Iconography Authority (IA) is a relatively new terminology that focuses on subjects and topics (e.g. Bouddha couché, Adoration of the Magi) (Harpring, 2018, p. 2). The Union List of Artist Names (ULAN) is also important for recording names as subjects; it not only contains the names of artists but also the persons associated with them (teachers, families, collaborators) and the names of corporate bodies (museums, workshops, art groups). The Getty Thesaurus of Geographic Names (TGN) is important for recording geographical names as subjects. All Getty Vocabularies have a SPARQL endpoint and are available as LOD, which is important for linking and enrichment from other vocabularies (i.e. VIAF for people as subjects, GeoNames for places as subjects, etc.).

The Social History and Industrial Classification (SHIC) is used in social and local history museums as a means to make links between objects through their context and background (e.g. functions of objects, human activities). The classification system is divided into four top facets (community life, domestic and family life, personal life, working life), which together represent all aspects of human activity (Social History and Industrial Classification, 2021).

2.3.4 Data content and cataloguing rules and guidelines

There are only a small number of cataloguing rules in the museum community. The book Cataloguing Cultural Objects: a Guide to Describing Cultural Works and Their Images (CCO) is the most influential set of guidelines for cataloguing cultural works and their visual surrogates. It addresses subject indexing in more detail, published in 2003 and available in a digital open access edition from 2006 (Baca et al., 2006).

The next important data content standard to consider is CDWA, which largely overlaps with CCO in content and structure. More importantly, guidelines are further developed and upgraded (the latest version of CDWA was revised in 2016 by editors Patricia Harpring and Murtha Baca). Cataloguing rules for subject analysis and indexing in CDWA are described in detail in the Subject Matter information group, which, according to the guidelines, is also one of the mandatory elements for cataloguing works.

That said about standards catering for subject indexing in museums, the software interfaces seem to indicate that this practice in museums is underused. There are few empirical studies that analyse subject indexing in museums. A rare example is an OCLC study survey of core fields (agreed by the CDWA and CCO guidelines) from 2010 which shows that nine prominent American art museums apply 17 core fields more or less consistently, but that subjectTerm is a conspicuous outlier in that it is used by only two institutions (Waibel et al., 2010). Since then CIDOC, Getty, the Collection Trust and others are working hard to develop standards to improve this situation.

4.2.1 Feature 1: Browsing by subject access points: subjects from controlled vocabularies, like subject headings, captions from classifications systems, free keywords

This feature is used by 10 museums' in-house search tools and 5 cross-search services, covering 73 museums in the sample (80.2%). Of the in-house search tools, Sörmland Museum's tool for browsing (https://www.sormlandsmuseum.se/utforska/), under Subject, allows the user to choose between 17 different broad categories by which all the objects in the museum collection are classified (e.g. love, war, clothes, etc.), rather than supporting subject browsing by concepts from a controlled vocabulary (https://www.sormlandsmuseum.se/utforska/?TypeCategory=&SubjectCategory=51&SortCategory=index&SearchText=). Similarly, the Museum of Work allows browsing using several simple categories related to motifs such as Swedish politicians, communism, equality, etc. (https://www.arbetetsmuseum.se/ewk-museet/sok-i-ewk-databas/). Likewise, in the Motala Industry Museum, the only way to explore the museum's collection is to browse through over 12,000 of its photographs. The user can choose one of 28 broad categories which are not from a controlled vocabulary but list topics like bridges, boats, employees, history, etc.; no search option is available (http://www.motala-industrimuseum.com/index_bild.php).

Figure 2 below shows what browsing features look like:

As in Figure 3 below, museums using Kulturhotell are different because of a much larger set of categories – or rather keywords used to describe museum objects (“Bläddra efter nyckelord”); there are over 17,000 keywords, and browsing is allowed based on an alphabetical order, number of objects and date when the object was added (https://blm.kulturhotell.se/items/tags?per_page=72). However, an alphabetical listing of so many keywords is not really user friendly; hierarchical subject browsing is much more suitable.

Of cross-search services, in Digitalt Museum, the user can seemingly browse through more than 220,000 topics although actually only 100 of the most common topics (those with most documents) are accessible for viewing and browsing (https://digitaltmuseum.org/search/?aq=): see Figure 4 below. As in the previous example, only 100 keywords, ordered only by popularity will not suffice when there are over 220,525 topics available for over 6,000,000 objects.

Europeana allows alphabetical browsing of topics (https://www.europeana.eu/en/collections/topics) resulting in 19 pages of categories (24 categories per page). No controlled vocabulary seems to be in use.

Google Arts and Culture, among many browsing options (e.g. by colour, popular topics, occasional topics, e.g. Easter or Christmas), supports browsing by six broad categories: artists, mediums, art movements, historic events, historical figures, and places, of which the last four are subject related. The categories are further subdivided into more specific topics (artists into artist names); at the second hierarchical level, very large categories can be listed alphabetically or in a timeline; specific objects by an artist can be ordered by popularity, time and colour (https://artsandculture.google.com/explore).

Wikimedia Commons supports browsing by an elaborate, multi-level hierarchical classification of topics with many levels (https://commons.wikimedia.org/wiki/Main_Page). The classification based on topics is structured into four main categories at the first level: (1) nature; (2) society culture [sic]; (3) science; and (4) the environment; these are further subdivided into a number of more specific hierarchical levels. This is closest to what a hierarchical browsing structure should look like, as the user learns about the concept space and acquires knowledge about the world based on how the classification presents it. Such a classification tree should ideally also have a browsable number of items – a few dozens, not hundreds or more as we often see in online search services.

Overall, it is important to observe that the lists of topics in the small number of services providing this feature do not seem to be taken from any kind of controlled vocabularies but are adjusted to suit the museum collections. The reason for this could be the fact that there are no national authority files available that would allow for subject vocabulary control and/or the fact that dominant English vocabularies such as AAT mentioned above have not been translated into Swedish. However, this situation prevents interoperability across collections and also puts unnecessary demand on the user to learn about new structures for every museum.

Furthermore, when providing options for subject browsing, the rationale is to provide the end users with an insight into the subject area; subject browsing is particularly useful when the user does not know what specifically to look for or is new to the subject area or museum and is therefore unable to form a good search query. The cross-search service eMuseumPlus states in its instructions that a classification system that it uses for object types (although not for subjects) could be a good tool to gain an overview of the collections or assist when one does not really know what one is looking for (eMuseumPlus, 2021a, b); this demonstrates that information scientists are aware of this. However, having only a small number of subject categories for a large number of objects is not very meaningful for the user; having only one or two dozen categories for hundreds or even thousands of objects will again result in a list which is of little use as few users would take the time to go through a long list of objects from the same category. Between a dozen and a few dozen objects per subject would be more practical for the user, and this is why multi-level detailed hierarchical classification systems such as those used by Wikimedia Commons are useful. If, in addition, they could be standardized and used across different museums, that would be ideal so that the user would need to learn about the specific concept space only once. The classification systems need to be developed on the basis of identified user requirements and updated regularly, at the same time following international standards and guidelines for creating controlled vocabularies (see also the section Background above).

4.2.2 Feature 2: Searching by subject access points from controlled vocabularies, including by individual words

The feature is used by 11 museums' in-house search tools (the Swedish Museum of Performing Arts, the Swedish History Museum, Stockholm County Museum, Sörmland Museum and 7 Kulturhotell) and 1 cross-search service (Alvin): 13 museums in total (14.3%). Of the former, the Swedish Museum of Performing Arts, which uses Axiell software, allows searching by many metadata elements. One that is subject related is a motif of person (https://calmview.musikverk.se/CalmView/Advanced.aspx), which gives the appearance of using an authority file; however, it is hard to say if it really uses one (no information is given there or under Help). The Swedish History Museum allows the user to search by 10 “topic-like” object categories (e.g. religion and cult, transport) (http://mis.historiska.se/mis/sok/sok.asp?qtype=f&page=4), as in Figure 5 below. Similarly, Stockholm County Museum allows the user to refine a search using six general categories (everyday, typical of the time, communication, lifestyle, change and my place) (https://samlingar.stockholmslansmuseum.se/?).

Users of Sörmland Museum, here using its advanced search tool (among many other search options) can search by a motif related to a person or a place shown in a picture (Figure 6 below).

Figure 7 shows searching by person in the photograph. The field for a person's name seems to be taking data values from an authority file (https://sokisamlingar.sormlandsmuseum.se/items/search).

As in Figure 8 below, all the museums that use Kulturhotell (https://blm.kulturhotell.se/items/search) allow the user to search by several subject-related metadata fields: 5 different subject fields (ämne) and 14 motif-related categories (mostly for location but even motif category); 5 of 7 museums using Kulturhotell even support searching by person in the photograph (“personer i bild” in Swedish).

Of the cross-search services, Alvin supports searching by subject; the field is reportedly controlled (SAO – Swedish Subject Headings and TGMII – Thesaurus for Graphic Materials II) but no list of terms is shown in the search tool, so the user cannot know which terms can be used.

Similarly, while Carlotta supports searching by motif (“motivkategori” in the Swedish interface), it is not clear where the terms are coming from and whether they are controlled (http://carl.kulturen.com/web). DigitaltMuseum allows for searching by topic in its advanced search tool, but as in the former case, it is unclear if the terms are controlled (https://digitaltmuseum.org/search/advanced). Therefore these two search services are not counted amongst those implementing this feature.

Of the museums with their own search tool, the Swedish Museum of Performing Arts uses Axiell's Calmview Advanced Search interface which allows searching by a field called “keyword” obviously not from a controlled vocabulary, so this cannot be counted (https://calmview.musikverk.se/CalmView/advanced.aspx?src=CalmView.Catalogue). Of the cross-search services, Alvin supports searching by subject in its Extended Search interface, but it does not specify whether the values are controlled and how to search (http://www.alvin-portal.org/alvin/advanced.jsf?dswid=-6194&searchType=EXTENDED&query=&aq=%5B%5B%5D%5D&aqe=%5B%5D). The Carlotta software (e.g. Kulturen at http://carl.kulturen.com/web), under Simple Search, provides a search by “name” but does not further specify whether the terms are taken from a controlled vocabulary, which implies that they are not. Advanced search is only for museum specialists since it supports searching by database fields using specialized terms such as FOLNAM, OBJIDN, etc. The Help page does not specify any use of KOS. The interface of eMuseumPlus (http://emuseumplus.lsh.se/eMuseumPlus?service=ExternalInterface&module=collection&moduleFunction=search) supports searching by person (field: “namn”) but does not use any controlled vocabulary. Kringla supports subject searching in its Detailed Search in two user-friendly named fields “What are you looking for?” and “Object title”; however, these do not seem to be taken from a KOS (http://www.kringla.nu/kringla/). So this cannot be counted either. Moderna museet allows subject searching in its Advanced Search through its Title field (https://sis.modernamuseet.se/sv/advancedsearch), but there are no KOS-based fields either.

4.2.3 Features 3 to 6

Features 3, 4, 5 and 6 take further advantage of characteristics of controlled vocabularies to support the end user in subject searching. Feature 3 is Browsing by facets, aspects and individual concepts from controlled vocabularies, such as individual terms from subject headings, as well as captions and notations representing individual concepts from synthesized classmarks (e.g. in Universal Decimal Classification). This would allow browsing by specific topics from existing categories and would require the use of advanced browsing interfaces: none of the interfaces support this. Feature 4, searching by any combination of individual concepts and facets (as outlined in the preceding features) would allow very specific query formulation and retrieval of highly precise results. This feature requires, however, that suitable, quality-controlled controlled vocabularies such as AAT (see Background above) are actually used. Wikimedia Commons allows searching by words using its categories. Feature 5, searching by major and minor themes represented by controlled vocabularies, if supported by the indexing policy, does not seem to be supported by any of the search tools. If it was, it would allow high precision in the retrieval because it would be possible for the user to specify whether a subject index term is of major or minor importance for the document at hand. Feature 6 includes presenting and browsing excerpts of concept hierarchies (e.g. a classification scheme or a thesaurus), matching words and phrases from search terms, including for disambiguation, narrower, broader and related searching. It again builds on subject indexing based on high-quality controlled vocabularies. None of the search tools in the study use this feature; if they did, the end user would be able to disambiguate a search term, for example, whether the term “bank” is related to banks as financial institutions or to banks of a river; and at the same time it would be able to look for narrower terms or broader terms as presented within a hierarchy of related concepts. Although Wikimedia Commons has an elaborate classification tree of categories, it does not support searching by categories except where relevant excerpts from the classification tree are retrieved for further search query formulation.

4.2.4 Features 7, 8 and 9

The following three features are closely related: feature 7, auto-completing search terms once the user begins typing; feature 8, auto-suggesting authorized controlled versions of entered search terms, presenting all the relationships and allowing further choice in browsing or searching the controlled vocabularies; and feature 9, suggesting corrections to mistypes. In the study sample, since few controlled vocabularies are used, none of the search tools seem to provide automatic translation into controlled terms (feature 8).

The auto-suggest feature proposes the user a list of suggested terms after the user has started typing a search phrase. The feature is used by 14 museums' in-house search tools (the Swedish Museum of Natural History, Sörmland Museum, both Moderna Museet museums, all 7 Kulturhotell and 3 individual eMuseumPlus museums), and 6 cross-search services (eMuseumPlus, Europeana, Kringla, Google Art and Culture, Wikimedia Commons, MIMO), a total of 75 museums (82.4%).

This feature works in a similar way in all the search services: the user is given a list of matching terms immediately after having typed the first few letters of a search term in the search box (as shown above in Figure 9, which shows its use in Alvin); however, 1 out of 2 Sörmland Museum's search tools retrieves records directly where the search string begins the words in the records retrieved. Figure 10 from Sörmland Museum shows how, upon typing the string “ann”, records will be retrieved which include this string in the names of objects or other metadata.

Sörmland Museum, Kulturhotell (5 Kulturhotell museums of 7 use this feature), Alvin and eMuseumPlus are the only services using controlled vocabularies in this way, while all the others use free keywords.

None of the museums or cross-search services provide feature 8, that is, the auto-suggesting function also presents related term relationships and allows further choice in browsing or searching the controlled vocabularies. However, the Swedish Museum of Natural History seems to retrieve matches which are not simply string matches although it is not clear how these suggestions are derived. As in Figure 11 below, if one starts typing the string “hun” under Classification, a drop-down list of terms containing the string appears, showing it in bold (e.g. “Hunterius swedenborgii” for a right whale, “Empria hungarica” for a parasite, etc.) while also listing terms which do not match the search string but are synonyms (e.g. the insect “Caenocoris sanguinarius” is a synonym of “Thunbergia sanguinarius”).

Of the cross-search services, MIMO is a good example of providing multilingual auto-suggestion in their simple search box. While Kringla also supports auto-suggestions, it also seems to suggest misspelled versions. Figure 12 below shows how, when entering “annn”, it will suggest misspelled versions with three “n” letters instead of two such as “annnat” (likely a misspelling for “annat” meaning “other”), “annnan” (likely a misspelling for “annan” meaning also “other”), and “annnars” (likely a misspelling for “annars” meaning also “otherwise”).

Feature 9 supports suggesting corrections for mistypes. None of the museums' own search tools seem to support this; two cross-search services do (66 museums, i.e. 72.5%). Kringla suggests the right word and asks “did you mean … ?”, but as in Figure 12 above, there are some objects that match the mistyped search string, in which case the suggestions are not corrected. Wikimedia Commons has implemented the feature in the same way, with “did you mean …” as in Figure 13 below.

4.2.5 Features 10, 11, 12, 13, 14

The following features are all related to searching. Feature 10, searching by words from various metadata elements and full-text is rarely implemented in museums because museum objects are rarely full-text documents. Instead, in this context, we take full-text to mean full metadata records, as searching on all metadata fields will improve recall. There are 20 museums with their own search tools and 5 cross-search services (83 museums, i.e. 91.2%) that allow searching through many different metadata fields. These are Kulturhotell (>100 metadata fields), Sörmland Museum (33 fields), Alvin (17), Carlotta (15), the Swedish Museum of Natural History (13), the Swedish History Museum (12), DigitaltMuseum (11), the Paleontological Museum of Uppsala University (10), the Swedish Museum of Performing Arts (9), eMuseumPlus (7), Kringla (7), Moderna Museet (6), Naturhistoriska museum (5), Gotland Museum (4) and Litografiska Museet (4). The highest number of possible metadata fields combined in one search is returned by Kulturhotell, which allows searching by over a hundred [sic] metadata fields in one search; however, it is doubtful if any of the objects in their collections is described using all of the fields.

Feature 11, combining controlled subject searching with searching by other bibliographic fields, is supported by those which are listed under feature 10, but which at the same time use at least one controlled field. There are 7 museums whose in-house search tools are in this group (the Swedish Museum of Performing Arts, Sörmland Museum and 5 out of 7 museums using Kulturhotell) and 1 cross-search service (Alvin), giving a total of 9 museums (9.9%). The controlled subject fields have been already described under feature 2.

Feature 12, highlighting search terms in retrieved metadata and resources, is implemented by two museums in their own search tools and two cross-search services (i.e. 65 museums or 71.4%). The Swedish Museum of Performing Arts, using Axiell software, highlights search terms in the retrieved metadata and resources (both in the simple and advanced search); however, the search term is highlighted only when the metadata record is open rather than in the list of retrieved snippets, as in Figure 14 below.

Sörmland Museum highlights the search string in the retrieved list of snippets but not in the open record. Both options are required for the best user experience. The museum allows this option only in the simple search interface, but it would be better to offer it in both the simple and advanced interfaces.

Of the two cross-search services, Wikimedia Commons highlights search terms in the results list and Kringla has the best implementation, both in the results list and after opening a selected metadata record.

Feature 13, advanced searching by Boolean and proximity operators, truncation, and wildcard search, is used in different ways by 15 museums in their own search tools and 6 cross-search services (i.e. 81 museums or 89%). A textbook example of the use of Boolean operators can be found in the Swedish Museum of Natural History, which allows combining the input of several search strings via different search boxes, which are then combined with Boolean operators (AND, OR, NOT) as in Figure 15 below.

In other museums the user is expected to write the Boolean operators in the search box. Interfaces which support this kind of implementation are

1. Digitala Stadsmuseet (http://digitalastadsmuseet.stockholm.se/fotoweb/), which requires the user to type “AND” (to retrieve documents matching both searched terms), “OR” (to retrieve document matching any of the search terms), “NOT” (to exclude a term from searching); it also supports the use of “*” for truncation (to search for all forms of a word) and “?” for a wildcard (a symbol that can represent any character).

2. Kulturhotell and Sörmland Museum, which requires the use of “+” for AND, “−” for NOT, “*” for truncation, and quotation marks for searching documents that match the exact phrase.

3. Gustavianum (https://pragmata.sia.uu.se/pragmata/), which requires the user to type “+” to give a search term higher priority (this is the phrase used in the interface), “−” for NOT, and quotation marks for an exact phrase.

4. Carlotta gives no specific instructions; the help page on searching says only that terms can be combined with each other. An exception is made for the use of a wild card (“*”), the usage of which is explained. However, upon testing a few examples, it seems that the following is supported: AND, OR, NOT, “?”, “*” and quotation marks.

5. MIMO is similar: some Boolean operators work (“AND”, “NOT”, “?” and “*”) but no help is provided on how to search.

6. DigitaltMuseum, eMuseumPlus and Europeana all support the use of “AND”, “OR” and “NOT”. In eMuseumPlus and Europeana truncation is available, and in Europeana the use of a wild card (“?”) and a similar spelling function (“∼”) are also available.

7. Wikimedia Commons provides a more user-friendly support for the use of Boolean operators in its advanced search interface, shown in Figure 16 below: “These words” (AND), “Not these words” (NOT) and “One of these words” (OR), as well as searching by phrase with “Exactly this text”.

Feature 14, linking each subject access point to its resources, is a key feature of hypertext that ought to be fully explored in online search services. There are 11 museums with their own search tools (Sörmland Museum, the Museum of Work, 2 of 2 Digitala Stadsmuseet museums and 7 of 7 Kulturhotell museums) and only 1 cross-search service (Carlotta) whose interface supports this functionality, a total of 25 museums (27.5%). All of these work in the similar way: the user can click on a keyword in the metadata record of the retrieved object in order to retrieve other objects matching the same keyword. Carlotta supports this feature in many metadata fields, of which only two are subject related: what kind of object is represented and the place shown on the image.

4.2.6 Features 15 and 16

Feature 15, linking subject access points from one controlled vocabulary to corresponding concepts in others, would allow interoperability and enhanced subject access enriched by related concepts from other vocabularies; however, this has not been implemented in any of the services.

Feature 16, adding, browsing and searching end user tags, would be useful to provide end-user perspectives on topics represented by museum objects, but has not been implemented by any services. The Swedish History Museum appears to have a beta version underway. At the time of this research, the tags are not social media tags but instead represent only a ranking of the most frequently used search words from their vocabularies. The users can also enter social tags, but they cannot search the museum objects. DigitaltMuseum claims that it gives logged-in users an option to add tags, but no tag clouds or related functionalities could be found.

4.2.7 Features 17 and 18

Feature 17, combining previous search formulations, is important for advanced search queries and in saving time for the end user. Six museums have their own search tools (the Swedish Museum of Natural History, Sörmland Museum, the Museum of Natural History and 3 individual eMuseumPlus museums), and four cross-search services (Alvin, Carlotta, Digitalt Museum and eMuseumPlus) use the feature, a total of 63 museums (69.2%). The feature is implemented similarly in all of them: when the user makes any subsequent search, the previous search phrase is still present in the search box and can be used again.

Feature 18, help on searching, is provided by 16 museums' in-house search tools and 6 cross-search services, a total of 80 museums (87.9%). The feature is implemented in two general ways. The most common is a dedicated web page with all the necessary instructions (the Swedish Museum of Performing Arts, Stockholm County Museum, Gustavianum, Alvin, Carlotta, DigitaltMuseum, eMuseumPlus, Europeana).

A more user-friendly approach is to provide contextual help: for example, by providing question mark icons at different places in the interface (the Swedish History Museum, Sörmland Museum, the Paleontological Museum of Uppsala University, Kulturhotell, Wikimedia Commons). Figure 17 below shows an example of the Paleontological Museum of Uppsala University.

4.2.8 Features 19, 20, 21

These three features all relate to images, which are a key characteristic of museum collections and should be offered at full scale. Feature 19, searching by image-related features (e.g. adding search criteria to retrieve results/records with or without corresponding images; image orientation, i.e. portrait or landscape, image size) is provided by six museums' in-house search tools (the Swedish Museum of Natural History, Moderna Museet and 3 individual eMuseumPlus museums), and four cross-search services (Carlotta, eMuseumPlus, Europeana and Kringla) use this feature (72 museums or 79.1%). All of these search tools except Europeana let the user choose to retrieve only results with images. Europeana does not have this option; however, it lets the user choose the image orientation (portrait or landscape) and image size.

Feature 20, searching by Content-based Image Retrieval (CBIR) methods (e.g. query by example image; query by sketch map; query by colour map) is supported by two cross-search tools: Europeana and Google Arts and Culture, that is, 60 museums (65.9%). Both services allow searching by colour. Figure 1 (in the Background section) shows an example of searching by colour in the Europeana interface.

Feature 21, encompassing functionalities encomrelated to implementation of the International Image Interoperability Framework (IIIF) open standards for enhanced user experience (e.g. deep zoom viewing, comparing, manipulating and annotating images, API interfaces), is applied in two cross-search services (4 museums or 4.4%). These support only deep zoom viewing: Google Arts and Culture after clicking the magnifier icon by the picture; and Wikimedia Commons, when choosing “Open in Media Viewer”.