Note on metadata profiles management

 

Version

Comments

1

Author : Jean Christophe Desconnets (jcd@teledetection.fr) ; Stéphane Clerc (clerc@teledetection.fr )

Created at : 16/11/2006 17:04:00

Modified at : 20/02/2008 16:05:00

Translation : Kim Agrawal (kim@auromail.net)

Comments on English version : english screen shot entry

 

  

Some basic ideas before we start….. 2

Concept of profile or metadata profile. 2

Concept of resources. 4

Concept of metadata sheets or references. 6

Concept of data-entry levels of a metadata sheet 6

Metadata profiles manager. 7

What does it do and who should use it?. 7

Limitations of the current version. 8

Operating procedure. 9

Accessing the profiles manager module. 14

Adding a metadata profile. 14

Modifying a metadata profile. 22

Deleting a metadata profile. 25

 

Some basic ideas before we start…

A profile, or adaptation, is a document or schema (in the sense of a data structure) that specifies the implementation options of a standard for a particular purpose. In essence, a profile does not contradict the standard to which it refers and does not introduce, in principle, new concepts. Rather, it describes the standard or a sub-set of it so that it can be implemented and used in a particular context. However, elements that do not exist in the standard (extended elements) can be included in it. These description elements will complement the standard and will be useful in the specific context in which the profile is going to be used. In addition, a profile of a standard allows an international standard to be adapted culturally or linguistically for a particular national or regional context.

A community can thus define profiles for particular types of data sets. For example, a profile for matrix or ‘raster’ data sets will retain only those elements specific to this data type. A profile can also manage certain specifics or rules that an organization may want to apply to metadata elements. A profile, for example, could specify which elements are mandatory and which are optional in a metadata sheet.

 


 

MDweb includes 9 profiles as standard. They correspond to 9 data types:

  • Types of data series :

o    Geographical database or geodatabase

o    Temporal databases

o    Digital map

Types of data series :

o    hardcopy map

o    Vector layer

o    image – aerial picture

o    Text document

o    Spreadsheet data

o    Bibliographical references

In principle, metadata standards, the international standard in particular, apply to digital data but they can also be applied to analogue documents such as maps, plans, aerial photographs, etc. In such cases, the documentation of the data and its cataloging always reference the actual document. Moreover, data sets of this type usually consist of a clearly identifiable collection of documents. On the other hand, for digital data, the definition of what is data, or a data set, is more difficult and often depends on the institutional or technological context of the organization that produced the data. In general, digital data can be broken up into a hierarchy going from data attribute to entity type to data set to, finally, data series. This perspective of data can be more simply described with the general term ‘resource’. It covers all the concepts associated with the data hierarchy shown in the figure.

To illustrate this concept, we have chosen as example the land use maps of a territory, in this case that of Oued Mird (Morocco). This resource, of type ‘digital map’, can be broken up into the hierarchy of resources mentioned above in a perspective of UML formalism. If we consider the highest level, we can speak of a data series. This is represented here by the collection of maps on the same theme but produced during different observation periods, those for land use in the 1990s, those for land use in the 2000 decade, etc. At the data set level, we will consider just one item from this collection, for example, the land use map of the 1990s. The next lower level, entity type, will correspond to all the thematic layers that make up the land use map of the 1990s. In our example, we have selected the ‘polygon’ layer of land-use classes. Other layers, such as the village layer, can be part of the map. Finally, the most basic level, or attribute type, is the set of properties of the ‘polygon’ layer. An example of this attribute type is given by the attribute ‘percentage of ligneous cover’.

The levels handled by MDweb are limited to:

      • data series
      • data sets

Definitions

Data series: A collection of distinct data sets related to each other by common characteristics such as their mode of acquisition or processing (satellite images), their spatial extent, the type of their contents, for example, a data series is synonymous with a data collection. This denomination is used in MDweb for the data types: ‘digital map’, ‘geodatabase’ and ‘Temporal database’.

Data set: Set of related data, unmistakeably identifiable as connected to each other by common characteristics such as their mode of acquisition or processing, their spatial extent, etc. A data set can be considered as a small set of data or a sub-set of it. This denomination is used in MDweb for the data types: ‘hardcopy map’, ‘vector layer, ‘image – aerial photo’, ‘text document’, ‘Spreadsheet data’ and ‘bibliographical reference’.

Hierarchy between data series and data sets

MDweb establishes a hierarchy between data types using the concept of parent and child profiles (see Concept of profile or metadata profile).

In the standard version, this is the hierarchy:

 

 

 

In this document, the concepts of the metadata sheet and of the reference are used in the same way. They both apply to the same object. A metadata sheet or reference is defined as a set of metadata elements filled in by a user to describe a data collection or data set or, more generally, a resource.

The concept of a metadata sheet relates to the structure and nature of the metadata elements that it consists of, with these elements originating from the ISO 19115 standard.

The concept of a reference additionally relates to a perspective of metadata as an item of a data catalog managed by MDweb.

The data-entry level relates to the number of elements (and their characteristics) used for describing a resource. It corresponds to different levels of metadata usage. In fact, the information required to describe a resource depends on the purpose of the metadata usage. For example, for purposes of searching and locating resources, information that is less detailed and less complete will suffice as compared to for documentation purposes, which will need greater detail and completeness because resources will need to be distributed and transferred. Thus, for cataloging of resources, which is the basis of searches for them, simplified metadata could be sufficient.

These different contexts or levels of metadata usage can lead to the definition of several metadata-detail levels. The international standard defines two levels of details or conformity. The first conformity level or ‘basic’ level corresponds to the purposes of resource cataloging. For this, it proposes a set of mandatory elements or ‘metadata core profile’ that consists of elements necessary to identify the resource and to provide a summary of its contents. It can only be used for cataloging services and as a basis for metadata services designed for locating resources. A second conformity level or ‘complete’ level includes metadata elements necessary to fully document a resource. This conformity level defines metadata elements necessary to identify, evaluate, extract, use and manage geographic resources.

On the basis of the international standard’s definitions, we have identified three levels of detail in the profiles for the metadata:

ü        a basic level,

ü        an extended level,

ü        and a complete level.

The basic level is based on the minimum metadata elements specified in the standard.

The extended level is based on the basic level and additionally includes those metadata elements that would allow the exchange and transfer of the resource and the accurate description of the resource’s origins (source data and processes used). This latter requirement is essential for the reuse of a resource for scientific purposes. For data types offered in the standard MDweb version, the extended and the complete levels are one and the same.

Metadata profiles manager

 

 

The profiles manager is the core of the MDweb tool because it allows the user to define the structures and descriptive elements on which the entry forms and the search engine are based.

Because of its powerful nature, its use should be limited to the application’s administrator having an in-depth knowledge of metadata and associated standards. Ill-advised changes in these structures and their modifications in the MDweb database can lead to serious malfunctioning of the tool.

 

General features of the profiles manager

The administrator can use the profiles manager to define (add) a new profile, modify the structure or properties of existing ones or delete them.

Concepts of metadata profiles and of the hierarchy between profiles are provided in this document’s introduction.

 

The profiles manager is currently in its initial version. Several limitations exist: 

-      Choice of the standard: the ISO 19115 (TC/211) standard in its FDIS 2003 version (http://jc.desconnets.free.fr/mdweb/docs/FDIS-19115.pdf) is the only standard we can currently use to construct a profile. All the sections of this standard are not described completely in MDweb. This is the list of the sections with the degree of completeness:

·         MD_Identification (only the MD_DataIdentification class)

·         MD_Constraints

·         DQ_DataQuality

·         MD_MaintenanceInformation

·         MD_SpatialRepresentation

·         MD_ReferenceSystem

·         MD_ContentInformation

·         MD_PortrayalCatalogueReference  

·         MD_Distribution

·         MD_MetadataExtensionInformation

·         MD_ApplicationSchemaInformation

-      Definition or modification of predefined lists attached to the elements: the capability of modifying or defining lists of predefined values associated with the metadata elements has not yet been implemented. So additions or modifications have to be made directly in the MDweb database in the elmnt­_mtd tables (list_short_name field used to define a predefined list for an element), code_list (definition of a new list of values) and the elmnt_code_list table (definition of elements). An added element is translated into other languages by the insertion of a new label in the label_stand table.

-      Operations for the definition or modification of a standard have not been implemented in this version. If you want to add a standard or complement the existing one, you have to insert elements in the elmnt_mtd and est_inclus tables and translate labels of the added elements in the label_stand table.

-      The requirement of defining a minimal sub-set (metadata core) is not enforced. You have to study the ISO 19115 standard and ensure that the profile you build respects the standard’s core if you want to be compatible with it (this sub-set is discussed in the following section).

 

Before running the profiles manager

The definition and editing of a profile are crucial stages that require you to select descriptive elements that are relevant to the data set you want to catalog. In collaboration with your project partners, you have to carefully arrive at a correct and functional definition of your profiles. Towards this end, we recommend an operating procedure which we break up into 5 steps:

  1. Studying and choosing your profile’s structure
  2. Defining element properties
  3. Details on the definition of element properties
  4. Finalizing the creation or modification of the profile
  5. Testing the created profile

 

1.    Studying and choosing your profile’s structure

 

Before even using MDweb, it is necessary to study the cataloging you want to do for your project and to specify how you want to describe your data sets. Some questions that help you do so are: 

What information will the users of the catalog (those who will search my data sets) need to locate the data set? Which elements will they need to judge the relevance to their requirements?

Do I want my application to allow users to access the data sets?

What is the number of metadata elements that the users will be ready to enter? 10, 20, 100, 200?

Using these questions, and others like them, as a starting point you can narrow down on descriptors required for your project, those that will be optional, etc. To define the structure of your profile (the different sections) and the elements therein (the fields in which data will be entered), it is necessary to match your requirements with the elements proposed in the ISO 19115 standard (http://jc.desconnets.free.fr/mdweb/docs/FDIS-19115.pdf). This will lead you to a first prototype profile.

You can also rely on the profiles that are included as standard with MDweb. There are nine of them and you can view their structures from the ‘Utilities’ section in the MDweb home page (Structure of profiles used) http://localhost/mdweb-demo15/test/test_gabarit.php

The essential elements (metadata core) of ISO 19115

The standard defines a sub-set of mandatory elements for referencing a data set. MDweb does not verify the conformity of your profile to this sub-set. These are the elements that are part of this sub-set:

 

Element label

ISO element name

MDweb’s handling

Record identifier

mdFileID

automatic

Metadata language

mdLang

automatic

Metadata character set

mdChar

automatic

Organization name

rpOrgName

manual entry

Organization role

role

manual entry

Record creation date

mdDateSt

assisted

Metadata standard name

mdStanName

automatic

Metadata standard version

mdStanVer

automatic

Abstract of contents of the set

idAbs

manual entry

Title of the data set

resTitle

manual entry

Keywords

keyword

assisted

Data set creation date

refDate

assisted

Western-most extent

westBL

assisted

Eastern-most extent

eastBL

assisted

Northern-most extent

northBL

assisted

Southern-most extent

southBL

assisted

 

Once you have defined the structure of your profile, you can start its construction using the Adding a profile module by using an existing profile (see the ‘Adding a metadata profile’ section).

At this stage, you have defined the structure of your profiles, i.e., the sections and the elements that the users will use to describe their data sets. But the construction of the profile is not yet complete. In fact, before MDweb can construct data-entry forms, store metadata, and provide search results, it is necessary to specify how each element has to be processed in MDweb (required field, level of entry, type of the field of the form, etc.).

 

 

2. Defining element properties

The profiles manager automates to some extent the definition of profiles. This has to be done after the construction of the profile’s structure (addition of elements). To automatize the processing of a profile’s elements in the data-entry forms and their display in the detailed records available during consultation, some properties have to be defined for each element. We provide here the list of these properties as well as their default values, if any.

Properties differ depending on item type (section, class or element). The profiles manager allows you to define four types of properties:

·         Attributes that specify an element of the standard

·         General properties that extend the standard’s properties

·         Properties for the construction of the data-entry forms within MDweb

·         Translation of the label of the element in the languages used by MDweb

 

An element’s attributes

Some elements of the standard can possess attributes that specify an element’s characteristics (for example, an attribute ‘measurement unit’ would allow the specification of the unit in which the element would be filled in). When an element has one or more attributes, the profiles manager proposes a default value per profile and per attribute (free-form value or to be selected from a predefined list).

 

General properties of profile elements

Property name

Definition

Range

Mandatory

Used to specify whether a value for the element is mandatory or optional

List of values:

‘true’, ‘false’

Entry level

Used to specify the data-entry level of an element. There are three levels. This property is used to offer the user a form with minimal elements (basic level) and to complement the entry forms with additional elements by attributing a higher entry level to them (extended, complete).

List of values:

‘basic’, ‘extended’, ‘complete’

Number of occurrences

Used to specify the number of occurrences of an element in the form. This property applies both to headings and elements. Ensure, however, that the value chosen conforms to the standard.

Integer:

Between 1 and 62 (both inclusive)

Default value

Specifies an element’s default value. This allows some elements to have a predefined constant value irrespective of the data set being described.

 

Appearance in the entry form

Specifies the form in which the element will appear. To reduce as much as possible the task of entering elements, MDweb offers different processing for different elements. They can be entered:

·    in the main entry form accessible via the ‘Create’ or ‘Modify’ modules

·    in the contacts directory only for the headings providing contact address information (CI_ResponsibleParty class)

·    by automatic insertion for those elements with constant default values

·    predefined values manager Preferences module > Predefined values which allows several sets of default values per profile. These sets of values are then used to fill in a record during its creation. In this way, these elements do not appear in the entry forms. 

List of values:

‘Entry forms only’,

‘automatic insertion (default value)’,

‘Management of predefined form values’,

‘contacts directory’

 

Contact type

This property is valid only for contact headings and elements (CI_ResponsibleParty class). It is used to specify the section (Metadata, Data set or Distribution) to which this element will be attributed.

List of values:

‘Metadata’,

‘Data set’,

‘Distribution’

 

General properties of elements of type section or heading of the profiles

This table complements the previous one by providing properties specific to sections and headings.

Property name

Definition

Range

Appearance in the entry form

Additional values for this property are available for elements of type section and heading. They define the appearance of the label in the section or the heading and not the elements actually contained therein:

·  Invisible: the label will not appear in any of the forms

· Visible in all forms 

· Entry form and predefined values

· Entry form and contacts directory

· Contacts directory and management of predefined values

List of values (complements those from the previous table):

‘Invisible’,

‘Visible in all forms’, 

‘Entry form and predefined values’, 

‘Entry form and contacts directory’,

‘Contacts directory and management of predefined values’

 

 

 

Tips:

Lighten your forms: When the default value is constant, you can select as property value ‘Appearance in the entry form’: Automatic insertion. Thus, this element’s entry field will not appear in any form.

Lighten your forms (2): by defining an element with the value ‘Management of predefined form values’ for its ‘Appearance in the entry form’ property.

 

Properties for constructing entry forms specific to MDweb

These properties apply only to elements of description to which we can assign values. They are used to define properties that will allow us to construct entry forms (type of entry field, data-entry control, type of value, number of characters, etc.).

Property name

Definition

Range

Field type

Defines the type of entry field that will appear in the form. Six values are allowed:

- textarea: a text area (number of lines and number of columns) for entering text of more than 200 characters

- text: a text field in which any text can be entered (text, integer, real, date) except those that are of the following types.

- url: text field for entering a url of type http://www.domain.com/. By thus using this field, it will be treated as a link (in the html sense) in the detailed sheet shown during consultation.

- mail: text field for entering an e-mail address. It will be treated as a mailto (in the html sense) in the detailed sheet shown during consultation.

- list: when a list of predefined values of the standard is attached to the element that you are defining. In this case, the list of values is proposed to you.

- read only: text field in which nothing can be entered.

List of values:

‘textarea’

‘text’

‘url’

‘mail’

‘list’ 

‘read only’

Field width

Defines the width of a text or textarea field

List of values:

5 to 60

Field height

Only for a textarea field, specifies the number of lines

List of values:

1 to 10

Name of the javascript control

6 javascript controls are available for being attached to the defined field:

- maximum character length controls the number of characters entered in the field. This control should be accompanied by the value of the number of characters in the ‘Number of characters’ property.

- test for a real field is used to verify whether the value entered is truly a real.

- test for an integer field, ditto for an integer

- test for a mail field, ditto for a field of type ‘mail’ (see ‘Field type’ property)

- test for a url field, ditto for a field of type ‘url’ (see ‘Field type’ property)

- date format verification, ditto for a text field in which we want to enter a date

List of values:

‘maximum character length’,

‘test for a real field’,

‘test for an integer field’,

‘test for a mail field’,

‘test for a url field’,

‘date format verification’

Events associated with the javascript control

User event that triggers the javascript control defined in the ‘Name of the javascript control’ property. Two values are available:

- onKeyPress: event triggered on entry of each character in a field

- onChange: event triggered on exit from a field after having modified its value

List of values:

‘onKeyPress’

‘onChange

Number of characters

Specifies the maximum number of characters the user can enter into a field of type text. A value is mandatory for configuring the ‘maximum character length’ javascript control

List of values:

25, 30, 50, 100, 200, 500

 

 

The translation of the label of the element in the languages used by MDweb

 

The profiles manager allows you to translate or modify labels of metadata elements in all the languages used by MDweb (English, French, Portuguese). This can be done during the definition of the properties of the profile’s elements.

 

3. Finalizing the creation or modification of a profile

The last stage is the finalization of the created or modified profile. It should only be undertaken when all the elements have been chosen and their properties defined in extenso.

Use the ‘Modify your profile’ button (frame at the bottom of the manager) to finalize your profile. The conformity of the defined properties will be verified and a position index calculated for each profile element. This index is especially useful for managing the multi-occurrence of headings and elements.

4. Test of the created profile using the entry form

Apart from the profiles manager, it is essential to test the structure and each element’s properties by creating test sheets. This will allow you to verify the operational correctness of your profile – and to take remedial measures, if necessary, by adding or modifying some field properties – before starting using the profile for entering actual metadata sheets.

In fact, when one or metadata sheets are created using a given profile, it becomes impossible to modify the profile’s structure without first deleting those sheets. However, you can always modify element properties. In any case, a comprehensive verification is strongly advised before any large scale use of the profile for entering sheets.


 

 

Access to the profiles manager module is available only to you if you are logged in as an administrator. Go to the MDweb Administration module and click on ‘Profiles’ on the Administrator menu bar.

 

 

Three major functions are available in the Profiles manager:

·         Add: to create a new profile using an existing one or from a standard

·         Modify: to modify a profile’s structure or the properties of its elements

·         Delete: to delete an existing profile

 

 

 

1. Go to the Adding a profile page from the Administration module

 

 

2. Define the new profile by filling in the form that opens.

 


A maximum of 5 fields will have to be filled in:

·         Name of the metadata standard: The metadata standard you want to use to create your new profile. In the current version, only the ISO 19115 standard can be used (FDIS 2003 version).

 

  • Profile name: By default, you have to provide the name in English of the profile you want to create with a maximum length of 50 characters (no special characters). It should be self-explanatory. You can translate it subsequently into your language by using the Labels > Add module.

 

·         Profile code: The profile code is the profile’s internal name. It should be short, explanatory and without spaces with a maximum length of 20 characters. For example, ‘vector_layer’ for the ‘Data vector’ profile.

 

  • Parent profile: MDweb allows you to define a profile as a data collection or a data set. This permits you to create a hierarchy between different profiles (see Concept of resources).
  • If you do not attach a parent profile to your new profile, it will be considered a data set.

 

  • Child profile: ditto as for the previous field but for creating a profile that will be considered a data collection if you define a child profile.
  • Create your profile from an existing profile: allows you to create your new profile based on one of the standard MDweb profiles. If you want to create a profile that is similar to an existing one, this option gains you substantial time, especially for defining properties.

 

Help on fields of the form

 

In the same way as for entry forms, each field has text help available for it. Move the mouse over the  icon to view it.

 

 


 

Managing the hierarchy between profiles

 

     Case of a new profile, child of an existing profile

 

     When creating a new profile, you can define it as a data set (see ‘Concept of resources’ section) and attach to it an existing profile as ‘parent profile’. See screen example below.

 

 

     Case of a new profile, parent of an existing profile

 

     In the opposite case to the one above, you can define a profile as a data collection (see ‘Concept of resources’ section) and attach to it an existing profile as ‘child profile’. See screen example below.

 

 


 

The layout of the profiles manager

 

The profiles manager page consists of 3 frames:

 

  • Frame on the left to display the profile as a tree structure
  • Frame on the right for modifying element properties