Copyright © 2002 W3C® (MIT, INRIA, Keio), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply.
This note describes a project for describing & retrieving (digitized) photos with (RDF) metadata. It describes the RDF schemas, a data-entry program for quickly entering metadata for large numbers of photos, a way to serve the photos and the metadata over HTTP, and some suggestions for search methods to retrieve photos based on their descriptions.
The data-entry program has been implemented in Java, a specific Jigsaw frame has been done to retrieve the RDF from the image through HTTP. The RDF schema uses the Dublin Core schema as well as additional schemas for technical data.
We already have a demo site and there is sample source code available for download.
The system can be useful for collections of holiday snapshots as well as for more ambitious photo collections.
This document is a NOTE made available by the W3C for discussion only. Publication of this Note by W3C indicates no endorsement by W3C or the W3C Team, or any W3C Members. Please send comments to the authors.
We plan to update this note after some more experience has been gained with the system and the schemas.
A list of current W3C technical reports and publications, including Working Drafts and Notes, can be found at http://www.w3.org/TR/.
The goals of the project are partially personal, partially to promote W3C technology. The personal reasons are that we, the authors, have large numbers of photos but always have difficulty finding the exact ones that we want to show to somebody. Digitizing them and describing them in RDF should make it quicker to find the ones we are looking for at any moment.
We also think that a concrete example of an RDF schema and a working system around it can help explain the potential of metadata on the Web, especially since traditional, text-based search engines as they are used for HTML document will clearly not work for photos. Also, using metadata will automatically provide a non-visual description of the photos, hence contributing to accessibility.
The project, then, is to take the existing pieces of technology (RDF [RDF], HTTP [HTTP] and Jigsaw [Jigsaw] from W3C; JPEG [JPEG], Java [Java] from elsewhere) and provide some glue between them to produce an interesting as well as useful application.
The system comprises the following, largely independent, pieces:
Some digital cameras are already producing information about the picture, which may be read and reformatted in RDF by scripts. We will not deal with that in this version of the metadata editor.
The RDF data is expressed in three separate schemas, one of which is the Dublin Core schema. The other two deal with technical data of the photo and with subject categories. The reason for using three schemas is solely to allow each of them to be used in other projects; to the users of the data-entry program the actual RDF is completely hidden.
The data-entry program is very simple. It has been designed to enable quick entry of metadata for lots of photos, under the assumption that the photos will usually be from one or a few series. Most fields therefore show by default the value that was entered for the previous photo, and give quick access to the values entered for the last few photos. Typically, only very few fields will have to be changed from one photo to the next and the amount of typing will be minimized.
The program is written in Java, but the user interface is in fact generated at run-time directly from a machine-readable version of the schemas (currently not the RDF syntax, but a transformation of it, with equivalent information). This means that the program does not need to be changed when we change the RDF schemas.
The RDF data is stored in the JPEG file in comment blocks (blocks of type "COM", as defined by ISO DIS 10918-1). According to the JPEG standard, a comment block can contain arbitrary text. There is no way to assign a type to the text. We simply rely on the fact that RDF can easily be distinguished from plain text by heuristics. JPEG limits each comment block to 64K, but there can be as many blocks as necessary, so arbitrary amounts of text can be added. In practice, the descriptions generated by the rdfpic program are typically only a few hundred bytes long.
To serve either the RDF version or the complete image using existing browsers and tools, the best way was to use Content Negotiation. Of course, that doesn't exclude the use of other techniques, such as HTTP extensions, to be able to retrieve and store metadata in a better way.
Using Content Negotiation has two benefits: it will work right away with all text-based browsers (lynx, emacs with emacsspeak, etc.) and the output can be rendered directly by selecting, e.g., the title or the description from the RDF. Also, an RDF crawler will be able to get all the descriptions of a collection of photos to create a knowledge database, just by asking for the right MIME type.
In Jigsaw [Jigsaw], a frame has been created, to simulate two different resources under the same URI, the one of the image itself. Those two resources have their own set of HTTP values, such as ETags, Content-Length and others and the result is sent out using the classic Content Negotiation of HTTP.
The RDF can be also be fetched directly without doing Content Negotiation, by just adding the wanted MIME type after a semicoln (;) e.g.: foo.jpg;application%2Frdf+xml ("%2F" is "/", escaped for occurrence in a URL.)
Note that it is also possible to modify the RDF description using the PUT method, provided the ETag of the description is in the HTTP header of the request.
The metadata is separated into three different schemas:
All the properties are optional. The more properties are given values, the better the photo will be described and the easier it will be to find it, but leaving properties undefined doesn't make the metadata invalid.
There are no dependencies between the properties: each property can be given a value independent of whether any other property has a value. The values are also independent, except for restrictions of common sense: a photo cannot have been taken after the date on which the film was developed...
We don't use all properties defined by the Dublin Core (that is to say: the others can be added, but are ignored by our metadata editor). Here is an interpretation of the Dublin Core properties, applied to photo material. A machine-readable schema is included in appendix B. In parentheses the label that is shown in the user interface of rdfpic, if it is different from the property name.
The technical schema is defined by this RDF schema (for the formal definition, see appendix B):
The content schema contains the keywords we use in the "subject" property of the Dublin Core schema. That property should contain as many of the following keywords as are applicable. The keywords have the following meaning:
Here are some ideas for extensions to the system that we are still studying. In no particular order:
A sample server has been set up, and some pictures are
available. Any request to text version of those pictures will give you the
RDF description of the picture. I.e., an HTTP request for MIME type
image/jpeg
or image/*
returns the photo, a request
for application/rdf+xml
or application/*
returns
the metadata. Or you can just view the metadata by adding
";application%2Frdf+xml" at the end of the pictures URI. Note that the index
page has been created by a script using the RDF embedded in the pictures for
the captions and alt text.
We plan to steadily increase the number of photos that are online.
The Jigsaw extension and the JPEG related classes are a available in the Jigsaw 2.0.4 distribution, the metadata editor rdfpic is available from the Jigsaw demo site.
An apparently very similar system to ours was developed by Jane Hunter and Zhimin Zhan [HunterZhan], but for the PNG image format and with PNG's built-in keyword/value format rather than RDF to express the metadata, although they use RDF to specify the metadata schemas.
The IPTC has a list of keywords for describing photo-journalistic images. Adobe Photoshop supports a subset of them.
The proposed DIG2000 [DIG2000] file format for the JPEG2000 [JPEG2000] image compression algorithm contains an XML-based metadata block with entries for people, places, events, GPS location, camera type, etc. It allows extensions with additional entries. The draft of October 1998 doesn't use RDF.
Since we first published this note and released the rdfpic program (in March 2000), Adobe has developed a similar technology, called "Extensible Metadata Platform" [XMP]. XMP was first published around September 2001. It stores the RDF in the APP1 chunk of JPEG, rather than the COM (comment) chunk and it adds a magic string ("W5M0MpCehiHzreSzNTczkc9d") at the start of the RDF to help distinguish XMP from other things that might be present in the JPEG file. XMP embedding is also defined for other formats than JPEG, such as TIFF and PDF. Like us, Adobe recommends the Dublin Core schema and offers additional schemas for recording version history, image manipulations, etc.
The first version of the rdfpic metadata editor has been written by Thierry Kormann (formerly of Bull, France). The second version is the work of Eamon Nerbonne. Colas Nahaboo (also formerly of Bull) has given valuable advise.
Janne Saarela (of Pro-Solution, now Profium, Finland) has written the original RDF schema from which the current schemas descend and has helped with checking and reviewing the schemas. His program SiRPAC has been a great help in checking and visualizing the schemas as well as the actual metadata generated by the metadata editor.
The three schemas below (Dublin Core, technical and content) are machine-readable schemas in the syntax proposed by the RDF schemas draft [Schema].
The schema below is a minimal RDF schema for the Dublin Core. The name of
the schema is http://www.w3.org/2000/PhotoRDF/dc-1-0
, but as the
schema shows, each property is in fact a restriction of the DC property of
the same name, as found in http://purl.org/dc/elements/1.1/
The
French translations of the labels are based on those by
Anne-Marie Vercoustre.
<rdf:RDF xmlns="http://www.w3.org/TR/1999/PR-rdf-schema-19990303#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" > <rdf:Property rdf:ID="title"> <label xml:lang="en">Title</label> <label xml:lang="fr">Titre</label> <label xml:lang="nl">Titel</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/title" /> </rdf:Property> <rdf:Property rdf:ID="creator"> <label xml:lang="en">Author/creator</label> <label xml:lang="fr">Auteur/créateur</label> <label xml:lang="nl">Auteur/maker</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/creator" /> </rdf:Property> <rdf:Property rdf:ID="subject"> <label xml:lang="en">Subject</label> <label xml:lang="fr">Sujet</label> <label xml:lang="nl">Onderwerp</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/subject" /> <range rdf:resource="http://www.w3.org/2000/PhotoRDF/content-1-0#Keywords"/> </rdf:Property> <rdf:Property rdf:ID="description"> <label xml:lang="en">Description</label> <label xml:lang="fr">Description</label> <label xml:lang="nl">Beschrijving</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/description" /> </rdf:Property> <rdf:Property rdf:ID="publisher"> <label xml:lang="en">Publisher</label> <label xml:lang="fr">Éditeur</label> <label xml:lang="nl">Uitgever</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/publisher" /> </rdf:Property> <rdf:Property rdf:ID="contributor"> <label xml:lang="en">Contributor</label> <label xml:lang="fr">Contributeur</label> <label xml:lang="nl">Medewerker</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/contributor" /> </rdf:Property> <rdf:Property rdf:ID="date"> <label xml:lang="en">Date</label> <label xml:lang="fr">Date</label> <label xml:lang="nl">Date</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/date" /> <!-- use http://www.w3.org/TR/NOTE-datetime format: YYYY[-MM[-DD[Thh:mm[:ss[.sTZD]]]]] example: 1999-10-01T17:53 if TZD is omitted the timezone is UTC --> </rdf:Property> <rdf:Property rdf:ID="type"> <label xml:lang="en">Resource type</label> <label xml:lang="fr">Type de ressource</label> <label xml:lang="en">Categorie</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/type" /> <!-- always "image in PhotoRDF --> </rdf:Property> <rdf:Property rdf:ID="format"> <label xml:lang="en">Format</label> <label xml:lang="fr">Format</label> <label xml:lang="nl">Formaat</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/format" /> <!-- always "image/jpeg in PhotoRDF --> </rdf:Property> <rdf:Property rdf:ID="identifier"> <label xml:lang="en">Number</label> <label xml:lang="fr">Numéro</label> <label xml:lang="nl">Nummer</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/identifier" /> </rdf:Property> <rdf:Property rdf:ID="source"> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/source" /> <!-- not used in PhotoRDF --> </rdf:Property> <rdf:Property rdf:ID="language"> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/language" /> <!-- not used in PhotoRDF --> </rdf:Property> <rdf:Property rdf:ID="relation"> <label xml:lang="en">Series</label> <label xml:lang="fr">Série</label> <label xml:lang="nl">Serie</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/relation" /> </rdf:Property> <rdf:Property rdf:ID="coverage"> <label xml:lang="en">Location</label> <label xml:lang="fr">Endroit</label> <label xml:lang="nl">Plaats</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/coverage" /> <!-- restricted to spatial coverage in PhotoRDF --> </rdf:Property> <rdf:Property rdf:ID="rights"> <label xml:lang="en">Rights</label> <label xml:lang="fr">Droits</label> <label xml:lang="nl">Rechten</label> <subPropertyOf rdf:resource="http://purl.org/dc/elements/1.1/rights" /> </rdf:Property> </rdf:RDF>
See the description above for detailed
explanations of each of the properties. The name of this schema is
http://www.w3.org/2000/PhotoRDF/technical-1-0#
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns="http://www.w3.org/2000/01/rdf-schema#"> <Class rdf:ID="Technical-data"> <comment xml:lang="en">A class that represents technical data about a photo</comment> <comment xml:lang="fr">Une classe qui réprésente les données techniques sur une photo</comment> <comment xml:lang="nl">Een class die de technische gegevens van een foto representeert.</comment> </Class> <rdf:Property rdf:ID="camera"> <label xml:lang="en">Camera</label> <label xml:lang="fr">Appareil photo</label> <label xml:lang="nl">Camera</label> <comment xml:lang="en">Brand and type of camera</comment> <comment xml:lang="fr">Marque et type de appareil photo</comment> <comment xml:lang="nl">Cameramerk en -type</comment> <domain rdf:resource="#Technical-data"/> </rdf:Property> <rdf:Property rdf:ID="film"> <label xml:lang="en">Film</label> <label xml:lang="fr">Pellicule</label> <label xml:lang="nl">Film</label> <comment xml:lang="en">Brand and type of film</comment> <comment xml:lang="fr">Marque et type de pellicule</comment> <comment xml:lang="nl">Filmmerk en -type</comment> <domain rdf:resource="#Technical-data"/> </rdf:Property> <rdf:Property rdf:ID="lens"> <label xml:lang="en">Lens</label> <label xml:lang="fr">Objectif</label> <label xml:lang="nl">Lens</label> <comment xml:lang="en">Brand and type of lens.</comment> <comment xml:lang="fr">Marque et type d'objectif.</comment> <comment xml:lang="nl">Merk en type van de lens.</comment> <domain rdf:resource="#Technical-data"/> </rdf:Property> <rdf:Property rdf:ID="devel-date"> <label xml:lang="en">Development date</label> <label xml:lang="fr">Date de développement</label> <label xml:lang="nl">Ontwikkeldatum</label> <comment xml:lang="en">Date on which the film was developed.</comment> <comment xml:lang="fr">Date à laquelle la pellicule a été developpée.</comment> <comment xml:lang="nl">Datum waarop de film is ontwikkeld.</comment> <domain rdf:resource="#Technical-data"/> <!-- use http://www.w3.org/TR/NOTE-datetime format: YYYY[-MM[-DD[Thh:mm[:ss[.sTZD]]]]] example: 1999-10-01T17:53 if TZD is omitted the timezone is UTC --> </rdf:Property> <!-- [more?] --> </rdf:RDF>
We left out the human-readable comments; see the
descriptions of the keywords above. The name of this schema is:
http://www.w3.org/2000/PhotoRDF/content-1-0#
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns="http://www.w3.org/2000/01/rdf-schema#" xmlns:content=""> <!-- "" is the same as "http://www.w3.org/2000/PhotoRDF/content-1-0#" --> <Class rdf:ID="Keywords"> <comment xml:lang="en">An enumeration of keywords to describe the subject of photos.</comment> <comment xml:lang="fr">Une énumération de mots-clef pour décrire le sujet d'une photo.</comment> <comment xml:lang="nl">Een opsomming van sleutelwoorden om het onderwerp van foto's te beschrijven.</comment> </Class> <content:Keywords rdf:ID="Portrait"> <label xml:lang="en">Portrait</label> <label xml:lang="fr">Portrait</label> <label xml:lang="nl">Portret</label> </content:Keywords> <content:Keywords rdf:ID="Group-portrait"> <label xml:lang="en">Group portrait</label> <label xml:lang="fr">Portrait de groupe</label> <label xml:lang="nl">Groepsportret</label> </content:Keywords> <content:Keywords rdf:ID="Landscape"> <label xml:lang="en">Landscape</label> <label xml:lang="fr">Paysage</label> <label xml:lang="nl">Landschap</label> </content:Keywords> <content:Keywords rdf:ID="Baby"> <label xml:lang="en">Baby</label> <label xml:lang="fr">Bébé</label> <label xml:lang="nl">Baby</label> </content:Keywords> <content:Keywords rdf:ID="Architecture"> <label xml:lang="en">Architecture</label> <label xml:lang="fr">Architecture</label> <label xml:lang="nl">Architectuur</label> </content:Keywords> <content:Keywords rdf:ID="Wedding"> <label xml:lang="en">Wedding</label> <label xml:lang="fr">Mariage</label> <label xml:lang="nl">Trouwerij</label> </content:Keywords> <content:Keywords rdf:ID="Macro"> <label xml:lang="en">Macro</label> <label xml:lang="fr">Macro</label> <label xml:lang="nl">Macro</label> </content:Keywords> <content:Keywords rdf:ID="Graphic"> <label xml:lang="en">Graphic</label> <label xml:lang="fr">Graphique[?]</label> <label xml:lang="nl">Grafisch</label> </content:Keywords> <content:Keywords rdf:ID="Panorama"> <label xml:lang="en">Panorama</label> <label xml:lang="fr">Panorama</label> <label xml:lang="nl">Panorama</label> </content:Keywords> <content:Keywords rdf:ID="Animal"> <label xml:lang="en">Animal</label> <label xml:lang="fr">Animal</label> <label xml:lang="nl">Dier</label> </content:Keywords> </rdf:RDF>
This is an example of the metadata in RDF format that is generated by rdfpic, and subsequently served by Jigsaw.
<?xml version='1.0' encoding='ISO-8859-1'?> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/TR/1999/PR-rdf-schema-19990303#" xmlns:s0="http://www.w3.org/2000/PhotoRDF/dc-1-0#" xmlns:s1="http://www.w3.org/2000/PhotoRDF/technical-1-0#" xmlns:s2="http://sophia.inria.fr/~enerbonn/rdfpiclang#"> <rdf:Description rdf:about=""> <s0:creator>Bert Bos</s0:creator> <s0:relation>Marian in the Tarn</s0:relation> <s0:rights>Bert Bos</s0:rights> <s0:type>image</s0:type> <s0:identifier>990621</s0:identifier> <s0:coverage>Montredon-Labessonié (Tarn)</s0:coverage> <s0:date>1999-06-26</s0:date> <s1:camera>Canon Eos 5</s1:camera> <s2:xmllang>en</s2:xmllang> <s0:title>Marian with sheep</s0:title> <s0:subject>Landscape, Animal</s0:subject> <s0:publisher>Bert Bos</s0:publisher> <s0:description>Marian brings the sheep to the field in the morning. The lamb she carries was born that night.</s0:description> <s0:format>image/jpeg</s0:format> </rdf:Description> </rdf:RDF>